As you may have noticed, since last year, we (Google’s CUDA compiler team) have contributed quite a lot to the effort of optimizing LLVM for CUDA programs. I think it’s worthwhile to write some docs to wrap them up for two reasons.
- Whoever wants to understand or work on these optimizations has some detailed docs instead of just source code to refer to.
- RFC on how to improve these optimizations so that other targets can benefit from them as well. They are currently mostly restricted to the NVPTX backend, but I see many potentials to generalize them.
So, I started from this overdue design doc on the straight-line scalar optimizations. I will send out more docs on other optimizations later. Please feel free to comment.
Out of curiosity, is there any plan to make the NVPTX-originated passes (separateconstantoffsetfromgep, slsr, naryreassociate) more generic? They seem very specialized for the nVidia GPU addressing modes despite the generic names, and in my tests tend to pessimize our target more often than not for that reason.
It’d be really nice to have something more generic, and I might look into helping with that sort of thing in the future if it becomes important for us.
We certainly would love to generalize them as long as the performance doesn’t suffer in general. If you have specific use cases that are regressed due to these optimizations, I am more than happy to take a look.
It’s not a big issue since they’re opt-in passes, naturally, but if we did want to use them, the main problem is that our addressing modes are of the form:
X + sext(Y) * scale
for i64 X and i32 Y and various (power of 2) scales.
A lot of these passes tend to move around the sexts in ways that prevent them from being folded into addressing modes. Another thing I’ve noticed (in separateGEPFromConstantOffset) ends up turning (X + sext(Y + C)) into (X + sext(Y)) + C. Unfortunately, unless this saves operations due to CSE, it ends up with us having:
tmp = X + sext(Y)
load with base = X and offset = C
tmp = Y + C
load with base = X and offset = tmp
Since 64-bit adds are more expensive than 32-bit adds on our GPU, this ends up being a pessimization.
These are just a few examples I spotted; it’s probably a more general LLVM problem as a whole that passes which manipulate GEPs and induction variables tend to be oblivious of addressing modes, or tuned towards a particular sort of addressing mode.
To add to Jingyue's answer - the reason these passes are not more generic
is very pragmatic - we've just optimized them for the NVIDIA targets we
care about and can run extensive benchmarking on. There's absolutely no
problem generalizing them if someone's interested - in fact, we'd be happy
to see that happen. This is what open-source is for IIRC some of the
optimization work was already generalized by the AMD backend folks, and
more can be done for sure. While PTX has its specific characteristics, many
of the general issues with GPU-oriented optimizations are common to other
GPU architectures and can be generalized in IR level passes.
That’s a very interesting case. I think add more addressing mode checks to PAR (SeparateConstOffset) may work for your cases. Something like if “a[i]” can be folded, don’t perform this optimization at all. The CSE benefit can often be achieved by global reassociation (though it needs to be extended to handle more binary operators).
The canonical form for a loop in LLVM IR is to have a single induction variable and derive all other variables from it. This is good for mid-level optimisers, but often ends up generating quite sub-optimal code on at least some targets. For example, turning an add-immediate of another variable in each loop iteration into a multiply (one instruction, multi-cycle latency) of the loop induction variable a constant (which takes one instruction to materialise on MIPS, as there’s no multiply-by-immediate instruction).
If we had some documentation of precisely *what* the canonical forms are for each phase in the optimisation pipeline, then it would be easier to understand these effects and know when we need some de-canonicalise pass before codegen.
Thank you for sharing this. It made for extremely interesting reading.