Unified Host–Device Compilation in ClangIR (CIR): Enabling Cross-Boundary Analysis and Optimization
Modern heterogeneous applications rely heavily on GPU offloading, yet today’s compilation pipelines strictly separate host and device compilation. This separation prevents whole-program reasoning across host–device boundaries and limits optimization opportunities such as kernel specialization, launch-bound inference, and cross-boundary constant propagation.
This project proposes extending ClangIR (CIR) with a combine pipeline that merges host and device CIR modules into a unified representation, enables joint analysis and transformation, and then safely re-splits the code for final target-specific lowering. The work builds directly on an existing work-in-progress implementation and aims to upstream a robust, tested, and documented workflow into the ClangIR incubator.
Background and Motivation
The Clang compiler today emits separate compilation artifacts for host and device code (e.g., CUDA/HIP). While this model simplifies backend handling, it introduces fundamental limitations:
No cross-boundary analysis: The compiler cannot reason about kernel launches, argument values, or launch configuration at the call site.
Missed optimization opportunities: Kernel specialization, dead code elimination, and launch-bound inference require joint host–device visibility.
Duplication of logic: Host and device pipelines evolve independently, increasing maintenance cost.
LLVM’s MLIR-based CIR provides a unique opportunity to address this limitation. Because both host and device code are represented in a structured, high-level IR, it becomes feasible to temporarily merge them, perform joint analyses and transformations, and then re-split them for conventional lowering.
This project directly explores that opportunity.
Related Work and Current State
ClangIR is actively being upstreamed and already supports emitting CIR from Clang’s AST and lowering to LLVM IR. Initial experiments have demonstrated that host and device CIR modules can be merged in a controlled way.
A work-in-progress implementation already exists which is capable of merging but there is still work in progress in incorporating the merging with the clang driver
This sounds really interesting and I’d love to learn more. Can you expand a bit on the second point, do you mean build some tool that stitches device and host code into a single unit?
Also if you had pointers on resources for learning more about CIR and the optimization passes you described (specifically, launch-bound inference) that would be super awesome!
When using original clang and compiling source code that includes GPU code, clang will compile the same file twice, once for the host (CPU) and once for the device (GPU). This separation will exist throughout the compilation process. The idea is to combine both the host code and the device code in a translation unit and optimize them together.
I don’t have any intelligent advice or feedback for this project, so I can only offer words of encouragement. My personal experience with offloading is very limited, but I’ve been hearing about this problem of cross host-device optimization for maybe 10 years, and I’m excited to see a real upstream project tackling it. Good luck!
Hi Konstantinos,
I am very interested in this GSoC project, and I have spent the past few days studying and researching it.
Based on my understanding, I believe there are currently two major technical challenges:
Multi-architecture support: The project’s backend currently defaults to a single architecture. However, in practice, fat binaries (multi-architecture) are very common, which directly hinders the optimization of the Pass pipeline. My current proposed approach is to lazily “instantiate” the corresponding calling code. By treating calls to global code similarly to a template instantiation process, we could embed the corresponding instantiated code directly into the respective modules.
Type erasure in global calls: Calls to global functions are currently type-erased. We may need to provide a StubAnalysis infrastructure to extract type information, allowing the pass pipeline to operate as if the types were not erased.
If you find these ideas interesting, I would be more than happy to discuss them further with you!
Hello, this sounds like a really cool project. Quick question, when you mention kernel specialization as one of the optimizations, what does that actually look like? Does it mean generating different kernel versions based on host side constants or launch configs?
This is a great project idea. I would love to work on this. I already have relevant experience contributing to llvm-project in areas of missed-optimizations, crashes. I have 6 merged and 4 to be merged PR’s.
I will research more on this topic and draft a proposal.
Hello, I have been studying how Clang currently handles cuda host-device separation and trying to understand where a unified CIR representation would fit in the existing driver pipeline.
To better understand the current separation, I implemented a small LLVM pass that analyzes how CUDA kernel launches are represented at the IR level.
From what I understand, the driver orchestrates two separate compilation flows that remain isolated until later stages. My question is about the intended merge point in the cir-based approach:
Is the idea to merge host and device CIR modules before target-specific lowering passes begin, enabling cross-boundary analysis at the CIR level? or would the merge happen after certain passes have already been applied independently?
I’m particularly interested in how this would interact with the driver’s phase orchestration, since the separation today seems deeply embedded in how CUDA compilation is structured.
I’m late for the discussion, but I’d be happy to answer questions specific to CUDA compilation in Clang. @yxsamliu may be able to help from the HIP prospective.
Overall, it’s a very interesting idea, and it would address the existing information gap, where we could optimize better, if optimizations could take into account both the host and the GPU sides.
Can you elaborate on how merging may work in this context? Each front-end subcompilation will produce slightly (or not so slightly) different output depending on the target (mostly due to _CUDA_GPU_ARCH_ macro), subsequent conditional preprocessing, and whatever C++ templates end up expanding into.
While I can see that the proposed approach is doable for simple examples, probably feasible for a lot of existing code, but my gut feeling that there will be corner cases the merged representation may have trouble with.
The WIP PR does not seem to have any tests that would illustrate how merging works and what it can/can’t do, so I don’t know whether it’s solvable or if it’s going to be a showstopper, like _CUDA_GPU_ARCH_ was for the single-front-end- compilation idea.
This would be useful in order to provide support for the SYCL 2020 Single-source Single Compiler Pass (SSCP) implementation option. See SYCL 2020 section 3.12, “Implementation options” for more details. There are no distinct host vs device predefined macros with that implementation approach.
Though I have no experience with it, my understanding is that the nvc++ compiler supports CUDA with a SSCP implementation. I don’t know the details of how it works, but it presumably doesn’t support distinct predefined macros for host vs device compilation. This proposal would presumably enable Clang to support a similar compilation model for CUDA.
Partially. Specifically it does not support the __CUDA_ARCH__, which means that one will have to be careful using standard CUDA headers (I’m not sure if they have special case for handling nvc++) and we’ll probably need to add extra magic into clang’s CUDA header wrappers:
If we constrain supported CUDA dialect to non-divergent subset, and, possibly augment it by GPU-specific target attributes to replace the functionality of __CUDA_ARCH__with something more sound (e.g. __global__(sm_120) void kernel(){} or __device__(sm_120) int func(){}, then we should be able to unify front-end compilation and then we’d have an option of either generating target-specific IR as we were intending to do long ago, or generate CIR, and generate target-specific IR later on.
Front-end improvements sound like a separate, semi-independent project that would be a companion for making heterogeneous CIR-based compilation suitable for practical use for CUDA compilation.
@jhuber6 If we were to have unified front-end compilation available (regardless of whether it produces an AST dump in PCH-like format, or CIR), what would we need to do to fit it into the new offloading driver? We’d want to split off per-target compilations later off in the actions DAG, after compilation, but it should not be too much of a change in principle. We may need to grow a new phase between compiler and back-end, but otherwise it should be doable.
The semantics of the language are somewhat separate from the realities of how it’s compiled. Officially, OpenMP is supposed to provide a unified interface. The way that works is by pushing code differences into the AST instead of the preprocessor so both sides see the same source. This is what #pragma omp begin/end declare variant is used for. In practice I’ve found these variants far less reliable than #ifdef, but it may be possible to design something more akin to function overloading.
The “New” offloading driver mostly just dictates the embedding and linking format. You could conceivably turn the split host / device compilation phases into a single job and it wouldn’t really care so long as it ends up with a host device that contains device binaries in the .llvm.offloading section. If you can handle that in a single CIR compiler step nothing significant would change.
Hi everyone! I would love to work on this project.
Regarding the feature’s goal, is it intended to be integrated to current pipelines as a modular step for joint analysis or would it change current pipelines?
In a modular structure, I believe it should come after the current pipelines, consuming host and device modules into the unified module, but before most passes (possibly after canonicalize). Passes that could benefit from joint visibility would be adapted to operate on it, re-splitting before target-specific passes.
On the other hand a closer integration/refactoring of current pipelines might reduce duplicate code and steps, like creating shadow variables only after dead-code optimizations. In that case the unified module would become an integral part of compilation for either pipeline, with joint visibility implemented as an optional toggle attribute.
How tightly integrated would the unified pipeline be with host and device pipelines?
I think the combined host-device CIR pipeline is a great idea and I fully support it. At least, we could decorate the LLVM IR with attributes or metadata found during the cross host/device analysis which will help the optimizations in the separate host/device compilations.
The primary idea is that you still keep split compilation on the early stages until you convert macro code and template resolution to CIR and we produce 1 CIR output for CPU and 1 for GPU (HIP/CUDA) compilation.
Once you have these 2 output files you can “merge” them into one and run a separate optimizer on top of the merged file. The merged file needs to keep track of the provenance of every symbol (is this a device or host symbol etc.). Once the optimizer is finished you split the single file into 2 separate CIR files one for host and one for device and you continue compilation as is.
At this point I would not consider compilations of multi-target gpu binaries. Although we should support them in the end. It sounds as too much to handle in a single project.
The idea would be that we keep early on the 2 completely separate compilation phases. So we keep the front end as is and instead of generating llvm ir on these 2 phases you generate CIR.
Then we write a tool that picks these CIR files and merged them into a single one and optionally optimizes the merged CIR. Lastly we need another tool to read in the merged CIR file and emit out 2 separate files one for host and one for device compilation. Once you emit these 2 files, you continue compilation as is.
The driver needs to properly handle the merging and splitting to properly handle the inclusion of the fatbin in the host image. This is doable by calling the proper tools at the right points.
