a couple of weeks ago I discussed with Peter how to improve LLVM's support for heterogeneous computing. One weakness we (and others) have seen is the absence of multi-module support in LLVM. Peter came up with a nice idea how to improve here. I would like to put this idea up for discussion.
## The problem ##
LLVM-IR modules can currently only contain code for a single target architecture. However, there are multiple use cases where one translation unit could contain code for several architectures.
cuda source files can contain both host and device code. The absence of multi-module support complicates adding CUDA support to clang, as clang would need to perform multi-module compilation on top of a single-module based compiler framework.
2) C++ AMP
C++ AMP  contains - similarly to CUDA - both host code and device code in the same source file. Even if C++ AMP is a Microsoft extension the use case itself is relevant to clang. It would be great if LLVM would provide infrastructure, such that front-ends could easily target accelerators. This would probably yield a lot of interesting experiments.
To fully automatically offload computations to an accelerator an optimization pass needs to extract the computation kernels and schedule
them as separate kernels on the device. Such kernels are normally LLVM-IR modules for different architectures. At the moment, passes have no way to create and store new LLVM-IR modules. There is also no way
to reference kernel LLVM-IR modules from a host module (which is necessary to pass them to the accelerator run-time).
## Goals ##
a) No major changes to existing tools and LLVM based applications
b) Human readable and writable LLVM-IR
c) FileCheck testability
d) Do not force a specific execution model
e) Unlimited number of embedded modules
## Detailed Goals
o No changes should be required, if a tool does not use multi-module
support. Each LLVM-IR file valid today, should remain valid.
o Major tools should support basic heterogeneous modules without large
changes. Some of the commands that should work after smaller
clang -S -emit-llvm -o out.ll
opt -O3 out.ll -o out.opt.ll
bugpoint -O3 out.opt.ll
b) All (sub)modules should be directly human readable/writable.
There should be no need to extract single modules before modifying
c) The LLVM-IR generated from a heterogeneous multi-module should
easily be 'FileCheck'able. The same is true, if a multi-module is
the result of an optimization.
d) In CUDA/OpenCL/C++ AMP kernels are scheduled from within the host
code. This means arbitrary host code can decide under which
conditions kernels are scheduled for execution. It is therefore
necessary to reference individual sub-modules from within the host
e) CUDA/OpenCL allows to compile and schedule an arbitrary number of
kernels. We do not want to put an artificial limit on the number of
modules they are represented in. This means a single embedded
submodule is not enough.
## Non Goals ##
o Modeling sub-architectures on a per-function basis
Functions could be specialized for a certain sub-architecture. This is helpful to have certain functions optimized e.g. with AVX2 enabled, but the general program being compiled for a more generic architecture.
We do not address per-function annotations in this proposal.
## Proposed solution ##
To bring multi-module support to LLVM, we propose to add a new type called 'llvmir' to LLVM-IR. It can be used to embed LLVM-IR submodules
as global variables.