I propose a header called gpuintrin.h that contains common definitions of GPU intrinsic functions and other attributes. These are intended to be usable between CUDA, OpenMP, HIP, and C/C++ as a more user-friendly way to access common GPU intrinsics between vendors. This is reminiscent of OpenCL, but much less restrictive.
The goal is that the interface is as common as possible, so it’s easier to port code between NVPTX and AMDGPU, and maybe SPIR-V targets in the future. To that end, the AMDGPU functions take a lane mask and the NVPTX functions return a 64-bit lane mask despite the warp size of 32. However, these will be placed in different headers so target specific things could be added as-needed.
Code
An example of using this header would be as follows.
#include <gpuintrin.h>
_kernel void saxpy(int n, float a, float *x, float *y) {
int i = _get_block_id_x() * _get_num_blocks_x() + _get_thread_id_x();
if (i < n)
y[i] = a * x[i] + y[i];
}
This can then be compiled into a GPU binary via one of the following compilations. Either that or they can just be used directly in CUDA or OpenMP as a replacement for those language’s functions.
I have a draft open at https://github.com/llvm/llvm-project/pull/110179. Comments would be appreciated, This is primarily an ease-of-use proposal, let me know if this is too specific to be useful to anyone but myself. The code here was simply adapted from the internal headers in the C library port for the GPU.
Aaron is away for the next two weeks I think, so we should probably want to wait to see his feedback. I’m interested in something like this in concept, but would like to do a deeper dive as the conversation progresses.
I want something like this. It’s annoying to DIY it in anything that targets amdgpu and nvptx.
Could go C declarations (with a prefix) and a bitcode implementation for ease of use by Fortran or header only (the C and C++ compatible incantation is probably static inline).
If it’s only usable by programs linking liboffload then value is reduced.
The other play is to just fill out the clang intrinsics, document them, and don’t bother with the header at all. That’s a hassle in implementation but avoids fretting about details like not having a GPU compiler-rt to put the implementation in. Lowering to common IR intrinsics would be helpful for lit tests of GPU code.
My experience is that GPU programming via C and C++ is very convenient, but requires a lot of internal compiler knowledge. Ideally this makes it easier to write common code on various platforms, since it’s something we can always rely on.
Yeah, I can see using _gpu so it’s more obvious, as names like thread_id are heavily oversubscribed.
Agreed.
You can build compiler-rt for the GPU currently (I should probably document this). If we need to expose these calls as a bitcode library I was thinking we could just make libclang_rt.gpu.a or libclang_rt.gpu.bc. That would be fairly trivial with the infrastructure I have built. It would just require the go-ahead from the compiler-rt team.
Common intrinsics are also doable, but they’re probably more difficult to manage since they’re baked into the compiler and some things are different between targets (i.e. the SIMT intrinsics). Managing it via a header is the “easy” solution, but not necessarily the only one.
The set of functions declared in the linked PR don’t appear to be specific to GPUs and might be useful for other kinds of devices. If so, then the naming should be made more general; perhaps offloadintrin.h and offload_ prefixed function names.
I’m unsure if I want to make the scope quite that wide, considering that this will assume a SIMT interface and grid-based kernels. However, it’s definitely possible that we could provide other targets, and I’m wondering if I should provide a fallback interface for the CPU that mostly no-ops this stuff (Mostly to make OpenMP happy.
It’s also possible to make a CPU environment that would behave like a grid based system so, maybe one could resurrect the “Simulate a GPU on a CPU for debugging” thing.
We most certainly should. We also should implement __shared such that the CPU impl will automatically do scaled allocations and block based indixing. But that is beyond this RFC.
Concrete comments for the RFC/PR:
Make it __offload_..., or similar.
Provide forward declares with comments that define the interface.
Consider macro/tblgen impl. especially since it’s so repetitive and we want at least 1-2 more versions of all the functions (C++ and Fortran).
Hook them up to one or more of the use sites for testing, e.g., libc and/or ompx_ extensions.
Consider offload as a location for this. As mentioned above, we want C++ and Fortran bindings for sure, likely other languages as well. We can keep them in the respective Frontends, or centralize them. The latter would make sense especially if we “generate” the bindings, that is we have a tblgen or macro impl. and just change the syntax per target language. Alternatively, we have the ground truth C impl. and the rest translates there. In that case we need IR linkage for optimization though.
I’m not a fan, this is intended to be more specific towards GPUs. The use-case is also intended to cover direct compilation (i.e. OpenCL or C++) which isn’t covered under offloading like CUDA or OpenMP.
Tablegen probably won’t work since none of the other resource headers are generated, macro magic could probably make some things easier so I’ll look into it. (Also looking into resolving your comments on the PR, thanks.)
I will write tests soon, mostly just to verify that they call the correct builtins. I could put runtime tests in libc/ very easily as well if needed.
The purpose of putting this in the clang resource directory is that it will always be available with any functional clang C/C++ compiler. I think that providing a library for this is a separate issue, I would like that to go in compiler-rt if we go that route since it fits the same niche as the builtins.
+1. It can be useful to have language neutral clang built-in functions and LLVM intrinsics for grid properties and some convergent instructions. All (?) GPUs and GPU languages expose concepts of thread ID, grid, blocks, barriers, etc.
As an example, Intel’s SYCL compiler uses one set of clang built-ins to target Intel, NVIDIA and AMD GPU. For historical reasons, there built-ins have SPIR-V target specific names, but the main idea is that implementation for these built-ins for specific GPU hooked at LLVM IR level. Usually, it’s a small wrapper around a target intrinsic. The same set of clang built-ins is also used for OpenMP offload to Intel GPU compilation mode.
I’m not sure, if it’s relevant at this point, but our compiler applied OpenCL’s technique for injecting built-in declarations into user’s code at AST level to avoid overhead on parsing declarations of GPU intrinsics. We use thousands of built-ins to implement math and conversion functions for a range of vector types, so parsing headers with intrinsics declarations take noticeable time.
This realistically wouldn’t be that difficult in the general sense, we just add some generic builtins and have some shim code in CGBuiltin.cpp in clang that converts them to the underlying call. The main issue with this is that the APIs and types are slightly different. I.e. NVPTX uses u32 for its SIMT mask while AMDGPU uses u64 and NVPTX has a mask argument for everything and AMDGPU doesn’t. I already put this into a common implementation in the headers, but it’s something we’d need to agree on and handle in codegen.
Also, there’s useful stuff that’s not just the builtins. A casual user isn’t going to know that __attribute__((amdgpu_kernel, visibility("protected"))) is required to make a callable GPU kernel, __scoped_atomic_fetch_add(x, 1, __ATOMIC_RELAXED, __SCOPE_DEVICE) does atomic scoping, or __attribute__((opencl_local)) lets you declare LDS. Though it’s possible this header could be much shorter if we just had common builtins.
The argument that offloading has to be CUDA or OpenMP but can’t be direct compilation via OpenCL or C++ is arbitrary. For one, OpenCL and CUDA are in the same class of languages. Second, as mentioned by @tahonermann, this is not only a GPU thing. Other accelerators have basically the same concepts, incl. multi-dim grids, synchronization on different levels, etc. It seems weird to say that we provide some generic wrappers to unify NVIDIA and AMD GPUs, but we intentionally stop there. That all said, we need some prefix, and if people really want it to be __gpu or __llvm_gpu, I’d be OK with that too.
Ah, thanks for pointing that out. I just did a quick scan of the clang/lib/Headers dir and didn’t find any .td files but I see that they’re kept elsewhere.
I was looking into doing GPU builtins, somewhat like the GPU dialect in MLIR, but found it was just too inconvenient trying to ‘alias’ builtins to LLVM intrinsics that were already defined. For cases handled directly in CGBuiltin.cpp this was just a matter of adding it above the switch statement, however for others handled in the LLVM .td file directly it wasn’t so simple and would result in a lot of extra validation work.
I don’t think that tablegen would help much here since it’s pretty much one-line redirects to some unique string anyway, maybe we could #define it but I don’t know if it’s worth the effort. I changed the prefix to __gpu and added some tests as well.