The libc implementation for the GPUs

@jdoerfert et al., I suppose the proposed libc implementation will be written to the LibC.{h,cpp} files, am I correct?

Currently only a few pieces of code there though.

Are there plans to cooperate with the LLVM libc?

2 Likes

The design of GPU libc is not decided yet. (let’s not jump to libc++ yet as it is even less clear and libm might also be a different story depending on what we do).

There are three options, and ongoing development (RPC support) that plays into this. Options:

  1. Implement our own device subset of libc, in LibC.{h,cpp}, and use RPC for functions we want to support via the host.
  2. Port a simple libc onto the device, use RPC for the hard parts or disable them for now. Simple libc implementations exist out there.
  3. Retarget llvm/libc for GPUs, RPC again for host parts we want to support.

The complexity grows as you move down the list but also the capabilities and code reuse. We are going to evaluate option 3 soon and report back. If people are interested in this, let me know.

An early RPC prototype, and some more libc function support (via option 1), were explored as part of this work recently: https://tianshilei.me/wp-content/uploads/2022/09/llvm-hpc-22.pdf
That said, we will not continue with exactly this RPC impl. as it requires manual stubs and writing libc functions (via option 1) turned out to be tedious…

2 Likes

During the mdspan discussion:

A similar topic came up: libc++ for GPUs.

I saw that, but I am unsure what this is supposed to mean. If you want the header only parts that do not pull in lib(std)c++ libraries, you can already use it. This is also unrelated of dynamic memory allocations or “inlineable”.

IIRC, LLVM libc is very modular. If you want to start with strlen and hate printf, you can create your own config.

@tschuett do they have a timeframe as to when the libc will have all the functions that GNU libc?

Unfortunately, no. I am not a libc developer. They made a lot of progress. And as I hinted above it is a modular design. You pick strlen today and in a month you can add memcpy.

1 Like

I’m interested in testing the option 3 prototype.

On the top left is a status category:
https://libc.llvm.org/index.html

There will be a porting guide for new targets, i.e., NVIDIA_GPU_SM30, soonish.
https://reviews.llvm.org/D136960

1 Like

Great. Will make it easier to try out a GPU port! Once we give it a shot we’ll comment here again.

1 Like

The order is libc first, then libc++ follows after that?

The first support forlibc functions was landed upstream with documentation here. Right now it only supports the string.h and ctype.h functions. We are currently developing an RPC mechanism to enable calling functions that require OS support. Feedback would be appreciated.

@jhuber6 I am trying to test it though at build time I get:

-- Configuring incomplete, errors occurred!
See also "/home/ubuntu/projects/llvm-project/build/runtimes/runtimes-bins/CMakeFiles/CMakeOutput.log".
See also "/home/ubuntu/projects/llvm-project/build/runtimes/runtimes-bins/CMakeFiles/CMakeError.log".
FAILED: runtimes/runtimes-stamps/runtimes-configure /home/ubuntu/projects/llvm-project/build/runtimes/runtimes-stamps/runtimes-configure
cd /home/ubuntu/projects/llvm-project/build/runtimes/runtimes-bins && /usr/bin/cmake -DCMAKE_C_COMPILER=/home/ubuntu/projects/llvm-project/build/./bin/clang -DCMAKE_CXX_COMPILER=/home/ubuntu/projects/llvm-project/build/./bin/clang++ -DCMAKE_ASM_COMPILER=/home/ubuntu/projects/llvm-project/build/./bin/clang -DCMAKE_LINKER=/home/ubuntu/projects/llvm-project/build/./bin/ld.lld -DCMAKE_AR=/home/ubuntu/projects/llvm-project/build/./bin/llvm-ar -DCMAKE_RANLIB=/home/ubuntu/projects/llvm-project/build/./bin/llvm-ranlib -DCMAKE_NM=/home/ubuntu/projects/llvm-project/build/./bin/llvm-nm -DCMAKE_OBJDUMP=/home/ubuntu/projects/llvm-project/build/./bin/llvm-objdump -DCMAKE_OBJCOPY=/home/ubuntu/projects/llvm-project/build/./bin/llvm-objcopy -DCMAKE_STRIP=/home/ubuntu/projects/llvm-project/build/./bin/llvm-strip -DCMAKE_READELF=/home/ubuntu/projects/llvm-project/build/./bin/llvm-readelf -DCMAKE_C_COMPILER_TARGET=aarch64-unknown-linux-gnu -DCMAKE_CXX_COMPILER_TARGET=aarch64-unknown-linux-gnu -DCMAKE_ASM_COMPILER_TARGET=aarch64-unknown-linux-gnu -DCMAKE_INSTALL_PREFIX=/home/ubuntu/opt/llvm -DLLVM_BINARY_DIR=/home/ubuntu/projects/llvm-project/build -DLLVM_CONFIG_PATH=/home/ubuntu/projects/llvm-project/build/bin/llvm-config -DLLVM_ENABLE_WERROR=OFF -DLLVM_HOST_TRIPLE=aarch64-unknown-linux-gnu -DLLVM_HAVE_LINK_VERSION_SCRIPT=1 -DLLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO=OFF -DLLVM_USE_RELATIVE_PATHS_IN_FILES=OFF -DLLVM_LIT_ARGS=-sv -DLLVM_SOURCE_PREFIX= -DPACKAGE_VERSION=16.0.0git -DCMAKE_BUILD_TYPE=Release -DCMAKE_MAKE_PROGRAM=/usr/bin/ninja -DCMAKE_C_COMPILER_LAUNCHER= -DCMAKE_CXX_COMPILER_LAUNCHER= -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DCOMPILER_RT_BUILD_BUILTINS=Off -DLLVM_INCLUDE_TESTS=ON -DLLVM_TARGET_TRIPLE=aarch64-unknown-linux-gnu -DLLVM_ENABLE_PROJECTS_USED=ON -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON -DLLVM_BUILD_TOOLS=ON -DCMAKE_C_COMPILER_WORKS=ON -DCMAKE_CXX_COMPILER_WORKS=ON -DCMAKE_ASM_COMPILER_WORKS=ON -DHAVE_LLVM_LIT=ON "-DLLVM_ENABLE_RUNTIMES=libc;openmp" -DLLVM_LIBCXX_USED=0 -DLLVM_LIBC_FULL_BUILD=ON -GNinja /home/ubuntu/projects/llvm-project/llvm/runtimes/../../runtimes && /usr/bin/cmake -E touch /home/ubuntu/projects/llvm-project/build/runtimes/runtimes-stamps/runtimes-configure
ninja: build stopped: subcommand failed.```

This is how I invoked cmake:

cmake -B build -S ./llvm -G Ninja \
   -DLLVM_ENABLE_PROJECTS="clang;lld;compiler-rt" \
   -DLLVM_ENABLE_RUNTIMES="libc;openmp" \
   -DCMAKE_BUILD_TYPE=Release \
   -DLLVM_LIBC_FULL_BUILD=ON \
   -DCMAKE_INSTALL_PREFIX=$HOME/opt/llvm

You are missing at least:

   -DLLVM_LIBC_TARGET_OS=gpu           \ # Build in GPU mode
   -DLLVM_LIBC_GPU_ARCHITECTURES=all   \ # Build all supported architectures
2 Likes