I’m attempting to learn LLVM, and my first task is to build a fully functional, self-contained toolchain that does not depend on any external components, e.g. libgcc_s, and as of this moment, I’m struggling .
I’m a little confused by all the different build approaches, e.g. bootstrap builds using -DCLANG_ENABLE_BOOTSTRAP=ON", distributions using cmake cache files (e.g. clang/cmake/caches/DistributionExample.cmake referenced by Building a Distribution of LLVM — LLVM 23.0.0git documentation, or several “stand-alone” builds done in order.
I’ve successfully built a toolchain from llvm-project monorepo git tip, using -DCLANG_ENABLE_BOOTSTRAP=ON, with the following config:
I’d appreciate any pointers on how to correctly build a self-contained LLVM toolchain? Or even which method I should be using. The documentation can be quite confusing.
For reference, I’m running a Gentoo host system with native LLVM + musl.
I’ve done something similar for the release builds. You can look at Release.cmake and the LLVM_RELEASE_ENABLE_LINK_LOCAL option.
Can you extract the linking command used to link libc++.so.1.0 from the build logs so we can see what it is doing. Also is ~/out/lib/libc++.so.1.0 from the stage1 build or the stage2 build? What command did you use to install it?
Amazing! This seems to be exactly what I was looking for, thanks. I could have sworn I looked at this file before, but I guess I skimmed right past the `LLVM_RELEASE_ENABLE_LINK_LOCAL` section.
I don’t have the build artifacts from that run anymore, but I can post them if I run that configuration again, unless you think this is a potential bug I could help diagnose? If so, I’ll try to reproduce.
Primarily, yeah.
But my goal is to do a deep dive into the LLVM project and all of its constituent parts, and hopefully become proficient enough to contribute upstream. I would be dogfooding the build for my primary Gentoo workstation and other development work.
I figure the best way to dogfood is to use a complete, self-contained toolchain comprised of as many of the in-tree components as possible, i.e. I would prefer to be using the checked out version of compilter-rt, incl its atomics, builtins, sanitizers, and not be pulling in libgcc for anything, that would “pollute” the toolchain (?).
Looks like we’re in business, though I got the following errors:
CMake Error at /home/user/src/llvm-project/libunwind/src/CMakeLists.txt:95 (message):
Compiler doesn't support generation of unwind tables if exception support
is disabled. Building libunwind DSO with runtime dependency on C++ ABI
library is not supported.
This was because I had LLVM_HOST_TRIPLE=x86_64-bdprom-linux-musl set, which I guess caused it to not find my GCC install, which it needs apparently (there goes my self-containment I guess?). Fixed by instead setting LLVM_HOST_TRIPLE=x86_64-pc-linux-musl.
I’m thinking the PROJECTS/RUNTIMES options might not be working, as it’s currently building 11,000 objects, which is like 4x what it usually is. Could this be because I’m building with ninja stage2 instead of some other correct CMake target?
Thanks @tstellar , I didn’t see an option to disable BOLT, so I just commented out the relevant sections in the .cmake, but now I get:
CMake Error at /home/user/src/llvm-project/libunwind/src/CMakeLists.txt:95 (message):
Compiler doesn't support generation of unwind tables if exception support
is disabled. Building libunwind DSO with runtime dependency on C++ ABI
library is not supported.
-- Configuring incomplete, errors occurred!
[7310/7314] Linking CXX executable bin/llvm-lto2
FAILED: [code=1] runtimes/runtimes-stamps/runtimes-configure /home/user/build/tools/clang/stage2-bins/runtimes/runtimes-stamps/runtimes-configure
cd /home/user/build/tools/clang/stage2-bins/runtimes/runtimes-bins && /usr/bin/cmake --no-warn-unused-cli -DCMAKE_C_COMPILER=/home/user/build/tools/clang/stage2-bins/./bin/clang -DCMAKE_CXX_COMPILER=/home/user/build/tools/clang/stage2-bins/./bin/clang++ -DCMAKE_ASM_COMPILER=/home/user/build/tools/clang/stage2-bins/./bin/clang -DCMAKE_LINKER=/home/user/build/tools/clang/stage2-bins/./bin/ld.lld -DCMAKE_AR=/home/user/build/tools/clang/stage2-bins/./bin/llvm-ar -DCMAKE_RANLIB=/home/user/build/tools/clang/stage2-bins/./bin/llvm-ranlib -DCMAKE_NM=/home/user/build/tools/clang/stage2-bins/./bin/llvm-nm -DCMAKE_OBJDUMP=/home/user/build/tools/clang/stage2-bins/./bin/llvm-objdump -DCMAKE_OBJCOPY=/home/user/build/tools/clang/stage2-bins/./bin/llvm-objcopy -DCMAKE_STRIP=/home/user/build/tools/clang/stage2-bins/./bin/llvm-strip -DCMAKE_READELF=/home/user/build/tools/clang/stage2-bins/./bin/llvm-readelf -DCMAKE_C_COMPILER_TARGET=x86_64-unknown-linux-gnu -DCMAKE_CXX_COMPILER_TARGET=x86_64-unknown-linux-gnu -DCMAKE_Fortran_COMPILER_TARGET=x86_64-unknown-linux-gnu -DCMAKE_ASM_COMPILER_TARGET=x86_64-unknown-linux-gnu -DCMAKE_INSTALL_PREFIX=/home/user/out -DLLVM_BINARY_DIR=/home/user/build/tools/clang/stage2-bins -DLLVM_CONFIG_PATH=/home/user/build/tools/clang/stage2-bins/bin/llvm-config -DLLVM_ENABLE_WERROR=OFF -DLLVM_HOST_TRIPLE=x86_64-unknown-linux-gnu -DLLVM_HAVE_LINK_VERSION_SCRIPT=1 -DLLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO=OFF -DLLVM_USE_RELATIVE_PATHS_IN_FILES=OFF -DLLVM_LIT_ARGS=-sv -DLLVM_SOURCE_PREFIX= -DPACKAGE_VERSION=23.0.0git -DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_MAKE_PROGRAM=/usr/bin/ninja -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DCOMPILER_RT_BUILD_BUILTINS=Off -DLLVM_INCLUDE_TESTS=ON -DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu -DLLVM_ENABLE_PROJECTS_USED=ON -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON -DLLVM_BUILD_TOOLS=ON -DCMAKE_C_COMPILER_WORKS=ON -DCMAKE_CXX_COMPILER_WORKS=ON -DCMAKE_Fortran_COMPILER_WORKS=ON -DCMAKE_ASM_COMPILER_WORKS=ON -DHAVE_LLVM_LIT=ON -DCLANG_RESOURCE_DIR= -DCOMPILER_RT_TEST_EXTERNAL_BUILTINS=ON "-DLLVM_ENABLE_RUNTIMES=compiler-rt;libcxx;libcxxabi;libunwind" -DFFI_INCLUDE_DIR= -DFFI_LIBRARY_DIR= -DCOMPILER_RT_BUILD_GWP_ASAN=OFF -DLIBCXX_HAS_MUSL_LIBC=ON -GNinja -C/home/user/build/tools/clang/stage2-bins/projects/runtimes/tmp/runtimes-cache-RELEASE.cmake -S /home/user/src/llvm-project/llvm/runtimes/../../runtimes -B /home/user/build/tools/clang/stage2-bins/runtimes/runtimes-bins && /usr/bin/cmake -E touch /home/user/build/tools/clang/stage2-bins/runtimes/runtimes-stamps/runtimes-configure
ninja: build stopped: subcommand failed.
FAILED: [code=1] tools/clang/stage2-stamps/stage2-build /home/user/build/tools/clang/stage2-stamps/stage2-build
cd /home/user/build/tools/clang/stage2-bins && /usr/bin/cmake --build /home/user/build/tools/clang/stage2-bins/ --config RELEASE && /usr/bin/cmake -E touch /home/user/build/tools/clang/stage2-stamps/stage2-build
ninja: build stopped: subcommand failed.
At this point I think I might try the clean sysroot approach, i.e. install musl-headers, then linux-headers, compiler-rt, etc etc. I found some potentially useful sources which I’ll attempt to adapt to my needs:
If you want to try a different approach, I wonder if Petr Hosek’s talk on the build might be helpful: https://www.youtube.com/watch?v=Dnubzx8-E1M . I think it does say that compiler-rt can be built standalone, but there are two designated top levels from which other components can be built, llvm and runtimes. But maybe it’s different building a runtime with the compiler just built, and with a host compiler. Just to suggest the talk in case it helps.
Thanks, I’ll watch this later. I’ve actually now managed to build a standalone toolchain, and if I was to start compiling software in a chroot, I think I could stop here. But, using this toolchain on my host system is still problematic, lots of broken paths and whatnot.
@bdprom I was just wondering if you’ve had any further success? I keep trying to make a MUSL build of llvm clang/flang but keep failing.
If it’s possible, it would make my life much nicer since the official LLVM release tarfiles are for a glibc version much too new for the supercomputing clusters I work on (OS updates aren’t done often, and SLES 15 isn’t exactly high priority OS). I’d just love to grab a tarfile, untar it, and boom, working clang/flang!.
I tried to follow the PancakeTAS example you pointed at, but just kept running into issues.
.
I could have sworn I looked at this file before, but I guess I skimmed right past the `