Bootstrapping standalone llvm/clang/libc++/lld toolchain

I’m trying to bootstrap a complete llvm/clang/libc++/lld toolchain on a standard Debian glibc/x86-64 system. (Ultimately, I want to do the same on a more unusual musl-based Linux distribution, but need to make the bootstrap work on a standard supported system before I attempt anything harder!)

Using a Debian/unstable chroot with up-to-date GNU toolchain, python3, and cmake, in a ‘build’ subdir of a checkout of release/12.x of llvm-project.git, I run:

cmake -G "Unix Makefiles" \
  -DLLVM_ENABLE_PROJECTS="compiler-rt;libunwind;libcxx;libcxxabi;lld;clang" \
  -DCLANG_DEFAULT_RTLIB=compiler-rt \
make -j16

This gives me libunwind, libstdc++, etc. correctly-built and independent of libgcc_s and libatomic:

# ldd lib/ (0x00007ffc31e5f000) => /lib/x86_64-linux-gnu/ (0x00007fde7be18000) => /lib/x86_64-linux-gnu/ (0x00007fde7bc53000) => /root/llvm-project/build/lib/ (0x00007fde7bc19000) => /root/llvm-project/build/lib/ (0x00007fde7bc02000)
/lib64/ (0x00007fde7bf1f000) => /lib/x86_64-linux-gnu/ (0x00007fde7bbfc000)

# ldd lib/ (0x00007ffd91d7f000) => /lib/x86_64-linux-gnu/ (0x00007f47c124f000) => /lib/x86_64-linux-gnu/ (0x00007f47c1249000) => /lib/x86_64-linux-gnu/ (0x00007f47c1227000)
/lib64/ (0x00007f47c1431000)

However, the compiler and other binaries have been linked against GNU libstdc++ and libgcc_s:

# ldd bin/clang (0x00007ffc5e5d8000) => /lib/x86_64-linux-gnu/ (0x00007f5ff97a5000) => /lib/x86_64-linux-gnu/ (0x00007f5ff979f000) => /usr/lib/x86_64-linux-gnu/ (0x00007f5ff95d2000) => /lib/x86_64-linux-gnu/ (0x00007f5ff948e000) => /lib/x86_64-linux-gnu/ (0x00007f5ff9474000) => /lib/x86_64-linux-gnu/ (0x00007f5ff92af000)
/lib64/ (0x00007f5fff181000)

To test the new compiler, I used the same bootstrap process again after a ‘make install’ to /usr/local: I re-ran the same cmake command in a clean ‘build2’ subdir with extra flags -DCMAKE_C_COMPILER=clang and -DCMAKE_CXX_COMPILER=clang++.

This time, the binaries were linked against libc++ correctly… but the libraries are now incorrectly linked to libgcc_s and libatomic - even though they weren’t at the end of the previous build!

# ldd bin/clang (0x00007ffee47f8000) => /lib/x86_64-linux-gnu/ (0x00007fd3740fa000) => /lib/x86_64-linux-gnu/ (0x00007fd3740ef000) => /lib/x86_64-linux-gnu/ (0x00007fd3740e9000) => /lib/x86_64-linux-gnu/ (0x00007fd373fa5000) => /root/llvm-project/build2/bin/../lib/ (0x00007fd373ed1000) => /root/llvm-project/build2/bin/../lib/ (0x00007fd373ec1000) => /root/llvm-project/build2/bin/../lib/ (0x00007fd373e7e000) => /lib/x86_64-linux-gnu/ (0x00007fd373cb9000)
/lib64/ (0x00007fd374122000) => /usr/lib/x86_64-linux-gnu/ (0x00007fd373caf000) => /lib/x86_64-linux-gnu/ (0x00007fd373c95000)

# ldd lib/ (0x00007fff264bb000) => /lib/x86_64-linux-gnu/ (0x00007fef1179c000) => /lib/x86_64-linux-gnu/ (0x00007fef115d7000) => /lib/x86_64-linux-gnu/ (0x00007fef11493000) => /lib/x86_64-linux-gnu/ (0x00007fef11488000) => /usr/lib/x86_64-linux-gnu/ (0x00007fef1147e000) => /root/llvm-project/build2/lib/ (0x00007fef1143d000) => /root/llvm-project/build2/lib/ (0x00007fef1142b000)
/lib64/ (0x00007fef11898000) => /lib/x86_64-linux-gnu/ (0x00007fef11411000) => /lib/x86_64-linux-gnu/ (0x00007fef1140b000)

# ldd lib/ (0x00007ffc0adf5000) => /lib/x86_64-linux-gnu/ (0x00007f3fd53c4000) => /lib/x86_64-linux-gnu/ (0x00007f3fd53aa000) => /lib/x86_64-linux-gnu/ (0x00007f3fd53a4000) => /lib/x86_64-linux-gnu/ (0x00007f3fd5382000)
/lib64/ (0x00007f3fd559f000)

Adding an additional -DCMAKE_CXX_FLAGS=-stdlib=libc++ -DCMAKE_EXE_LINKER_FLAGS=-stdlib=libc++ doesn’t change this either.

I expect I’m making an obvious beginner mistake, but I’m stumped! What am I doing wrong here? I aim to end up with a standalone toolchain, using libc++, llvm libunwind, etc., with compiler-rt providing builtins instead of libgcc_s/libatomic. (My sanity check is then that it can build itself correctly.)

PS The behaviour is identical with both llvm-project 11.0.1 and release/12.x. I’m using Debian unstable because buster/stable has binutils 2.31.1 with a bug that breaks the build: bug 42994.

1 Like

the README (faq and build directions) say if you have a custom location of the gcc tree you’ll need to supply that as build options or the linkage might find the wrong .so (or include parts of gcc in llvm’s tree - see the directions)

in your case on 2nd try it worked. unless your making the debian Relase for debian, your really done, so

No, the aim was to produce a toolchain and runtime entirely independent of the host distribution toolchain and runtime, but the second build was picking up a stray libgcc_s and libatomic linkage.

For what it’s worth, the fix was an additional


The stray gcc dependency was creeping in through libunwind.