Failed to find compiler-rt library

Beginners
#1

Hello,

I am using LLVM 14.0.4 to cross build musl libc, below is the build command that I use:

./configure CC="clang --target=aarch64-linux-gnu" --target=aarch64-linux-gnu CFLAGS="--rtlib=compiler-rt" AR=/usr/local/llvm-14.0.3/bin/llvm-ar RANLIB=/usr/local/llvm-14.0.3/bin/llvm-ranlib --syslibdir=/usr/local/lib LIBCC=-L/usr/local/llvm-14.0.3/lib/clang/14.0.3/lib/linux LDFLAGS="-fuse-ld=lld"
make

But it report the below error:

ld.lld: error: undefined symbol: __subtf3
>>> referenced by cacosl.c
>>>               obj/src/complex/cacosl.lo:(cacosl)
>>> referenced by casinl.c
>>>               obj/src/complex/casinl.lo:(casinl)
>>> referenced by casinl.c
>>>               obj/src/complex/casinl.lo:(casinl)
>>> referenced 102 more times

ld.lld: error: undefined symbol: __addtf3
>>> referenced by casinl.c
>>>               obj/src/complex/casinl.lo:(casinl)
>>> referenced by casinl.c
>>>               obj/src/complex/casinl.lo:(casinl)
>>> referenced by catanl.c
>>>               obj/src/complex/catanl.lo:(catanl)
>>> referenced 261 more times

ld.lld: error: undefined symbol: __multf3
>>> referenced by casinl.c
>>>               obj/src/complex/casinl.lo:(casinl)
>>> referenced by casinl.c
>>>               obj/src/complex/casinl.lo:(casinl)
>>> referenced by casinl.c
>>>               obj/src/complex/casinl.lo:(casinl)
>>> referenced 244 more times

ld.lld: error: undefined symbol: __divtf3
>>> referenced by catanl.c
>>>               obj/src/complex/catanl.lo:(catanl)
>>> referenced by catanl.c
>>>               obj/src/complex/catanl.lo:(catanl)
>>> referenced by floatscan.c
>>>               obj/src/internal/floatscan.lo:(__floatscan)
>>> referenced 22 more times

ld.lld: error: undefined symbol: __getf2
>>> referenced by catanl.c
>>>               obj/src/complex/catanl.lo:(catanl)
>>> referenced by floatscan.c
>>>               obj/src/internal/floatscan.lo:(__floatscan)
>>> referenced by floatscan.c
>>>               obj/src/internal/floatscan.lo:(__floatscan)
>>> referenced 5 more times

ld.lld: error: undefined symbol: __fixtfdi
>>> referenced by catanl.c
>>>               obj/src/complex/catanl.lo:(catanl)
>>> referenced by __rem_pio2l.c
>>>               obj/src/math/__rem_pio2l.lo:(__rem_pio2l)
>>> referenced by llrintl.c
>>>               obj/src/math/llrintl.lo:(llrintl)
>>> referenced 3 more times

ld.lld: error: undefined symbol: __floatditf
>>> referenced by catanl.c
>>>               obj/src/complex/catanl.lo:(catanl)

ld.lld: error: undefined symbol: __trunctfsf2
>>> referenced by catanl.c
>>>               obj/src/complex/catanl.lo:(catanl)
>>> referenced by catanl.c
>>>               obj/src/complex/catanl.lo:(catanl)
>>> referenced by atanhl.c
>>>               obj/src/math/atanhl.lo:(atanhl)
>>> referenced 7 more times

ld.lld: error: undefined symbol: __extendsftf2
>>> referenced by catanl.c
>>>               obj/src/complex/catanl.lo:(catanl)
>>> referenced by catanl.c
>>>               obj/src/complex/catanl.lo:(catanl)
>>> referenced by floatscan.c
>>>               obj/src/internal/floatscan.lo:(__floatscan)
>>> referenced 3 more times

ld.lld: error: undefined symbol: __trunctfdf2
>>> referenced by ccoshl.c
>>>               obj/src/complex/ccoshl.lo:(ccoshl)
>>> referenced by ccoshl.c
>>>               obj/src/complex/ccoshl.lo:(ccoshl)
>>> referenced by cexpl.c
>>>               obj/src/complex/cexpl.lo:(cexpl)
>>> referenced 31 more times

ld.lld: error: undefined symbol: __extenddftf2
>>> referenced by ccoshl.c
>>>               obj/src/complex/ccoshl.lo:(ccoshl)
>>> referenced by ccoshl.c
>>>               obj/src/complex/ccoshl.lo:(ccoshl)
>>> referenced by cexpl.c
>>>               obj/src/complex/cexpl.lo:(cexpl)
>>> referenced 39 more times

ld.lld: error: undefined symbol: __muldc3
>>> referenced by cpow.c
>>>               obj/src/complex/cpow.lo:(cpow)
>>> referenced by csqrt.c
>>>               obj/src/complex/csqrt.lo:(csqrt)

ld.lld: error: undefined symbol: __mulsc3
>>> referenced by cpowf.c
>>>               obj/src/complex/cpowf.lo:(cpowf)

ld.lld: error: undefined symbol: __unordtf2
>>> referenced by cpowl.c
>>>               obj/src/complex/cpowl.lo:(cpowl)
>>> referenced by cpowl.c
>>>               obj/src/complex/cpowl.lo:(cpowl)
>>> referenced by vfprintf.c
>>>               obj/src/stdio/vfprintf.lo:(printf_core)

ld.lld: error: undefined symbol: __multc3
>>> referenced by cpowl.c
>>>               obj/src/complex/cpowl.lo:(cpowl)

ld.lld: error: undefined symbol: __floatsitf
>>> referenced by floatscan.c
>>>               obj/src/internal/floatscan.lo:(__floatscan)
>>> referenced by floatscan.c
>>>               obj/src/internal/floatscan.lo:(__floatscan)
>>> referenced by floatscan.c
>>>               obj/src/internal/floatscan.lo:(__floatscan)
>>> referenced 15 more times

ld.lld: error: undefined symbol: __floatunsitf
>>> referenced by floatscan.c
>>>               obj/src/internal/floatscan.lo:(__floatscan)
>>> referenced by floatscan.c
>>>               obj/src/internal/floatscan.lo:(__floatscan)
>>> referenced by floatscan.c
>>>               obj/src/internal/floatscan.lo:(__floatscan)
>>> referenced 6 more times

ld.lld: error: undefined symbol: __netf2
>>> referenced by floatscan.c
>>>               obj/src/internal/floatscan.lo:(__floatscan)
>>> referenced by floatscan.c
>>>               obj/src/internal/floatscan.lo:(hexfloat)
>>> referenced by floatscan.c
>>>               obj/src/internal/floatscan.lo:(hexfloat)
>>> referenced 21 more times

ld.lld: error: undefined symbol: __eqtf2
>>> referenced by floatscan.c
>>>               obj/src/internal/floatscan.lo:(__floatscan)
>>> referenced by acosl.c
>>>               obj/src/math/acosl.lo:(acosl)
>>> referenced by asinl.c
>>>               obj/src/math/asinl.lo:(asinl)
>>> referenced 15 more times

ld.lld: error: undefined symbol: __lttf2
>>> referenced by __rem_pio2l.c
>>>               obj/src/math/__rem_pio2l.lo:(__rem_pio2l)
>>> referenced by __tanl.c
>>>               obj/src/math/__tanl.lo:(__tanl)
>>> referenced by __tanl.c
>>>               obj/src/math/__tanl.lo:(__tanl)
>>> referenced 18 more times

ld.lld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)

Since I think those undefined symbols e.g., __subtf3 should be in the compiler-rt, do it mean that lld failed to find compiler-rt library? But I have used the build option “LIBCC=-L/usr/local/llvm-14.0.3/lib/clang/14.0.3/lib/linux” to specify the compiler runtime library, is there anything else I should do? Thanks!

#2

If I remember correctly you have to pass the full path and library name in the LIBCC variable in MUSL - so it should be LIBCC=<path>/clang_rt.builtins.x86_64.a

#3

Hello，

Thanks for you help, I tried LIBCC=/usr/local/llvm-14.0.3/lib/clang/14.0.3/lib/linux, but the problem is still here.

I check the folder /usr/local/llvm-14.0.3/lib/clang/14.0.3/lib/linux, and fiind that the below files are in this folder:

clang_rt.crtbegin-x86_64.o                libclang_rt.hwasan_aliases-x86_64.a.syms      libclang_rt.scudo_minimal-x86_64.a
clang_rt.crtend-x86_64.o                  libclang_rt.hwasan_aliases-x86_64.so          libclang_rt.scudo_minimal-x86_64.so
libclang_rt.asan-preinit-x86_64.a         libclang_rt.hwasan_aliases_cxx-x86_64.a       libclang_rt.scudo_standalone-x86_64.a
libclang_rt.asan-x86_64.a                 libclang_rt.hwasan_aliases_cxx-x86_64.a.syms  libclang_rt.scudo_standalone-x86_64.so
libclang_rt.asan-x86_64.a.syms            libclang_rt.hwasan_cxx-x86_64.a               libclang_rt.scudo_standalone_cxx-x86_64.a
libclang_rt.asan-x86_64.so                libclang_rt.hwasan_cxx-x86_64.a.syms          libclang_rt.stats-x86_64.a
libclang_rt.asan_cxx-x86_64.a             libclang_rt.lsan-x86_64.a                     libclang_rt.stats_client-x86_64.a
libclang_rt.asan_cxx-x86_64.a.syms        libclang_rt.memprof-preinit-x86_64.a          libclang_rt.tsan-x86_64.a
libclang_rt.asan_static-x86_64.a          libclang_rt.memprof-x86_64.a                  libclang_rt.tsan-x86_64.a.syms
libclang_rt.builtins-x86_64.a             libclang_rt.memprof-x86_64.a.syms             libclang_rt.tsan-x86_64.so
libclang_rt.cfi-x86_64.a                  libclang_rt.memprof-x86_64.so                 libclang_rt.tsan_cxx-x86_64.a
libclang_rt.cfi_diag-x86_64.a             libclang_rt.memprof_cxx-x86_64.a              libclang_rt.tsan_cxx-x86_64.a.syms
libclang_rt.dd-x86_64.a                   libclang_rt.memprof_cxx-x86_64.a.syms         libclang_rt.ubsan_minimal-x86_64.a
libclang_rt.dfsan-x86_64.a                libclang_rt.msan-x86_64.a                     libclang_rt.ubsan_minimal-x86_64.a.syms
libclang_rt.dfsan-x86_64.a.syms           libclang_rt.msan-x86_64.a.syms                libclang_rt.ubsan_minimal-x86_64.so
libclang_rt.dyndd-x86_64.so               libclang_rt.msan_cxx-x86_64.a                 libclang_rt.ubsan_standalone-x86_64.a
libclang_rt.fuzzer-x86_64.a               libclang_rt.msan_cxx-x86_64.a.syms            libclang_rt.ubsan_standalone-x86_64.a.syms
libclang_rt.fuzzer_interceptors-x86_64.a  libclang_rt.orc-x86_64.a                      libclang_rt.ubsan_standalone-x86_64.so
libclang_rt.fuzzer_no_main-x86_64.a       libclang_rt.profile-x86_64.a                  libclang_rt.ubsan_standalone_cxx-x86_64.a
libclang_rt.gwp_asan-x86_64.a             libclang_rt.safestack-x86_64.a                libclang_rt.ubsan_standalone_cxx-x86_64.a.syms
libclang_rt.hwasan-x86_64.a               libclang_rt.scudo-x86_64.a                    libclang_rt.xray-basic-x86_64.a
libclang_rt.hwasan-x86_64.a.syms          libclang_rt.scudo-x86_64.so                   libclang_rt.xray-fdr-x86_64.a
libclang_rt.hwasan-x86_64.so              libclang_rt.scudo_cxx-x86_64.a                libclang_rt.xray-profiling-x86_64.a
libclang_rt.hwasan_aliases-x86_64.a       libclang_rt.scudo_cxx_minimal-x86_64.a        libclang_rt.xray-x86_64.a

It seems that there is no library file for aarh64, all for x86_64. I am not sure whether runtime library is platform dependent or not, should there be the runtime library for aarch64, e.g., libclang_rt.builtins-aarch64.a ?

Below is the cmake command which I use to build llvm from source code.

cmake -S llvm -B build -G "Unix Makefiles" -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;compiler-rt;lld" -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi" -DCMAKE_INSTALL_PREFIX=/usr/local/llvm-${LLVM_VERSION}/ -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD="ARM;X86;AArch64"
#4

You need to tell the runtime builds what targets it should build:

-DLLVM_RUNTIME_TARGETS=x86_64-unknown-linux-gnu;aarch64-unknown-linux-gnu for example. But that most likely means you need to have a aarch64 sysroot and support headers.

You can specify variables like this:

-DRUNTIMES_aarch64-unknown-linux-gnu_CMAKE_SYSROOT=<path>

I think the best example for this is the Fuchsia caches, these are a bit complicated, but hopefully would get the idea across:

llvm-project/Fuchsia-stage2.cmake at main · llvm/llvm-project (github.com)

#5

Oh actually if you only want builtins for aarch64 you can specify:
-DLLVM_BUILTINS_TARGETS=aarch64-unknown-linux-gnu and -DBUILTINS_aarch64-unknown-linux-gnu_CMAKE_SYSROOT=<path>

if you are bootstrapping your own compiler for arm64/musl - be prepared for some pain and undocumented stuff though. I really should re-create some of my scripts from my old work and publish this, since they worked through all this.