libc++ failed to link against musl

I'm trying to build LLVM, Clang, LLD, compiler-rt, libc++, libc++abi and libunwind with musl-based toolchain.

The configuration is the following:

     LIBCXX_HAS_MUSL_LIBC=ON
     LIBCXX_HAS_GCC_S_LIB=OFF
     CLANG_DEFAULT_CXX_STDLIB=libc++
     CLANG_DEFAULT_LINKER=lld
     CLANG_DEFAULT_RTLIB=compiler-rt
     LLVM_DEFAULT_TARGET_TRIPLE=x86_64-pc-linux-musl
     LLVM_TARGET_ARCH=x86_64
     LLVM_TARGETS_TO_BUILD=X86

When linking libc++.so there are a lot of undefined references to __cxa_allocate_exception, __cxa_begin_catch, __cxa_end_catch, __cxa_free_exception, __cxa_guard_abort, __cxa_guard_acquire, __cxa_guard_release, __cxa_pure_virtual, __cxa_rethrow, __cxa_throw, __gxx_personality_v0, _Unwind_Resume, vtable for __cxxabiv1::__class_type_info, vtable for __cxxabiv1::__si_class_type_info, vtable for __cxxabiv1::__vmi_class_type_info. The full output of the linking command is too long and is attached gzipped. The linking command is the following:

x86_64-linux-musl-g++ -fPIC -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -ffunction-sections -fdata-sections -O3 -Wl,-z,defs -fPIC -nodefaultlibs -shared -Wl,-soname,libc++.so.1 -o lib/libc++.so.1.0 projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/algorithm.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/any.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/bind.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/chrono.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/condition_variable.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/debug.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/exception.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/functional.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/future.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/hash.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/ios.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/iostream.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/locale.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/memory.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/mutex.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/new.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/optional.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/random.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/regex.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/shared_mutex.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/stdexcept.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/string.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/strstream.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/system_error.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/thread.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/typeinfo.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/utility.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/valarray.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/variant.cpp.o -lpthread -lc -lm -lrt

Please suggest how to fix these linking errors.

Regards,
Dmitry

libcxxlinking.txt.gz (11.8 KB)

I'm trying to build LLVM, Clang, LLD, compiler-rt, libc++, libc++abi and libunwind with musl-based toolchain.

The configuration is the following:

     LIBCXX_HAS_MUSL_LIBC=ON
     LIBCXX_HAS_GCC_S_LIB=OFF
     CLANG_DEFAULT_CXX_STDLIB=libc++
     CLANG_DEFAULT_LINKER=lld
     CLANG_DEFAULT_RTLIB=compiler-rt
     LLVM_DEFAULT_TARGET_TRIPLE=x86_64-pc-linux-musl
     LLVM_TARGET_ARCH=x86_64
     LLVM_TARGETS_TO_BUILD=X86

When linking libc++.so there are a lot of undefined references to __cxa_allocate_exception, __cxa_begin_catch, __cxa_end_catch, __cxa_free_exception, __cxa_guard_abort, __cxa_guard_acquire, __cxa_guard_release, __cxa_pure_virtual, __cxa_rethrow, __cxa_throw, __gxx_personality_v0, _Unwind_Resume, vtable for __cxxabiv1::__class_type_info, vtable for __cxxabiv1::__si_class_type_info, vtable for __cxxabiv1::__vmi_class_type_info. The full output of the linking command is too long and is attached gzipped. The linking command is the following:

Those symbols are from libc++abi, and it isn't on your link line.

Jon

Hi Jonathan,

Thank you very much. I have solved this problem by configuring with additional options:

     LIBCXXABI_LIBCXX_PATH=/path/to/llvm/projects/libcxx
     LIBCXXABI_LIBCXX_INCLUDES=/path/to/llvm/projects/libcxx/include
     LIBCXX_LIBCXXABI_INCLUDES_INTERNAL=/path/to/llvm/projects/libcxxabi/include

I wonder why those paths were not detected automatically.

Regards,
Dmitry

It depends on how you want to package libc++ whether you actually want to link to libc++abi. If you’re on a GNUish system, you almost certainly don’t, because you then won’t be able to combine libc++ and libstdc++ in the same binary. You want to add -lstdc++ to the link line (or, ideally, -lsupc++, if your distribution ships a separate libsupc++.so).

On FreeBSD, we don’t link libc++ to any ABI library. Instead, both libc++.so and libstdc++.so are linker scripts that instruct anything that tries to link libc++ to link libc++.so.{version} and libcxxrt.so.{version}.

David

Hi David,

Neither is the case. The system that I want to build with this toolchain is Linux-based, but not GNUish. I would like to use musl instead of glibc and libc++ instead of libstdc++, only use binutils provided by LLVM. I think that in that case I will link libc++abi and libunwind to libc++ statically, so I will not have to add -lunwind and -lc++abi to my LDFLAGS. Is it okay to do that? There is an experimental LIBCXX_ENABLE_STATIC_ABI_LIBRARY option, I will try to enable it.

The host system that I'm compiling on is Alpine Linux (musl-based distro), if it matters.

Regards,
Dmitry

Note that this is not 100% true, our static libc++.a contains objects from libcxxrt. But this was unavoidable, at least not if you didn't want to inconvenience users.

-Dimitry

It probably makes sense for libc++abi or libcxxrt, but it makes less sense for libunwind because C libraries that need to interoperate with C++ will link with this (by default assuming libgcc_s.so provides the relevant symbols). If both do then you’re going to have conflicts, if libgcc_s.so or equivalent doesn’t then you’re going to have link failures in pure-C programs that use C/C++ compatible libraries.

David

As far as I can understand, all the functions of libgcc are provided by compiler-rt and libunwind, so it is possible to build a toolchain that will not have any parts of GCC at all. For now my toolchain can only produce statically liked binaries, I have some trouble with shared (https://bugs.llvm.org/show_bug.cgi?id=32425#c8), and it only works for C, the C++ test program segfaults.

Regards,
Dmitry

As far as I can understand, all the functions of libgcc are provided by compiler-rt and libunwind, so it is possible to build a toolchain that will not have any parts of GCC at all.

Correct, though on FreeBSD we still install the libunwind / compiler-rt combination as libgcc_s.so, because then we don’t break existing binaries or compilers.

For now my toolchain can only produce statically liked binaries, I have some trouble with shared (32425 – Regression with Musl after adding support for outputting to /dev/null), and it only works for C, the C++ test program segfaults.

You might find it informative to look at how the FreeBSD toolchain works.

David