libcxxabi cmake test problem on Linux x86-64

Ismail_Donmez · July 11, 2014, 5:29pm

Hi,

Unless I apply the patch below exception tests fail on Linux x86-64.

— libcxxabi.orig/src/CMakeLists.txt
+++ libcxxabi/src/CMakeLists.txt
@@ -48,7 +48,7 @@ include_directories("${LIBCXXABI_LIBCXX_
set(libraries ${LIBCXXABI_CXX_ABI_LIBRARIES})
append_if(libraries LIBCXXABI_HAS_C_LIB c)

-target_link_libraries(cxxabi ${libraries})
+target_link_libraries(cxxabi ${libraries} gcc_s pthread)

Setup flags.

append_if(compile_flags LIBCXXABI_HAS_FPIC_FLAG -fPIC)

Dan_Albert · July 11, 2014, 6:56pm

How are you building/running the tests? test/lit.cfg will link gcc_s into each test executable. If you’ve enabled the llvm unwinder, the tests are expected to fail on Linux.

Dan

Ismail_Donmez · July 11, 2014, 7:18pm

I am not enabling the llwn unwinder. Compiling with:

Dan_Albert · July 11, 2014, 7:28pm

Hmm. Just tested ToT libcxxabi and it passes for me.

When running make check-libcxxabi, you should see the following line:
lit.py: lit.cfg:215: note: inferred link_flags as: [’-lc++abi’, ‘-lgcc_eh’, ‘-lc’, ‘-lm’, ‘-lpthread’, ‘-lgcc_s’]

Ismail_Donmez · July 12, 2014, 5:59am

Hi,

If I disable ld’s --as-needed support [0] tests pass for me too. But this means we are underlinking. For example pthread symbols are used in libcxxabi so we must link to pthread explicitly. Same for unwinding support those symbols are defined in gcc_s.

[0] http://wiki.gentoo.org/wiki/Project:Quality_Assurance/As-needed#What_is_–as-needed.3F

Dan_Albert · July 12, 2014, 8:59pm

Ah. Okay, that makes sense. Odd that I don’t see it though.

-lpthread is fine to include unconditionally, but -lgcc_s should be linked based on LIBCXXABI_USE_LLVM_UNWINDER (if true, link libunwind, else link libgcc_s). libgcc is unfortunately far reaching, in that it has all of the compiler replacement functions and the unwinder, so that may not be quite correct. We may need to do -lunwind -lcompiler-rt in the case of LIBCXXABI_USE_LLVM_UNWINDER=ON, else -lgcc_s.

Ismail_Donmez · July 13, 2014, 9:00am

Hi,

How about the attached patch?

libcxxabi-libs.patch (1.04 KB)

Dan_Albert · July 14, 2014, 5:10pm

Submitted (with some slight modifications, s/libgcc_s/libgcc_eh/) as r212958. Tested with and without the LLVM unwinder, and things look good. Thanks again!

Ismail_Donmez · July 16, 2014, 11:10am

Hi,

thakis · July 16, 2014, 3:43pm

Hi,

Submitted (with some slight modifications, s/libgcc_s/libgcc_eh/) as
r212958. Tested with and without the LLVM unwinder, and things look good.
Thanks again!

Changing gcc_s with gcc_eh doesn't look good. libcxxabi tests pass but
llvm MCJIT exception tests fail with:

"terminating with uncaught exception of type int"

Can you try to make a reduced test case for libcxxabi from this?

Ismail_Donmez · July 17, 2014, 8:05am

Hi,

Ismail_Donmez · August 16, 2014, 5:24am

Ping?

Evgenii_Stepanov · November 26, 2014, 12:20pm

I've got the same issue.
On Linux (Trusty/x86_64), build clang with libc++/libc++abi (just
plain ninja clang cxx cxxabi w/o any special cmake flags), then use it
to bootstrap clang like this:

CC=/code/llvm/build2/bin/clang \
CXX=/code/llvm/build2/bin/clang++ \
LDFLAGS="-lc++abi -L/code/llvm/build2/lib -Wl,-rpath,/code/llvm/build2/lib" \
cmake \
  -GNinja \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_ENABLE_LIBCXX=ON \
  ..

In the second tree, MCJIT tests fail with the same error:

./bin/lli -relocation-model=pic -code-model=large
../test/ExecutionEngine/MCJIT/eh-lg-pic.ll
terminating with uncaught exception of type int
0 lli 0x0000000000a80a9a
_ZN4llvm3sys15PrintStackTraceEP8_IO_FILE + 42
1 lli 0x0000000000a81eeb
2 libpthread.so.0 0x00007f83baabf340
3 libc.so.6 0x00007f83b9d2fbb9 gsignal + 57
4 libc.so.6 0x00007f83b9d32fc8 abort + 328
5 libc++abi.so.1 0x00007f83bb5121a7
6 libc++abi.so.1 0x00007f83bb512355
7 libc++abi.so.1 0x00007f83bb55c3b3
8 libc++abi.so.1 0x00007f83bb55bb66 __cxa_throw + 166
9 libc++abi.so.1 0x00007f83bb98802d __cxa_throw + 4375917
10 libc++abi.so.1 0x00007f83bb98803d __cxa_throw + 4375933
11 lli 0x0000000000849649
_ZN4llvm5MCJIT11runFunctionEPNS_8FunctionERKNSt3__16vectorINS_12GenericValueENS3_9allocatorIS5_EEEE
+ 1209
12 lli 0x00000000007ecd08
_ZN4llvm15ExecutionEngine17runFunctionAsMainEPNS_8FunctionERKNSt3__16vectorINS3_12basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEENS8_ISA_EEEEPKPKc
+ 1192
13 lli 0x000000000050c434 main + 10404
14 libc.so.6 0x00007f83b9d1aec5 __libc_start_main + 245
15 lli 0x00000000005092fb
Stack dump:
0. Program arguments: ./bin/lli -relocation-model=pic
-code-model=large ../test/ExecutionEngine/MCJIT/eh-lg-pic.ll
Aborted (core dumped)

EricWF · November 26, 2014, 8:37pm

Evgeniy,

Would you be willing to reproduce this again but in debug mode instead
of release so we get a better stack trace?

/Eric

Evgenii_Stepanov · November 26, 2014, 9:04pm

#0 0x00007ffff62b5bb9 in __GI_raise (sig=sig@entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff62b8fc8 in __GI_abort () at abort.c:89
#2 0x00007ffff7b60807 in abort_message (format=0x7ffff7bcdf13
"terminating with %s exception of type %s") at
../projects/libcxxabi/src/abort_message.cpp:66
#3 0x00007ffff7b60ad6 in default_terminate_handler () at
../projects/libcxxabi/src/cxa_default_handlers.cpp:68
#4 0x00007ffff7bc454e in std::__terminate (func=0x7ffff7b60900
<default_terminate_handler()>) at
../projects/libcxxabi/src/cxa_handlers.cpp:68
#5 0x00007ffff7bc31dd in __cxxabiv1::failed_throw
(exception_header=0x1e87240) at
../projects/libcxxabi/src/cxa_exception.cpp:149
#6 0x00007ffff7bc3136 in __cxa_throw (thrown_object=0x1e872b8,
tinfo=0x7ffff7dd83b0 <typeinfo for int>, dest=0x0) at
../projects/libcxxabi/src/cxa_exception.cpp:244
#7 0x00007ffff7ff502d in throwException ()
#8 0x00007ffff7ff503d in main ()
#9 0x0000000000beef04 in llvm::MCJIT::runFunction (this=0x1e74c30,
F=0x1e6b4b0, ArgValues=...) at
../lib/ExecutionEngine/MCJIT/MCJIT.cpp:500
#10 0x0000000000b26f90 in llvm::ExecutionEngine::runFunctionAsMain
(this=0x1e74c30, Fn=0x1e6b4b0, argv=..., envp=0x7fffffffe0d0)
at ../lib/ExecutionEngine/ExecutionEngine.cpp:393
#11 0x0000000000567cc4 in main (argc=4, argv=0x7fffffffe0a8,
envp=0x7fffffffe0d0) at ../tools/lli/lli.cpp:613

Chandler_Carruth · December 9, 2015, 10:06am

(Moving to the new mailing list)

#0 0x00007ffff62b5bb9 in __GI_raise (sig=sig@entry=6) at
…/nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff62b8fc8 in __GI_abort () at abort.c:89
#2 0x00007ffff7b60807 in abort_message (format=0x7ffff7bcdf13
“terminating with %s exception of type %s”) at
…/projects/libcxxabi/src/abort_message.cpp:66
#3 0x00007ffff7b60ad6 in default_terminate_handler () at
…/projects/libcxxabi/src/cxa_default_handlers.cpp:68
#4 0x00007ffff7bc454e in std::__terminate (func=0x7ffff7b60900
<default_terminate_handler()>) at
…/projects/libcxxabi/src/cxa_handlers.cpp:68
#5 0x00007ffff7bc31dd in __cxxabiv1::failed_throw
(exception_header=0x1e87240) at
…/projects/libcxxabi/src/cxa_exception.cpp:149
#6 0x00007ffff7bc3136 in __cxa_throw (thrown_object=0x1e872b8,
tinfo=0x7ffff7dd83b0 , dest=0x0) at
…/projects/libcxxabi/src/cxa_exception.cpp:244
#7 0x00007ffff7ff502d in throwException ()
#8 0x00007ffff7ff503d in main ()
#9 0x0000000000beef04 in llvm::MCJIT::runFunction (this=0x1e74c30,
F=0x1e6b4b0, ArgValues=…) at
…/lib/ExecutionEngine/MCJIT/MCJIT.cpp:500
#10 0x0000000000b26f90 in llvm::ExecutionEngine::runFunctionAsMain
(this=0x1e74c30, Fn=0x1e6b4b0, argv=…, envp=0x7fffffffe0d0)
at …/lib/ExecutionEngine/ExecutionEngine.cpp:393
#11 0x0000000000567cc4 in main (argc=4, argv=0x7fffffffe0a8,
envp=0x7fffffffe0d0) at …/tools/lli/lli.cpp:613

Any update here Eric? Or others? Would really like to get the bootstrap clean with libc++abi.

EricWF · December 9, 2015, 7:21pm

I don’t remember what’s going on here but alot has changed in a year. I should be able to look into it later today.

Chandler, I’m assuming your still able to reproduce the issue?

/Eric

Chandler_Carruth · December 9, 2015, 7:27pm

Yep.

EricWF · December 11, 2015, 1:26am

I’ve spent most of today looking into this and I still have no clue what’s going on. I think I need to know more about how lli provides libc++.

I was able to reproduce the bug using the following steps:

export CC=clang CXX=clang++
cmake --DLLVM_ENABLE_LIBCXX=ON -DLLVM_BUILD_PROJECTS=OFF…/llvm
./bin/lli llvm/test/ExecutionEngine/MCJIT/eh.ll

The above steps assume that you already have libc++.so installed somewhere the linker can find it
and that libc++.so is a linker script that includes -lc++abi.

I turned LLVM_BUILD_PROJECTS off so that we aren’t contending with two different libc++abi binaries.

Any help with this bug would be appreciated.

/Eric

EricWF · December 11, 2015, 3:50am

Alright, so the problem is that libgcc_eh.a marks most of the _Unwind symbols as hidden. This means that when libgcc_eh.a is linked into a shared library exceptions cannot be thrown beyond the library boundary. I think that both libgcc_s and libgcc_eh were being linked into lli causing the failures. Since we typically build libc++abi as a shared library we need to use libgcc_s.so instead. (I think we will still need to use libgcc_eh when creating a static libc++abi)

I think that libc++abi should link the default libraries exactly as clang would if we didn’t specify '-nodefaultlibs`. This resulting link line for libc++abi.so should look something like:

-lm -lgcc_s -lpthread -lc -lgcc_s

The resulting libc++abi.so will still have undefined references to libgcc.a’s _Unwind symbols but libgcc.a should always be linked into the final executable.
When using and testing libc++abi (via libc++) the link line should look something like:

-lc++ -lm -lgcc_s -lgcc -lpthread -lc -lgcc_s -lgcc

When we want to use llvm’s unwinder we will probably need to use -lunwind -lcompiler-rt instead but I’m not too familiar with this configuration.

I’ll put a patch together tonight.

/Eric