libc++ pretty printer test dependencies

Subprojects LLDB
1

So I’m trying to run a clean “ninja check-lldb” and I’m running into some difficulties with the libc++ pretty printer tests.

  1. They’re “unsupported” if my host compiler is gcc:

$ ninja check-lldb-api-functionalities-data-formatter-data-formatter-stl-libcxx

[2/3] Running lit suite …/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx

Testing Time: 1.57s

Unsupported: 23

Looking at the logs (see other thread: perhaps those logs should actually be part of the test output - especially for buildbots where you’d have no ability to read separate log files): "

unittest2.case.SkipTest: could not find library matching ‘libc++’ in target a.out"

So, looks like it built with the just-built clang, but without libc++? (since libc++ isn’t built yet in this tree) - but the logs don’t show the commands that built the binary - should I be able to find that somewhere? It seems important for debugging exactly what’s under test, etc.

  1. Oh, but if I explicitly ninja cxx the tests fail instead of “unsupported”

Now they fail, rather than unsupported. The log isn’t especially helpful so far as I can see:

…
runCmd: setting set target.prefer-dynamic-value no-dynamic-values

output:

FAIL

<bound method SBProcess.Kill of <lldb.SBProcess; proxy of <Swig Object of type ‘lldb::SBProcess *’ at 0x7fb67858ecf0> >>: success

Traceback (most recent call last):

File “/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbtest.py”, line 1823, in test_method

return attrvalue(self)

File “/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/vector/TestDataFormatterLibcxxVector.py”, line 53, in test_with_run_command

(self.target, process, thread, bkpt) = lldbutil.run_to_source_breakpoint(

File “/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py”, line 970, in run_to_source_breakpoint

return run_to_breakpoint_do_run(test, target, breakpoint, launch_info,

File “/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py”, line 892, in run_to_breakpoint_do_run

test.assertEqual(process.GetState(), lldb.eStateStopped)

AssertionError: 10 != 5

Config=x86_64-/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang

Session info generated @ Sun Oct 17 22:23:34 2021

But if I run lldb on the binary (which isn’t mentioned in the logs… probably should be?) under test:

$ ./bin/lldb ./lldb-test-build.noindex/functionalities/data-formatter/data-formatter-stl/libcxx/vector/TestDataFormatterLibcxxVector.test_ref_and_ptr_dwarf/a.out

(lldb) target create “./lldb-test-build.noindex/functionalities/data-formatter/data-formatter-stl/libcxx/vector/TestDataFormatterLibcxxVector.test_ref_and_ptr_dwarf/a.out”

Current executable set to ‘build/release/lldb-test-build.noindex/functionalities/data-formatter/data-formatter-stl/libcxx/vector/TestDataFormatterLibcxxVector.test_ref_and_ptr_dwarf/a.out’ (x86_64).

(lldb) r

Process 1896861 launched: ‘build/release/lldb-test-build.noindex/functionalities/data-formatter/data-formatter-stl/libcxx/vector/TestDataFormatterLibcxxVector.test_ref_and_ptr_dwarf/a.out’ (x86_64)

build/release/lldb-test-build.noindex/functionalities/data-formatter/data-formatter-stl/libcxx/vector/TestDataFormatterLibcxxVector.test_ref_and_ptr_dwarf/a.out: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or directory

Process 1896861 exited with status = 127 (0x0000007f)

OK, so this looks like it’s related to libc++.so.1 not being in the ld library path - but there’s no rpath or LD_LIBRARY_PATH to find the library.

The libc++ tests build test binaries with “-Wl,-rpath,/usr/local/google/home/blaikie/dev/llvm/build/release/./lib/x86_64-unknown-linux-gnu”

Ah, here we go - by modifying the test’s source file so it would fail to compile, I am able to observe the compilation command:

build/release/bin/clang -std=c++11 -g -O0 -fno-builtin -m64 -Isrc/lldb/packages/Python/lldbsuite/test/make/…/…/…/…/…/include -Isrc/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/vector -Isrc/lldb/packages/Python/lldbsuite/test/make -include src/lldb/packages/Python/lldbsuite/test/make/test_common.h -fno-limit-debug-info -gsplit-dwarf -O0 -DLLDB_USING_LIBCPP -stdlib=libc++ --driver-mode=g++ -MT main.o -MD -MP -MF main.d -c -o main.o src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/vector/main.cpp

So… how does this work for everyone else? I’m not sure how it’s meant to work.

From some offline discussion with Pavel:

This is generally broken - it either uses the system libc++.so (making the build non-hermetic), or if you don’t enable libc++ in CMake the tests flag as unsupported/don’t fail (though other tests not specifically testing libc++ but using any standard library features are still non-hermetic, they’ll be using the system C++ standard library)

That’s all unfortunate… would love to know if anyone’s got plans/ideas/priorities for fixing this? I guess probably in a similar way to the way the libc++ tests work, explicitly -rpath when building.

2

Ping on this. I came across another/slightly related issue: “check-lldb” ended up running my just-built lldb, but it loaded my locally (home directory) installed lldb.so to run the tests, so I got failures about missing python support (if my locally installed lldb didn’t have missing python support I’m not sure how much harder it would’ve been for me to realize that “check-lldb” wasn’t actually checking the just-bulit lldb)

Anyone encountered this/verify this is a known failure or that I’m holding something especially wrongly?

(@jgorbe, @aprantl, @labath)

3

IIRC you had a similar issue a year ago and then the problem was that you had LD_LIBRARY_PATH set (like in your original message). I don’t know how this works on Linux, but on macOS, the equivalent DYLD_LIBRARY_PATH takes precedence over anything else if dyld can resolve the shared library that way. Which is to say that the behavior you’re describing is expected. There’s also DYLD_FALLBACK_LIBRARY_PATH which, as the name suggest, is only checked if the shared library couldn’t be found using the RPATHs. Maybe the Linux variant works more like the latter?

4

oh, looks like registerSharedLibrariesWithTarget overwrites/hardcodes the LD_LIBRARY_PATH for the test, so any attempts to add to that outside running the tests gets lost, which explains why I can set it correctly and run the test binary directly, but the test still fails.

5

Yeah, sorry I’ve lost that precise context, but it does sound vaguely like some situation I was in.

I upgraded build machines, so whatever carefully balanced situation I had I’ve lost/can’t reproduce so I may be stumbling through the same problems again.

In the situation I’m in at the moment I don’t have LD_LIBRARY_PATH set. The tests aren’t built with rpath. So without a library path or rpath set, I don’t know how the tests would find the libc++.so in any platform, yeah?

6

Checking the failure log (~/dev/llvm/build/default/lldb-test-build.noindex/functionalities/data-formatter/data-formatter-stl/libcxx/vector/TestDataFormatterLibcxxVector.test_with_run_command_dwarf/Failure.log)

"/usr/local/google/home/blaikie/dev/llvm/build/default/bin/clang"  -std=c++11 -g -O0 -m64  -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make/../../../../../include -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/vector -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make -include /usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make/test_common.h -fno-limit-debug-info    -DLLDB_USING_LIBCPP -stdlib=libc++ -O0 --driver-mode=g++ -MT main.o -MD -MP -MF main.d -c -o main.o /usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/vector/main.cpp
"/usr/local/google/home/blaikie/dev/llvm/build/default/bin/clang" main.o -g -O0 -m64  -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make/../../../../../include -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/vector -I/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make -include /usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/make/test_common.h -fno-limit-debug-info     -stdlib=libc++ -Wl,-rpath,/usr/local/google/home/blaikie/dev/llvm/build/default/./lib --driver-mode=g++ -o "a.out"

So it is using the just-built clang, and building with libc++ enabled, but isn’t setting rpath, and lldb test runner is emptying LD_LIBRARY_PATH - so I’m not sure how this is ever mean to work?

7

ah, found the more relevant/recent thread: Some API test failures are really opaque/could be improved - #11 by Teemperor

I’ll continue over there