I’ve been trying to follow the Testing libc++ documentation in order to exercise some tests against a modified glibc installation. Unfortunately the documentation seems to be incorrect or incomplete, particularly in the Usage section.
In order to preempt other “did you turn it on” questions, I’ll state that
make check-cxx works perfectly fine for me.
First, the documentation suggests running
make -C <build> cxx-test-depends from the root of the monorepo.
<build> is presumably intended to mean the top of the build directory for the full LLVM build (e.g., the current directory when
cmake was run unless the
-B option was specified). This is implied by the follow on commands that indicate that
llvm-lit is expected to be found in the
<buid>/bin directory. However, the
cxx-test-depends is not recognized at that level.
$ make -C ../llvm-project-build cxx-test-depends make: Entering directory '/.../llvm-project-build' make: *** No rule to make target 'cxx-test-depends'. Stop. make: Leaving directory '/.../llvm-project-build'
<build>/bin/llvm-lit commands that follow that in the documentation don’t seem to work for me. I was eventually able to figure out how to get an individual test to run, but instead of doing as the documentation indicated:
$ ../llvm-project-build/bin/llvm-lit -sv libcxx/test/std/depr/depr.c.headers/stdlib_h.pass.cpp llvm-lit: /home/tom/projects/llvm-project/libcxx/test/lit.cfg.py:7: fatal: You seem to be running Lit directly -- you should be running Lit through <build>/bin/llvm-lit, which will ensure that the right Lit configuration file is used.
I had to do:
$ ../llvm-project-build/bin/llvm-lit -sv ../llvm-project-build/runtimes/runtimes-bins/libcxx/test/std/depr/depr.c.headers/stdlib_h.pass.cpp ... Testing Time: 0.17s Passed: 1
Finally, once I figured that out, I wanted to try running tests with an alternate system include root. The documentation suggests doing something like:
<build>/bin/llvm-lit ... --param=compile_flags='-isystem /path/to/include'
But that seemed to have no effect. I was eventually able to do the testing I wanted to do by modifying the
runtimes/runtimes-bins/libcxx/test/lit.site.cfg file in the
<build> directory, but I suspect a better method exists.
I’ll be happy to submit a patch for the documentation, but will need guidance on what the best practices actually are for doing one off testing like this these days.