I am sending this email after some time consuming exploration into how
LLVM executes its tests. I first noticed this issue when Rafael
noticed that my patch I sent in to llvm-commits broke some tests.
I was initially very skeptical, as I had run the make check test from
the build directory before sending in the patch, but as it turns out,
several issues came together in a perfect storm to give a royal
punkin' to yours truly.
After several time-consuming make clean/make check cycles among my
various llvm build directories, I began to doubt my sanity when across
certain recompile make clean/make check operation pairs on the SAME
SOURCE directory, different tests began to fail from the list by
Rafael, or sometimes, NO tests failed (and in one case, 100% of tests
The answer turned out to be both simpler and more complex than I had
imagined. Whenever you have complex integration tests that call out to
various utilities, it is wise to make sure that at the very least, the
locations of said utilities are anchored to specific directories.
At least at first it seemed to be the case, according to the
autogenerated test/lit.site.config file in the individual llvm build
directories, recreated here for your reading pleasure: