Running llvm test-suite

Hi llvm-dev,

Are we still able to run llvm test-suite without LNT?

Also, is there any example of configuring test-suite to run against 3rd party backend rather than llc?

Thanks
Hongbin

Yes the test-suite can be run without lnt (you can always just look at the LNT logfiles to see what commands it is running and run those yourself).

Currently I would recommend using the cmake/lit version of the test-suite. You typically do something like:

mkdir build && cd build
cmake -GNinja -DCMAKE_C_COMPILER=/path/to/my/clang -DTEST_SUITE_ARCH_FLAGS="-arch arm64" /path/to/test-suite
ninja # to build the benchmarks
llvm-lit # to run the benchmarks

You should check "ccmake"/"cmake -LA" to learn about the available options as well as llvm-lit. You can either use llvm-lit from an llvm build or use the pypy packaged version (pip install lit). The cmake/lit version has no support to run custom backends through llc at the moment, you need to have a clang version that accepts -arch XXX for your target.

- Matthias

Hi Matthias,

Thanks a lot for the explanation.

It’s probably possible by using one of the existing TEST.* files or writing your own. Unfortunately this system can get complicated and is not documented much. I don’t really know how it works.

An alternative to this could be that you write your own little python/shell script that acts like a compiler on the outside but internally translates to clang, opt, llc commands you want. You could point CMAKE_C_COMPILER to that script.

  • Matthias