I’m trying to use lit for a project, but when building LLVM from sources I can’t find llvm-lit among the built binaries. I built with the following flags:
-DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_LINK_LLVM_DYLIB=ON -DLLVM_INSTALL_UTILS=ON -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;compiler-rt;libclc;libcxx;libcxxabi;lld;lldb;mlir;openmp"
I then installed it using pip, but the AddLLVM CMake configuration file still searches in the llvm_install_path/bin folder. What am I missing? To be honest, I can’t also understand what that llvm-lit executable should do, apart from being a simple wrapper to the pip-installed package.
I think the current practice is that llvm-lit and lit are only used at build time. After you install llvm, then there’s not an easy mechanism to use llvm-lit to run tests. I think this is a bit of a whole in a number of scenarios, not the least of which is when cross-compiling llvm, where it is very natural to want to run tests on an installed area. I’ve been experimenting with how to enable this, but nothing has made it upstream yet. Is this more or less what you’re trying to do?
Well not exactly. I’ve my own project and some tests that I’d like to run using lit, and thus I’m using the add_lit_testsuites function provided by AddLLVM.cmake file. Problem is that it is going to search for llvm-lit inside the llvm_install_path/bin folder, where there is none once installed. From your answer it seems to me that what I’m doing should be done only for projects living inside the LLVM tree, and thus built together with it, while I should instead just rely on the cmake’s add_test function and provide a custom command to run the pip-installed lit. Am I understanding right?
EDIT: I see now this: llvm-project/mlir/examples/standalone at main · llvm/llvm-project · GitHub
Despite having look into the toy example many times I always missed the standalone example. I will try with the external lit parameter. I think if would be cool though to also have lit as auto detected if installed through pip
I think the easy solution right now is to build your project against a build tree of LLVM, rather than an install tree. I think the more right solution is to install llvm-lit into the install tree. There are a bunch of python things that are becoming more important (like python APIs for LLVM and MLIR, so I think that embracing more python stuff (like llvm-lit) in installations is good. (Now I just need to convince more people about this 
If you’re building against an llvm build tree, then there should be a reasonable default and specifying LLVM_EXTERNAL_LIT shouldn’t be necessary. I’m not quite clear about where pip comes in: are you installing lit through pip? (note that llmv-lit is a small wrapper around lit that does some extra stuff and I don’t think comes with lit proper.
Yes, I installed lit through pip because at a first attempt I couldn’t manage to find llvm-lit. Only by reading your message I realized llvm-lit is only for build trees. I will think a bit about building against a build tree; despite the fact future people that will work on my project will have to build llvm anyway (and thus would have llvm-lit) I don’t feel it as a fully elegant solution, but that’s just my opinion that can easily be wrong. May I ask you what are the reasons about the non-inclusion of llvm-lit in install trees, and its additional features with respect to lit?
I have no idea… I think it’s an oversight that llvm-lit is not installed (at least optionally). I think it’s important to be able to run tests on an install area.
To answer your other question, llvm-lit is a small wrapper that picks up a few configuration files that are generated by LLVM’s cmake system.
I believe that the installation of LLVM is intended for “shipping”, and we don’t ship the test and the test infrastructure. So for “testing” we use a built-tree of LLVM instead of a packaged version.
As Stephen was mentioning, what about cross-compiling? I would agree with him about running the tests on the target machine 
I don’t quite see the relationship with cross compiling right now. The way I’ve been running tests on an ARM64 device from my X86 laptop was to use the remote features in lit: from my cross compiling build directory I would invoke lit with the SSH configuration to the device and it would handle remote-running the tests (that was 5 years ago, I don’t remember the exact setup).