enabling virtual environment for multiple clang versions

I now often have to switch between several versions of LLVM including the ones built locally when benchmarking and optimizing different AI libraries, and it's kind of a pain to substitute cmake or environment variables.

That’s why we decided to automate this process by introducing "virtual environments" for LLVM and related tools similar to Python virtualenv in the CK framework (GitHub - mlcommons/ck: This toolkit helps you to add simple, human-readable and platform-independent CLI, API and metadata to ad-hoc MLOps and DevOps scripts and artifacts to make them more portable, reusable, interoperable, customizable and deterministic across continuously changing software & hardware. It is used to modularize ML Systems and AI.). We also make it work in the same way on Linux, MacOS and Windows. If it’s of interest, you can try it as follows:

$ (sudo) pip install ck
$ ck pull repo:ck-env

$ ck detect soft --tags=compiler,llvm

CK will search for all installed clang instances on your machine (including Microsoft Visual Studio dependency on Windows) and will ask you which one to register for the CK virtual environment. You can then repeat this process and register multiple versions you use for testing. You can then see all registered virtual environments as follows:

$ ck show env
$ ck show env --tags=compiler,llvm

Now you can start a specific virtual environment as follows:

$ ck virtual env --tags=compiler,llvm

CK will set up PATH, LD_LIBRARY_PATH and other variables to point to a specific clang version, and will start bash on Linux/MacOS or shell on Windows. You can then use different environment variable specific to a given Clang instance and starting from CK_ in your own scripts:

$ export | grep "CK_"

We also added the possibility to install pre-built versions of LLVM on different platforms and automatically register CK virtual environment:

$ ck search package --tags=compiler,llvm
$ ck install package --tags=compiler,llvm,v6.0.0
$ ck show env --tags=compiler,llvm
$ ck virtual env --tags=compiler,llvm,v6.0.0
> ${CK_CC_FULL_PATH} --version

Finally, you can also rebuild LLVM from a trunk and again automatically register it in the CK virtual environment:

$ ck install package --tags=compiler,llvm,vtrunk

Hope it will be of some help - if you are interested to know more, feel free to check this page or to get in touch: First steps · mlcommons/ck Wiki · GitHub

Grigori Fursin