Installing LLVM cmake targets into different root directories

I’m maintaining a downstream compiler toolchain that has found the need to install LLVM and its utilities multiple times into different root directories. That is:

  • Build LLVM and friends once
  • Install a number of targets to /path/to/root1
  • Install a number of non-identical targets to /a/different/root2

Effectively, I’d like the behavior of the illegal command
cmake -DCMAKE_INSTALL_PREFIX=/path/to/root1 --build <llvm> --target install-runtimes

I’ve gotten this to work when I know the exact components using the basic cmake -P cmake_install.cmake, but this requires prior knowledge of every component my options create. This is less-than-ideal for the components generated by bootstrap runtimes builds, since those can often be programmatic in nature, and defined in a cache file.

This is easily solvable by building LLVM more than once, but with large/slow Debug builds, and generally wasted CPU time, I’d like to avoid that.

I’ve found support for “multi-distributions”

https://llvm.org/docs/BuildingADistribution.html#multi-distribution-configurations

But this doesn’t seem to mention how one would change the install prefix of the new install-${distribution}-distribution targets. That would mean, then, that all install-${distribution}-distribution targets would be installed into the same location, and that seems wrong to me. What would be the use of such a feature in that case?

I’ve found that I can install individual components via externally executing 'cmake --install --prefix , but that only works on components, not targets like ‘runtimes’ or ‘install-distribution’, whose install steps are comprised of many components.

Is there something I’m missing, either in stock cmake or the LLVM build system?

If it helps/hurts, I’m building LLVM from my own mega-project composed of a number of ExternalProjects.

We’ve used the DESTDIR environment variable for this purpose; we set our CMAKE_INSTALL_PREFIX to empty and solely use DESTDIR to control the installation path. With the multi-distribution setup, you could then issue commands for e.g. DESTDIR=/foo cmake --target install-foo-distribution and DESTDIR=/bar cmake --target install-bar-distribution. Would that work for you?

DESTDIR is Unix-only, but our builds currently happen on multiple platforms (I have a desire to move to cross-compiling every platform on Unix, but alas, no time and so much restructuring to do…).

However, if it’s just the case that DESTDIR doesn’t work on Windows because CMAKE_INSTALL_PREFIX begins with a drive letter, then I can do the same, set CMAKE_INSTALL_PREFIX to empty, and then DESTDIR might work as expected? Sounds somewhat hacky, but there’s potential…

Looks like maybe that might be the case. There’s no obvious check for Unix before prepending $ENV{DESTDIR}. Of course, with the complexity of cmake, it might be elsewhere! :slight_smile:

Currently suspecting DESTDIR will break on Windows. I must set it to ‘/’, an absolute path, otherwise cmake treats it as relative and gives me /path/to/build/dir.

Another issue I’ve come across, that I’m wondering if you’ve ever seen since you’ve enabled the MultiDistribution support.

I’m currently following your example closely, and have added cxx-headers to LLVM_RUNTIME_DISTRIBUTION_COMPONENTS as well as to my particular distribution, but the libcxx build system is not generating a ‘cxx-headers’ target, causing a failure.

There exist ‘generate-cxx-headers’ and ‘install-cxx-headers’, but the interface library ‘cxx-headers’ does not get a build target. Would you have any idea why this might be the case? I’ve tried this on both cmake 3.17 as well as 3.24.

-DCMAKE_INSTALL_PREFIX=

Seems to correctly set the prefix to completely empty. I was being confused because of the default behavior of ExternProject INSTALL_DIR specification. So, I might be ok on windows systems using DESTDIR.

Still, I have this issue with cxx-headers not existing as a buildable target. Maybe you have some insight?

Glad you figured out DESTDIR on Windows! I haven’t tried that myself, but I also didn’t notice any OS checks around that logic in CMake (despite what the documentation said).

What failure message are you seeing around the cxx-headers target? That should be generated by llvm-project/CMakeLists.txt at 0cbaed3e14448de4b9ed32b7f531471bc6d4347f · llvm/llvm-project · GitHub. It’s an INTERFACE target so I wouldn’t expect it to show up in the build system, but CMake should still be able to handle it correctly for the purposes of the distribution setup.

EDIT: Ah, I think I see the problem. When we build the distribution, we’ll explicitly try to build the cxx-headers target, which will fail because it’s an INTERFACE target. Installing the distribution will still work because install-cxx-headers exists, but that’s not ideal. Lemme think about a good way to avoid this.

Glad you were able to reproduce! I’m in discord talking about it as well (started in #libcxx, now #build_systems).

I’ve worked around this myself by adding downstream a separate build target that depends on cxx-headers (the interface library), and adding an analogous install/install-stripped target. I wanted to be careful with my change because this exact code shape is shared between a couple of the runtimes (see libcxxabi/include/CMakeLists.txt).

@DragonDisciple we should discuss this because my distro, NixOS (and its underlying package set Nixpkgs, we are also a homebrew replacement among other things) does quite fine-grained standalone builds of LLVM (all projects built separately, including libcxx, libcxxabi, and libunwind separately) and everything gets installed to different disjoint paths.

I have been working on adding “GNU Install Dirs” support for LLVM so custom paths work better and more idiomatically. I have also made ⚙ D132298 [libcxxabi][cmake] Allow building without libcxx again to restore the ability to build libcxxabi and libcxx separately.