Removing clang version number from resource dir in a distribution

Hi,

Sony’s PS4 and PS5 LLVM/Clang distributions have omitted the LLVM version from resource paths to make it easier to integrate with other parts of our SDK (e.g. IDE). This has been implemented as a local change but we’re considering whether we can remove this. Setting CLANG_RESOURCE_DIR CMake variable to “../lib/clang” seems to work. This results in a slightly untidy include path with “..” in (e.g. <path>\\bin\\../lib/clang\\include) but I think this would be easy to improve.

If I understand correctly the version in the resource path is to support having multiple versioned compilers (e.g. clang-21, clang-22 etc.), and each being able to find their own resource files. I’ve looked at a few distributions that I have access to and it seems more common to deliver multiple toolchains as separate folders (e.g. 21/bin/clang, 22/bin/clang). On Windows, (based on https://github.com/llvm/llvm-project/releases/download/llvmorg-22.1.0/LLVM-22.1.0-win64.exe), we don’t even ship clang-22.exe. Yet distributions still seem to include the version in the resource path. I think @jyknight was making this point here: https://discourse.llvm.org/t/should-we-continue-embed-the-full-llvm-version-in-lib-clang/62094/9.

I’m wondering:

  • Have other distributors considered removing the clang version from the resource path?
  • Is there a reason to keep the clang version in for distributions that just have one compiler in? Is it just conformity or is there a technical reason?
  • Is there any reason CLANG_RESOURCE_DIR=../lib/clang CMake variable shouldn’t be used in a distribution build?

Thanks
Russ

1 Like

Ping. Any comments from other distributors (@petrhosek , @smithp35, @hansw2000 , @mstorsjo, @tstellar)?

Edit: I understand no-one can guarantee this will work. I’m just hoping to learn whether this approach has any known problems.

Thanks

The libraries are expected to always match the compiler version, so this gives us a way to enforce that. If you want to move it to a non-standard location you could possibly use -resource-dir <path> and put that in a clang config file. Clang config files are very limited however.

We’ve only distributed standalone toolchains that are installed in separate directories as we’ve distributed a full set of includes and libraries each time to accompany the tools.

I’m not aware of any fundamental reason why the version couldn’t be removed. We’ve not considered changing it as it hasn’t been enough of an inconvenience to change it.

One area it would affect would be peoples scripts that reference the resource directory location directly (rather than using –print-resource-dir), but that is probably a minor concern.

In my day-job (C++ database work @ Salesforce), we use our own, slightly patched version of clang as part of a hermetic Bazel toolchain. We explicitly specify the resource directory as part of the Bazel toolchain configuration (otherwise, the libraries simply wouldn’t be available in Bazel’s hermetic sandbox).

Currently, every time we update to a new clang version, I have to update the version number in roughly 5 places in our toolchain config (could probably be factored out a bit better…). Not a lot of work, but I would prefer not having to do it.

As such, I would be in favor of removing the clang version number from the resource directory :slightly_smiling_face:

Thanks. Yes, in our case, we’re enforcing the match with compiler version in how we deliver the toolchain.

It sounds like using -resource-dir is a more common approach. Setting CLANG_RESOURCE_DIR asks the driver to specify -resource-dir so I feel like theses are pretty similar, with some slight differences (e.g. setting CLANG_RESOURCE_DIR changes where those resources are put in the build directory).

I’m not aware of any fundamental reason why the version couldn’t be removed. We’ve not considered changing it as it hasn’t been enough of an inconvenience to change it.

Okay, thanks.

One area it would affect would be peoples scripts that reference the resource directory location directly (rather than using –print-resource-dir), but that is probably a minor concern.

Yes. I guess there is a cost to change.

Currently, every time we update to a new clang version, I have to update the version number in roughly 5 places in our toolchain config (could probably be factored out a bit better…). Not a lot of work, but I would prefer not having to do it.

Yes, sounds like a similar case to us.

As such, I would be in favor of removing the clang version number from the resource directory :slightly_smiling_face:

Yes, we’ve been happy with this (implemented locally) for over 10 years and it seems fine.

I’m not pushing for this to change by default (there’s probably a lot of inertia there, and there are uses of the current model, just not in a lot of distribution models). I believe that you can implement this in your own build with CLANG_RESOURCE_DIR=../lib/clang if you wish.

This would be fine for my toolchain distribution (llvm-mingw); I only have one build of Clang in the toolchain root.

I haven’t considered removing it, as it doesn’t bother me. (Anywhere in my build scripts where I need to know the path, I use clang --print-resource-dir to find what it is.)

My initial hunch for where this would matter would be for e.g. linux distributions, where you’d have multiple builds of Clang installed in /usr. But indeed, at least the debian based ones seem to have those tools installed in separate directories trees, like /usr/lib/llvm-18 etc. I’m not sure if all distributions do something like that, though.

Our sources are intended for users that do cmake -S . -B build && make && make install. If there is an older version of Clang installed (e.g. from the OS distribution), sharing the resource files with the older Clang version (e.g. clang-21 and clang-22) would likely yield two broken toolchains. As mentioned in this thread, some OS distributions use their own versioning scheme, as you do, but for upstream LLVM this should be irrelevant. I hope you are not suggesting we make everyone else’s, especially non-developers, life more difficult so you don’t need to update 5 places. Configurable via CMake option: no objections, but not by default.

For Fedora, we do rely on the versioned directory. We support multiple Clang versions, and the resource directory for them is installed in the same path prefix, only distinguished by version.

We explicitly pass -DCLANG_RESOURCE_DIR=../lib/clang/%{maj_ver} to the build though, so we don’t actually depend on the default value.

Yes, I’ve seen similar. Yes, I agree it would depend on the distribution, so it would be for the distributor to decide.

I am not proposing that we always remove the clang version number. I’m not even proposing that we change the default. I think that the current default has uses as you say.

All I’m looking to do is to make sure the path is clear for distributors to remove the version in their distributions, if they wish. It seems like this could be used in a number of LLVM/Clang distributions but isn’t, so I wanted to understand if there was a reason why it wasn’t used (in case I’d missed something).

+1

Thanks, interesting to know about.

Okay. Please note I’m not proposing changing the default (which I believe is “”, from llvm-project/clang/CMakeLists.txt at 95db43fdb4704fd7f177eb257b0f69c1de1dc110 · llvm/llvm-project · GitHub).

So I believe you would be exercising this path llvm-project/clang/lib/Options/OptionUtils.cpp at fd43a646fc05758650c5edae15f199221cb1a2e4 · llvm/llvm-project · GitHub in your distribution then. If we add a llvm::sys::path::remove_dots() here: llvm-project/clang/lib/Options/OptionUtils.cpp at fd43a646fc05758650c5edae15f199221cb1a2e4 · llvm/llvm-project · GitHub to make this slightly tidier, we should make sure you’re on the review, to make sure it doesn’t make any problems for you.