Rationale for removing versioned libclang? --> Middle ground to keep it behind option?

@tstellar @tobiashieta
Would you consider a revert of ⚙ D129160 libclang.so: Make SONAME the same as LLVM version + the introduction of a CMake switch?

I chatted to Tom about this yesterday. We are open to reverting it if we think it will go over better this time. But we didn’t think a flag is a good idea - the risk of confusion with two different versioning systems is probably high.

1 Like

I know the change made 14 unusable for me as we kept multiple versions of the LLVM and Clang libraries in a visible in a single tool tree with logic to ensure compiles of a given context always reproduce the exact same object files. The reuse of the 13 so broke stuff when trying to install as it mixed up the dependent libraries used such the llvm libraries. If this is reinstated a canonical way to set the so number to match the major version would be useful to have.

Could you explain this in more detail? Since libclang.so has a pretty strict ABI guarantee, you should generally just be able to drop in a new version.¹ And to my knowledge it can’t generate object files at all, it’s purely a frontend. It’s also not used by the compiler or any tool other than c-index-test, which should probably not be used productively.

Possibly you want an IDE to pick the libclang.so for the compiler that you’re using? But then you can’t have multiple versions coexist because they would conflict (unless there is some weird trick with RTLD_LOCAL that I don’t know about). You would have to fire up separate processes using the different library versions.

Edit: I think I got it. You’re simply saying that you can’t put the files in a common tree, which is certainly true because the libclang.so.13 links conflict between 13 and 14. Here’s some ideas:

  • Just don’t build or ship it. It’s not used for compiling, only for IDEs and similar tools.
  • Do whatever you’re doing with the libclang.so link. This also conflicts between major versions.
  • If you have some packaging system, package the library separate from the remainder and have LLVM 14’s libclang.so.13 supersede LLVM 13’s. (That’s what we’re doing and likely most other Linux distros—in whichever way they can supersede stuff.)

¹ To be fair, there are issues with Clang’s placement of builtin headers, but that’s an issue even for new patch-level versions. A solution for that would likely also work with major version updates.

What’s the status here? LLVM 15.x release status says we have <24h left for changes going into LLVM 15, so such a revert would be quite urgent now (I’d understand the choice not to gate behind a variable BTW, but it’d b a pity to throw away all the work that had gone into this, and the benefits we’d be foregoing)

Hi!

It would be nice if we could solve this before 15.0.0-final since otherwise we have to wait for 16.x. As I see it we have a few options:

  1. status quo - don’t change anything and keep the SOVERSION == LLVM release version
  2. change it to mean ABI breaks as it’s “supposed” to mean. We probably should have a new version instead of 14 in that case to avoid confusion
  3. make the soversion configurable with the default being a ABI stable number or the version number

Please give feedback so that we can close this issue in the coming two weeks.

If we don’t solve it before -rc3 I am still happy to include it and maybe do a rc4 if we have to. we just need to commit to a solution.

1 Like

Ha, seems we were pretty much simultaneous in thinking about this. :upside_down_face:

Comments:

  1. “Status quo” is a bit fuzzy because there’s at least two: last released version (!=) and tip of tree (==)
  2. What do you mean by “as it’s ‘supposed’ to mean”? AFAIU increment-for-ABI-breaks was already the status as of LLVM 14?

Assuming I’m not overlooking something, I’d prefer 2b. (keep status as of LLVM 14, i.e. SOVERSION=13, increment it for ABI breaks), but could happily live with 3. and would like to avoid 1. if possible.

Thanks for giving this such attention!

I am leaning towards having a option actually - just to not sit this one out for another version, it would also be very low-risk. I think we should make it obvious that the option is temporary and it will be removed in a future release.

What do you think @h-vetinari - could you prepare such patch?

I started looking into this. It’s a bit confusing to me because there already exists an option called LIBCLANG_LIBARY_VERSION which seems like it should control exactly that.

Assuming we still want to add a single option, it’d IMO look something like

# in clang/CMakeLists.txt
option(CLANG_FORCE_MATCHING_LIBCLANG_SOVERSION
  "Force the SOVERSION of libclang to be equal to CLANG_MAJOR" OFF)

# in clang/tools/libclang/CMakeLists.txt
if(NOT CLANG_FORCE_MATCHING_LIBCLANG_VERSION)
  # renamed from previous, less-than-ideal variable name CLANG_SONAME
  set(LIBCLANG_SOVERSION 13)
else()
  set(LIBCLANG_SOVERSION ${CLANG_VERSION_MAJOR})
endif()

if(LIBCLANG_SOVERSION != LIBCLANG_LIBARY_VERSION)
  # error to avoid inconsistent options;
  # harmonization of the two variables can happen for clang 16
endif()

WDYT?

Hmm - if it there is already an option I wonder if that would be enough? But otherwise it looks fine - but I think maybe we should have a warning if it’s enabled and warn about deprecation.

Well, that option did not affect the linker script[1], and didn’t help people from getting tripped up by this. It also somewhat complicates the respective build scripts because one would have to pass the version to some cmake invocation (which obviously changes from version to version, and integrators would prefer not to touch those scripts as much as possible), rather than a simple boolean switch which doesn’t need to be changed.

How about adding the revert + boolean option for LLVM 15.0.0-rc3, and figuring out a more comprehensive clean-up (incl. deprecations) later, with backports as reasonable?


  1. in the current implementation ↩︎

Yep that seems like a great plan so we don’t delay the release that much.

I’m not familiar with phab (and too short on time to start now…), so here’s two branches:

  • one for main (no release note)
  • one for release/15.x (with release note in last commit); full diff

Doing separate commits should also make the review much easier I hope, but I’ll have to leave it to you how you deal with that. Hope this helps nevertheless.

I think I did a diff in phab here for several commits (I am also not a phab expert) - can you check if it looks alright? :gear: D132486 Revert “libclang.so: Make SONAME the same as LLVM version”

Thanks a lot. Commented there, LGTM

@mgorny @sylvestre do you guys have any thoughts on this?

I don’t think ⚙ D132486 SONAME introduce option CLANG_FORCE_MATCHING_LIBCLANG_SOVERSION is the right approach.
I think we should aim at getting consistency in term of packaging of the llvm toolchain in various distro, not more differences.
Giving this option will fragment it a bit more.

Now, about the naming discussion, I think llvm SONAME == clang SONAME, ABI is hard, no need to add more complexity.

Distros are quite insular in what standards they pursue across many different packages (often overruling upstreams directly; e.g. shared vs. static builds, C++ standard versions, build flags for arches and many more). Consistency is good, but can it be expected today to exchange artefacts across distros and have them work across the board…? I’d be very doubtful of that.

Whether the libclang soversion matches between distros is IMO way less impactful than making it impossible for those who are able to deal with different soversions to benefit from the advantages of not having to rebuild & redistribute everything quite so often.

Yes, ABI is hard. So why throw away useful work (and information) by those who take care of it (and take pains not to change it)…? I feel the complexity/utility trade-off is highly subjective here. I’m aware that this affects me equally, but “ABI is hard” is not really a thorough argument IMO.

I don’t find the change to a fixed version (13) particularly useful, it breaks existing software [1] and IMO risks confusion. In the FreeBSD ports context the need to rebuild to use a new version is just not a concern and people typically have several versions of LLVM installed due to differences between versions required for e.g., Mesa and Firefox.

If D132486 is committed, I will build FreeBSD packages with CLANG_FORCE_MATCHING_LIBCLANG_SOVERSION enabled.

[1] Sure, one could argue (and people have) that the software has a broken build system, but the change was not widely telegraphed and software stopped working as a result…