Compiler support in libc++

Project Infrastructure LLVM Dev List Archives
1

Dear LLVM community,

I’ve wanted to address the topic of which compilers are supported by libc++ for a long time. LLVM documents that it supports GCC >= 5, Clang >= 3.5 and other fairly old compilers. I think this makes a lot of sense for codebases like LLVM and Clang, since it means you can bootstrap a compiler with your system compiler in many cases. It’s also fairly easy to enforce that, since you just have to code in a supported subset of C++.

However, for a library like libc++, things are a bit different. By its very nature, libc++ needs to rely on a recent compiler in order to implement most recent library features. Not being able to rely on a recent compiler leads to problems:

  • Adding new features is significantly more complicated because we need to implement them conditionally on compiler support, not just on support for a C++ Standard. There can also be interactions between what compiler the library is built with and what compiler the headers are used with.

  • We accumulate technical debt around the code base. Some of these #ifdef code paths are not in use anymore, others don’t compile anymore or they contain bugs.

  • It creates a false sense of support: people think they can use a libc++ built with e.g. Clang 3.5, but in reality doing so is a terrible idea. The library might not contain runtime support for features that will be advertised as available by the headers (the char8_t RTTI and the upcoming support for come to mind). Those are serious ABI issues that you’ll only notice when trying to use the feature.

I think it’s important to stress that the current state of things is that we don’t actually support much older compilers - the documentation claims we do, but that is misleading. While things may happen to work on older compilers, I wouldn’t recommend relying on that for anything serious, since it’s mostly untested.

Furthermore, the actual value of supporting old compilers isn’t obvious. Indeed, the best way of building libc++ is to bootstrap Clang and then build libc++ with it, which is easily achieved with the LLVM Runtimes build. Of course, we also support different shipping mechanisms (including non-Clang compilers), but in all cases it should be reasonable to expect that someone building libc++ at the tip is able to do so using a recent compiler.

For all these reasons, I think we must adjust the official support policy we currently document. Concretely, the following modified policy solves the issues I mentioned above and makes it so that the stated support reflects the reality of what we truly support:

  • At any given point in time, libc++ supports back to the latest released version of Clang. For example, if the latest major release of Clang is 14, libc++ (on main) supports Clang 14. When Clang 15 is released (and libc++ 15 with it), libc++ (on main) is free to assume Clang 15. As a result, any released libc++ will always support the previously (and the currently) released Clang, with the support window moving as newer Clangs are released.

  • We support the latest major release of GCC, as advertised on https://gcc.gnu.org/releases.html.

  • We support the latest major release of AppleClang.

The above policy is reasonable from libc++’s perspective, and it also reflects what we test on a regular basis with the CI. Furthermore, supporting up to the last release instead of requiring a trunk compiler (like MSVC’s STL and libstdc++) gives vendors with alternate delivery vehicles approximately 6 months to update their compiler if they want to jump on the next release of libc++, which I think is an important property to retain.

This message is both a heads up about the current state of things, an explanation of where we (the libc++ contributors) want to end up, and an invitation to have a discussion with the rest of the community.

I propose that we maintain our current level of support for older compilers (i.e. keep things roughly building) until the next LLVM release, after which the above policy would become official and libc++ development would be allowed to assume a compiler as documented above. That would give approximately 6 months (from now to the next release) for people managing build bots to migrate to the Runtimes build, and approximately 6 months (from the next release to the next-next release) for external users to adjust to this policy if needed.

Thanks,

Louis

P.S.: There is no mention of other compilers besides Clang, AppleClang and GCC above. That’s because no other compiler is tested on a regular basis, so the status of support for other compilers is unknown. If you’d like to add official support for a new compiler, I’ll be happy to help you set up the required testing.

2

As clarification, would this apply only to building libc++ or also using it?
If not, what will be the new policy for that?

3

As a libc++ contributor a +1

Cheers,
Mark de Wever

4

It would apply to both building the library (the .a, .so, .dylib or whatever), and to building a TU against the headers.

Louis

5

+1 from me.

If you're using a recent standard library you should be using an as recent compiler.
Should help clean up quite a bit of code.

6

As a (rare) stl contributor I am also strongly in favor of the proposal.

I greatly reduces the maintenance burden for us.

–Michael

7

+1 on the compiler support.

I’d love to see a more clearly defined policy for other aspects as well, like supported C libraries and supported OSes.

8

Hi,

What isn’t clear to me is the difference between “building libcxx” and “using the installed here in client code”?
If we ship libc++ on a system, what is the restriction on the system for someone to build a c++ application?

Thanks,

9

Hi,

What isn’t clear to me is the difference between “building libcxx” and “using the installed here in client code”?
If we ship libc++ on a system, what is the restriction on the system for someone to build a c++ application?

The compiler requirements would be the same for building libc++ and for using its headers to build a client application. So basically, you’d be required to use a recent compiler when building an application against recent libc++ headers.

The basic idea is that someone shipping libc++ as part of a toolchain would update Clang at the same time as they update libc++, and any application would be built against a combination of that Clang and the matching libc++. As I said, we’d actually support something more lenient than that, i.e. libc++ would support up to the last stable release of Clang. That way, people who don’t ship libc++ as part of a LLVM-based toolchain would have a 6 month grace period to update their compiler at each release of libc++.

Louis

10

Presumably though, someone building against old headers, but running against a new libc++ would still be supported, right? In other words, you’re still going to maintain binary compatibility?

11

So how does this prevent the libstdc++ mess that you need to lock step
the RTL with the compiler and more importantly, get constantly screwed
over when you need to upgrade or downgrade the compiler in a complex
environment like an actual Operating System?

I consider this proposal a major step backwards...

Joerg

12

Presumably though, someone building against old headers, but running against a new libc++ would still be supported, right? In other words, you’re still going to maintain binary compatibility?

Yes, of course. You can always build your application against a version of libc++ and then link/run it against a newer version of the library (.so or .dylib). If you specify the right deployment target, you can also build against a newer version of libc++ (headers and .so/.dylib), and then actually run it against an older dylib provided your application doesn’t use symbols that didn’t exist in the old dylib you’re running against.

Those guarantees don’t change.

Louis

13

As a libc++ contributor, I am strongly in favor of this. I’d like to re-iterate three main points:

  1. Currently, we are telling users that libc++ supports Clang 3.5 (for example) when there is no proof of that. We are basically guessing (actually, we’re not even guessing; if I had to guess, I’d say that Clang 3.5 probably won’t work).
  2. This will make the QoI way better. Bugs are hidden in macros when we have to support many/old compilers. We can also remove a lot of dead code, which will make it easier to reason about the implementation logic.
  3. Users of old compilers can download old versions of libc++ (in the uncommon case when this is required) by simply heading to https://releases.llvm.org/download.html.

Thanks for pushing this forward, Louis!

14

However, for a library like libc++, things are a bit different.

So how does this prevent the libstdc++ mess that you need to lock step
the RTL with the compiler and more importantly, get constantly screwed
over when you need to upgrade or downgrade the compiler in a complex
environment like an actual Operating System?

Could you please elaborate on what issue you’re thinking about here? As someone who ships libc++ as part of an operating system and SDK (which isn’t necessarily in perfect lockstep with the compiler), I don’t see any issues. The guarantee that you can still use a ~6 months old Clang is specifically intended to allow for that use case, i.e. shipping libc++ as part of an OS instead of a toolchain.

I consider this proposal a major step backwards...

To be clear, we only want to make official the level of support that we already provide in reality. As I explained in my original email, if you’ve been relying on libc++ working on much older compilers, I would suggest that you stop doing so because nobody is testing that and we don’t really support it, despite what the documentation says. So IMO this can’t be a step backwards, since we already don’t support these compilers, we just pretend that we do.

Louis

15

Hi,

It seems to me that this would require one extra stage of bootstrap in CI for many buildbots.
For example, today I have a Linux bot with a clang-8 host compiler and libstdc++. The goal is to ensure that MLIR (but it is applicable to any project) builds with clang and libc++ at the top of the main branch.
So the setup is:

  • stage1: build clang/libc++ with host clang-8/libstdc++
  • stage2: build test “anything” using stage1 (ninja check-all in the monorepo for example, but applicable to any other external project)

With this proposal, the setup would be:

  • stage1: build just clang with host clang-8/libstdc++

  • stage2: build clang/libc++ with stage1 clang and host libstdc++

  • stage3: build test “anything” using stage2 (ninja check-all in the monorepo for example, but applicable to any other external project)

The only way to avoid adding a stage in the bootstrap is to keep updating the bots with a very recent host clang (I’m not convinced that increasing the cost of maintenance for CI / infra is good in general).

We should aim for a better balance: it is possible that clang-5 is too old (I don’t know?), but there are people (like me, and possibly others) who are testing HEAD with older compiler (clang-8 here) and it does not seem broken at the moment (or the recent years), I feel there should be a strong motivation to break it.
Could we find something more intermediate here? Like a time-based support (2 years?) or something based on the latest Ubuntu release or something like that. That would at least keep the cost of upgrading bots a bit more controlled (and avoid a costly extra stage of bootstrap).

Thanks,

16

Hi,

It seems to me that this would require one extra stage of bootstrap in CI for many buildbots.
For example, today I have a Linux bot with a clang-8 host compiler and libstdc++. The goal is to ensure that MLIR (but it is applicable to any project) builds with clang and libc++ at the top of the main branch.
So the setup is:

  • stage1: build clang/libc++ with host clang-8/libstdc++
  • stage2: build test “anything” using stage1 (ninja check-all in the monorepo for example, but applicable to any other external project)

With this proposal, the setup would be:

  • stage1: build just clang with host clang-8/libstdc++

  • stage2: build clang/libc++ with stage1 clang and host libstdc++

  • stage3: build test “anything” using stage2 (ninja check-all in the monorepo for example, but applicable to any other external project)

Would it be possible to change the build system so that libc++ can be built like compiler-rt, using the just-built clang? That would then avoid the need for the extra stage? (though it would bottleneck the usual build a bit - not being able to start the libc++ build until after clang build)

& again, this isn’t so much a proposal of change, but one of documenting the current state of things - which reveals the current situations are sort of unsupported? (though it also goes against the claim that they’re untested) - so I’ll be curious to hear from the libc++ folks about this for sure.

17

Hi,

It seems to me that this would require one extra stage of bootstrap in CI for many buildbots.
For example, today I have a Linux bot with a clang-8 host compiler and libstdc++. The goal is to ensure that MLIR (but it is applicable to any project) builds with clang and libc++ at the top of the main branch.
So the setup is:

  • stage1: build clang/libc++ with host clang-8/libstdc++
  • stage2: build test “anything” using stage1 (ninja check-all in the monorepo for example, but applicable to any other external project)

With this proposal, the setup would be:

  • stage1: build just clang with host clang-8/libstdc++

  • stage2: build clang/libc++ with stage1 clang and host libstdc++

  • stage3: build test “anything” using stage2 (ninja check-all in the monorepo for example, but applicable to any other external project)

Would it be possible to change the build system so that libc++ can be built like compiler-rt, using the just-built clang? That would then avoid the need for the extra stage? (though it would bottleneck the usual build a bit - not being able to start the libc++ build until after clang build)

That’s a good point:

  • stage1: build just clang with host clang-8/libstdc++
  • stage1.5: build libc++ with stage1 clang
  • stage 2: assemble toolchain with clang from stage1 and libc++ from stage2
  • stage3: build test “anything” using stage2 (ninja check-all in the monorepo for example, but applicable to any other external project)

Since this “stage 2” is the new “stage1”, I believe that this should be made completely straightforward to achieve. Ideally it should boil down to a single standard CMake invocation to produce this configuration.

& again, this isn’t so much a proposal of change, but one of documenting the current state of things - which reveals the current situations are sort of unsupported? (though it also goes against the claim that they’re untested) - so I’ll be curious to hear from the libc++ folks about this for sure.

Right: I’m absolutely not convinced by the “we’re documenting the current state of things” actually.
In particular my take in general on what we call “supported” is a policy that “we revert if we break a supported configuration” and “we accept patches to fix a supported configuration”. So the change here is that libc++ would not accept to revert when they break an older toolchain, and we wouldn’t accept patches to libc++ to fix it.
We don’t necessarily have buildbots for every configuration that we claim LLVM is supporting, yet this is the policy, and I’m quite wary of defining the “current state of things” based exclusively on the current public buildbots setup.

18

Stage 1.5 is exactly what cmake
-DLLVM_ENABLE_RUNTIMES=libcxxabi;libcxx should do.

When I last looked at it, it did work and haven't noticed work on that
front. However, I just re-tried and it actually does work. Thanks to
anyone who fixed it :wink:

Michael

19

Hi,

It seems to me that this would require one extra stage of bootstrap in CI for many buildbots.
For example, today I have a Linux bot with a clang-8 host compiler and libstdc++. The goal is to ensure that MLIR (but it is applicable to any project) builds with clang and libc++ at the top of the main branch.
So the setup is:

  • stage1: build clang/libc++ with host clang-8/libstdc++
  • stage2: build test “anything” using stage1 (ninja check-all in the monorepo for example, but applicable to any other external project)

With this proposal, the setup would be:

  • stage1: build just clang with host clang-8/libstdc++

  • stage2: build clang/libc++ with stage1 clang and host libstdc++

  • stage3: build test “anything” using stage2 (ninja check-all in the monorepo for example, but applicable to any other external project)

Would it be possible to change the build system so that libc++ can be built like compiler-rt, using the just-built clang? That would then avoid the need for the extra stage? (though it would bottleneck the usual build a bit - not being able to start the libc++ build until after clang build)

That’s a good point:

  • stage1: build just clang with host clang-8/libstdc++
  • stage1.5: build libc++ with stage1 clang
  • stage 2: assemble toolchain with clang from stage1 and libc++ from stage2
  • stage3: build test “anything” using stage2 (ninja check-all in the monorepo for example, but applicable to any other external project)

Since this “stage 2” is the new “stage1”, I believe that this should be made completely straightforward to achieve. Ideally it should boil down to a single standard CMake invocation to produce this configuration.

I think the Runtimes build is exactly what you’re looking for. With the runtimes build, you say:

$ cmake -S “${MONOREPO_ROOT}/llvm” -B "${BUILD_DIR}”
-DLLVM_ENABLE_PROJECTS="clang”
-DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi”
-DLLVM_RUNTIME_TARGETS="x86_64-unknown-linux-gnu”

And then you can just do:

$ make -C $BUILD_DIR cxx

That will bootstrap Clang and then build libc++ with the just-built Clang. I don’t know whether you consider that to be one or two stages, but it happens automatically in that single CMake invocation. And since building libc++ is basically trivial, this takes approximately the same time as building Clang only.

& again, this isn’t so much a proposal of change, but one of documenting the current state of things - which reveals the current situations are sort of unsupported? (though it also goes against the claim that they’re untested) - so I’ll be curious to hear from the libc++ folks about this for sure.

Right: I’m absolutely not convinced by the “we’re documenting the current state of things” actually.
In particular my take in general on what we call “supported” is a policy that “we revert if we break a supported configuration” and “we accept patches to fix a supported configuration”. So the change here is that libc++ would not accept to revert when they break an older toolchain, and we wouldn’t accept patches to libc++ to fix it.
We don’t necessarily have buildbots for every configuration that we claim LLVM is supporting, yet this is the policy, and I’m quite wary of defining the “current state of things” based exclusively on the current public buildbots setup.

To be clear, what we do today to “fix” older compilers is usually to mark failing tests in the test suite with XFAIL or UNSUPPORTED annotations. We don’t actually provide a good level of support for those compilers. There’s also other things that we simply can’t fix, like the fact that a libc++ built with a compiler that doesn’t know about char8_t (for example) won’t produce the RTTI for char8_t in the dylib, and hence will produce a dylib where some random uses of char8_t will break down. This is just an example, but my point is that it’s far better to clarify the support policy to something that we know will work, and that we can commit to supporting. There’s a small upfront cost for people running build bots right now, but once things are setup it’ll just be better for everyone.

The only way to avoid adding a stage in the bootstrap is to keep updating the bots with a very recent host clang (I’m not convinced that increasing the cost of maintenance for CI / infra is good in general).

We should aim for a better balance: it is possible that clang-5 is too old (I don’t know?), but there are people (like me, and possibly others) who are testing HEAD with older compiler (clang-8 here) and it does not seem broken at the moment (or the recent years), I feel there should be a strong motivation to break it.

Libc++ on Clang 8 doesn’t look broken because it builds. And it builds because you’ve been pinging us on Phabricator when we break you with a change, and we add a “workaround” that makes it build. But there’s no guarantee about the “quality" of the libc++ that you get in that case though. That’s exactly what we want to avoid - you get something that “kinda works”, yet we still have to insert random workarounds in the code. It’s a lose/lose situation.

Could we find something more intermediate here? Like a time-based support (2 years?) or something based on the latest Ubuntu release or something like that. That would at least keep the cost of upgrading bots a bit more controlled (and avoid a costly extra stage of bootstrap).

As I said above, I don’t think there’s any extra stage of bootstrap. The only difference is that you build your libc++ using the Clang you just built, instead of against the system compiler. In both cases you need to build both Clang and libc++ anyway.

Furthermore, we specifically support the last released Clang. If you were in a situation where you didn’t want to build Clang but wanted to build libc++, you’d just have to download a sufficiently recent Clang release and use that.

Louis

20

Strongly in favour of this :slight_smile: