Bumping the CMake requirement for libc++ and libc++abi

Project Infrastructure LLVM Dev List Archives
1

Hi,

The minimum CMake version currently advertised for libc++ and libc++abi is currently 3.4.3. I think the oldest version of CMake actually being tested on any builder is 3.7.0, so advertising 3.4.3 is somewhat of a lie (I'm pretty sure we're using features that require a more recent version already). However, we do need to bump it to 3.8.0 at least because CMake 3.7 doesn't know about C++17 in its compilation features, and we'll need that to build libc++ properly going forward. This will mean for bot owners:
1. They need to upgrade CMake on the builders to at least 3.8.0 (which is really easy), or
2. they can disable processing of libc++ and libc++abi's CMake files by making sure they do not appear in LLVM_ENABLE_PROJECTS

Any objections?

Cheers,
Louis

2

In October, there was a discussion about updating CMake to 3.15: http://lists.llvm.org/pipermail/llvm-dev/2019-October/136295.html. No decision was made, but maybe we should revisit that proposal? If we’re going to require a newer version of CMake for some subprojects, I’d prefer to bump the minimum CMake version for all of LLVM.

3

In October, there was a discussion about updating CMake to 3.15: http://lists.llvm.org/pipermail/llvm-dev/2019-October/136295.html. No decision was made, but maybe we should revisit that proposal? If we’re going to require a newer version of CMake for some subprojects, I’d prefer to bump the minimum CMake version for all of LLVM.

+1

For libc, we want 3.13 or above. Not that it matters immediately as we don’t expect llvm-libc it to be meaningfully useful at least for a few months more. But, considering that a bump like this will take a few months anyway, I do want to push for 3.13+.

4

In October, there was a discussion about updating CMake to 3.15: [llvm-dev] RFC: Updating to CMake 3.15.0. No decision was made, but maybe we should revisit that proposal? If we're going to require a newer version of CMake for some subprojects, I'd prefer to bump the minimum CMake version for all of LLVM.

Yes, I agree we should bump the version for all of LLVM, but I don't
think we should bump the version without a long-term cmake usage plan.
e.g. something like: After every release branch, we bump the cmake version
to whatever version of cmake is X months old.

I think the concern that this was our one chance to bump the CMake version
led to the choice of 3.15 as the next version, which would be too new for some Linux distros.
I think if we had a planned upgrade path, it would be easier for those of us that
want something really new to settle on a release that is a little bit older.

-Tom

5

In October, there was a discussion about updating CMake to 3.15: [llvm-dev] RFC: Updating to CMake 3.15.0. No decision was made, but maybe we should revisit that proposal? If we're going to require a newer version of CMake for some subprojects, I'd prefer to bump the minimum CMake version for all of LLVM.

My personal opinion is that there's a tendency to view all subprojects under the LLVM umbrella as a single, monolithic project. That leads to the desire to make decisions for the whole project, which is often difficult, as opposed to making the right decision for each subproject, which is often easier. This results on subprojects being blocked from doing the right thing for them, like we've seen happen for pre-commit CI. But that's a much larger (non-technical) discussion than the scope of a simple CMake version bump.

Let's try to bump CMake for all of LLVM and see how that goes.

Yes, I agree we should bump the version for all of LLVM, but I don't
think we should bump the version without a long-term cmake usage plan.
e.g. something like: After every release branch, we bump the cmake version
to whatever version of cmake is X months old.

I think the concern that this was our one chance to bump the CMake version
led to the choice of 3.15 as the next version, which would be too new for some Linux distros.
I think if we had a planned upgrade path, it would be easier for those of us that
want something really new to settle on a release that is a little bit older.

Ok, how about the following policy:

  After every release branch, we bump the CMake version to whatever version of CMake is 12 months old.

This is simple, straightforward, and it gives a full year of old CMakes being supported. If we did this right now, this would take us to CMake 3.14.0, released around March 14th, 2019 (Release v3.14.0 · Kitware/CMake · GitHub). I believe the expectation should be that recent CMakes are upgraded using some package manager or download from the site -- we can't really expect the CMake version to be the one provided by the system, because that is either non-existent or very old on most Linux distributions AFAICT. Fortunately, installing a new CMake is incredibly easy.

Is everybody OK with the above policy? What would be the preferred place to document it?

Cheers,
Louis

6

Hello,

To me 12 months makes sense - I know that people maintaining distro packages might disagree. But my view is that it’s hard to accommodate everyone in this regard and it’s hard to hold back real needs because of older distributions.

Both CMake and pip contains up-to-date versions of CMake so installing a more recent version shouldn’t be a problem.

Thanks,
Tobias

7

In October, there was a discussion about updating CMake to 3.15: [llvm-dev] RFC: Updating to CMake 3.15.0. No decision was made, but maybe we should revisit that proposal? If we're going to require a newer version of CMake for some subprojects, I'd prefer to bump the minimum CMake version for all of LLVM.

My personal opinion is that there's a tendency to view all subprojects under the LLVM umbrella as a single, monolithic project. That leads to the desire to make decisions for the whole project, which is often difficult, as opposed to making the right decision for each subproject, which is often easier. This results on subprojects being blocked from doing the right thing for them, like we've seen happen for pre-commit CI. But that's a much larger (non-technical) discussion than the scope of a simple CMake version bump.

Let's try to bump CMake for all of LLVM and see how that goes.

Yes, I agree we should bump the version for all of LLVM, but I don't
think we should bump the version without a long-term cmake usage plan.
e.g. something like: After every release branch, we bump the cmake version
to whatever version of cmake is X months old.

I think the concern that this was our one chance to bump the CMake version
led to the choice of 3.15 as the next version, which would be too new for some Linux distros.
I think if we had a planned upgrade path, it would be easier for those of us that
want something really new to settle on a release that is a little bit older.

Ok, how about the following policy:

  After every release branch, we bump the CMake version to whatever version of CMake is 12 months old.

This is simple, straightforward, and it gives a full year of old CMakes being supported. If we did this right now, this would take us to CMake 3.14.0, released around March 14th, 2019 (Release v3.14.0 · Kitware/CMake · GitHub). I believe the expectation should be that recent CMakes are upgraded using some package manager or download from the site -- we can't really expect the CMake version to be the one provided by the system, because that is either non-existent or very old on most Linux distributions AFAICT. Fortunately, installing a new CMake is incredibly easy.

Is everybody OK with the above policy? What would be the preferred place to document it?

12 months is fine with me.

I'm not sure the best place to document the policy. Some suggestions are here:
Building LLVM with CMake — LLVM 18.0.0git documentation or in the Programmer's manual.

-Tom

8

In October, there was a discussion about updating CMake to 3.15: http://lists.llvm.org/pipermail/llvm-dev/2019-October/136295.html. No decision was made, but maybe we should revisit that proposal? If we’re going to require a newer version of CMake for some subprojects, I’d prefer to bump the minimum CMake version for all of LLVM.

My personal opinion is that there’s a tendency to view all subprojects under the LLVM umbrella as a single, monolithic project. That leads to the desire to make decisions for the whole project, which is often difficult, as opposed to making the right decision for each subproject, which is often easier. This results on subprojects being blocked from doing the right thing for them, like we’ve seen happen for pre-commit CI. But that’s a much larger (non-technical) discussion than the scope of a simple CMake version bump.

Let’s try to bump CMake for all of LLVM and see how that goes.

Yes, I agree we should bump the version for all of LLVM, but I don’t
think we should bump the version without a long-term cmake usage plan.
e.g. something like: After every release branch, we bump the cmake version
to whatever version of cmake is X months old.

I think the concern that this was our one chance to bump the CMake version
led to the choice of 3.15 as the next version, which would be too new for some Linux distros.
I think if we had a planned upgrade path, it would be easier for those of us that
want something really new to settle on a release that is a little bit older.

Ok, how about the following policy:

After every release branch, we bump the CMake version to whatever version of CMake is 12 months old.

This is simple, straightforward, and it gives a full year of old CMakes being supported. If we did this right now, this would take us to CMake 3.14.0, released around March 14th, 2019 (https://github.com/Kitware/CMake/releases/tag/v3.14.0). I believe the expectation should be that recent CMakes are upgraded using some package manager or download from the site – we can’t really expect the CMake version to be the one provided by the system, because that is either non-existent or very old on most Linux distributions AFAICT. Fortunately, installing a new CMake is incredibly easy.

Is everybody OK with the above policy? What would be the preferred place to document it?

12 months is fine with me.

I’m not sure the best place to document the policy. Some suggestions are here:
https://llvm.org/docs/CMake.html or in the Programmer’s manual.

Okay, so assuming nobody objects to this, the process would then be:

  • I put up a Phabricator review updating the documentation and the cmake_minimum_required fields for (all?) LLVM projects I can find.
  • I wait for before committing the change for build bot owners to update their CMake
  • I commit the change – at that point using a too-old CMake will error out and we can point any remaining failing bot to the policy

Does that sound like a reasonable plan? What should be the ? I’d suggest something like two weeks to give folks ample time to update builders.

Louis

9

If we're going to be updating the cmake version requirements llvm-wide
(and in particular, if we're going to make a policy out of that), I
think that should be done in a new RFC thread with an appropriate
subject -- people may not notice this discussion because they don't
follow libc++ and ignore any discussions relating to it.

As for the policy itself, I can't say I am fan of automatically bumping
version requirements, just because. A similar idea was proposed when we
were bumping the compiler versions, and was eventually rejected in favor
of the process described at
<https://llvm.org/docs/DeveloperPolicy.html#updating-toolchain-requirements&gt;\.
The current process involves proposing a specific version and discussing
the trade-offs implied by it.

Now, bumping the cmake version is not as involved as bumping the
compiler, but maybe it would be good if the process for doing it was
similar? If for nothing else, then for consistency?

regards,
Pavel

10

The build compiler and library version updating involved reverting the change when it is known that bots are failing and recommitting when known issues are resolved. It may take several rounds to flush everything out.

11

The next Ubuntu LTS release 20.04 will be available April 23.

That release will have cmake version 3.16.3.

It may take 6 months or so for the LTS users to upgrade in that the automatic update without reinstalling will not be ready for a while. Some people can create a 20.04 VM as soon as the release is available.

Neil Nelson

12

I think the relevant metric here needs to be “which version is available on current LTS distros”, and not “how old is the release”. The current cmake version I have on Ubuntu 18.04 is cmake 3.10.2, so I personally would be happy with a bump to that version, but not something newer. People using other distributions probably have other considerations. If someone wants to perform a cmake version bump, I believe the first step would be to gather what the cmake version availability across distros looks like right now, so an informed decision can be made.

Regards,

Nikita

13

I’ll actually suggest that we might want to do something else entirely. If we depend on the version available in distros, then we are at the mercy of those distros not updating. It also means that we need to look at several distributions and try to satisfy everybody. That’s a lot harder than requiring that people download a recent CMake from cmake.org.

CMake has the benefit of being incredibly easy to install, either from source or with the binaries provided on their website. Do you think it’s an unreasonable requirement to ask folks to download CMake instead of using the one provided with their OS?

Louis

14

This repeats a discussion from late last year and tends to revolve around a philosophy of how software distribution should be done and the cmake issue has brought that into focus. We may be able to glide past this issue with the Ubuntu 20.04 LTS version due shortly and then save the cmake issue for another day if it later needs to be revived.

Neil Nelson

15

I would like to just chime in and say that I’m fairly strongly opposed to any blanket version increases without justification. Having a low version requirement is a feature. It means that more people can build the codebase. We should increase the minimum CMake version requirement only if we need to do so in order to use a new feature. We should be disciplined in deciding when to use new features if they result in the minimum CMake version being increased too much. What recent Linux LTS distros ship can help inform these decisions, but we shouldn’t just upgrade the minimum CMake version to version X just because Linux distro Y ships that version.

Anybody who has ever needed to build something on some ancient version of CentOS because work refuses to upgrade will thank you.

16

I would like to just chime in and say that I’m fairly strongly opposed to any blanket version increases without justification. Having a low version requirement is a feature. It means that more people can build the codebase. We should increase the minimum CMake version requirement only if we need to do so in order to use a new feature. We should be disciplined in deciding when to use new features if they result in the minimum CMake version being increased too much. What recent Linux LTS distros ship can help inform these decisions, but we shouldn’t just upgrade the minimum CMake version to version X just because Linux distro Y ships that version.

The reason for wanting to upgrade is evidently the use of new features that would simplify and clean up the build system quite a bit. I can't speak for other projects, but libc++ and libc++abi are suffering a lot from the lack of e.g. compile_features in recent CMakes.

Anybody who has ever needed to build something on some ancient version of CentOS because work refuses to upgrade will thank you.

Honestly, CMake is so easy to install that I have a hard time relating:

  curl -L https://github.com/Kitware/CMake/releases/download/v3.17.0/cmake-3.17.0-Linux-x86_64.sh | bash /dev/stdin --skip-license --prefix=<YOUR-INSTALLATION-PREFIX>

This takes about 10 seconds to run and you've got yourself the latest CMake. I understand how upgrading some tools is difficult, but CMake just isn't.

Louis

17

In October, there was a discussion about updating CMake to 3.15: http://lists.llvm.org/pipermail/llvm-dev/2019-October/136295.html. No decision was made, but maybe we should revisit that proposal? If we’re going to require a newer version of CMake for some subprojects, I’d prefer to bump the minimum CMake version for all of LLVM.

My personal opinion is that there’s a tendency to view all subprojects under the LLVM umbrella as a single, monolithic project. That leads to the desire to make decisions for the whole project, which is often difficult, as opposed to making the right decision for each subproject, which is often easier. This results on subprojects being blocked from doing the right thing for them, like we’ve seen happen for pre-commit CI. But that’s a much larger (non-technical) discussion than the scope of a simple CMake version bump.

Let’s try to bump CMake for all of LLVM and see how that goes.

Yes, I agree we should bump the version for all of LLVM, but I don’t
think we should bump the version without a long-term cmake usage plan.
e.g. something like: After every release branch, we bump the cmake version
to whatever version of cmake is X months old.

I think the concern that this was our one chance to bump the CMake version
led to the choice of 3.15 as the next version, which would be too new for some Linux distros.
I think if we had a planned upgrade path, it would be easier for those of us that
want something really new to settle on a release that is a little bit older.

Ok, how about the following policy:

After every release branch, we bump the CMake version to whatever version of CMake is 12 months old.

This is simple, straightforward, and it gives a full year of old CMakes being supported. If we did this right now, this would take us to CMake 3.14.0, released around March 14th, 2019 (https://github.com/Kitware/CMake/releases/tag/v3.14.0). I believe the expectation should be that recent CMakes are upgraded using some package manager or download from the site – we can’t really expect the CMake version to be the one provided by the system, because that is either non-existent or very old on most Linux distributions AFAICT. Fortunately, installing a new CMake is incredibly easy.

Is everybody OK with the above policy? What would be the preferred place to document it?

12 months is fine with me.

I’m not sure the best place to document the policy. Some suggestions are here:
https://llvm.org/docs/CMake.html or in the Programmer’s manual.

Okay, so assuming nobody objects to this, the process would then be:

  • I put up a Phabricator review updating the documentation and the cmake_minimum_required fields for (all?) LLVM projects I can find.
  • I wait for before committing the change for build bot owners to update their CMake
  • I commit the change – at that point using a too-old CMake will error out and we can point any remaining failing bot to the policy

Does that sound like a reasonable plan? What should be the ? I’d suggest something like two weeks to give folks ample time to update builders.

FWIW, I agree that it’s easy to install a non-system cmake and using that (we do this on all our bots, and it’s what people have to do on mac and windows already)), and that “what is available in LTS distros” shouldn’t hold things up.

For the actual rollout process, I could imagine that most folks would miss the phab. So I’d suggest what we’ve done for similar changes in the past:

  1. Land a change that makes using an older cmake an error that goes like “We’ll require cmake $version soon. You can define LLVM_SUPPRESS_CMAKE_$version_ERROR to turn this into a warning for a few weeks, but update your bots soon”

  2. Wait 2-4 weeks

  3. Make the error unconditional (and remove the wording about the suppression).

18

Not sure what the rush is. Ubuntu 20.04 will be available in a month with cmake 3.16.3 which appears to be recent enough to handle the issues indicated. Those on older distributions should upgrade as soon as they can and though that may take a few months, getting that coordinated all around does not seem that unreasonable.

The idea of downloading and installing individual pieces of software that are not verified through the distribution process has been discouraged for over a decade. Modern computer systems are far too complex including time and money to be taking unnecessary risks. The argument is never how easy it is to turn your computer into a boat anchor. It is what are the risks against the gain.

The gain could well be worth it. But without recognizing the risks there is no argument for gain. That others take risks we can avoid is not a good argument.

As an aside, a few days ago my new, custom built computer would not complete boot when it was just fine the day before and I was thinking: was it the Xubuntu update, was it my change to fstab, did my nvme ssd die, did I just toast several weeks of work, will my computer live again. It is not a happy situation.

Neil Nelson

19

Wanting to use new features is a good reason to upgrade. You say you need 3.8 in order to compile C++17 code, which is needed by your subproject. This is a good reason to upgrade the version. The proposed policy of bumping the CMake minimum version periodically to within some distance of bleeding edge “just because” is what I object to. We should decide what cmake features we want to use, justify them to ourselves, and then upgrade the version to the lowest version that supports the required new features. We shouldn’t institute a policy of “we upgrade the CMake version to (latest release – N) every T time interval”.

As for the difficulty of upgrading CMake on a target machine; no amount of “it’s really easy to do a thing” can make up for “my organization will not allow me to do a thing.”

20

I understand organization restrictions and old operating systems (I use CentOS 7 myself), but I’ll note that the only requirement for running a new CMake is the ability to download and untar a tarball; in particular, you don’t require sudo. (I understand that there may be restrictions around running arbitrary executables downloaded from the internet, which of course make sense, but I wanted to clarify that you don’t need to sudo.)