RFC: Update LLVM_VERSION_SUFFIX CMake variable for release candidates

Project Infrastructure Release Testers
1

Hi,

I would like to propose that we start using the LLVM_VERSION_SUFFIX
CMake variable for release candidates. For example:
after the release/13.x branch is created, instead of changing
LLVM_VERSION_SUFFIX from "git" to "", we would change it to "rc1",
then after the 13.0.0-rc1 release is tagged, we would update the
variable to "rc2", etc. Then right before the final release has been
tagged, we would set it to ""

The library SONAME's currently include LLVM_VERSION_SUFFIX, so this
change will cause each release candidate to have a different SONAME
for libraries. This is correct for X.Y.0 releases, since it's possible
for a library's ABI to change between release candidates. However,
for X.Y.1 releases, we do not want to modify the SONAME's at all, so
the build system will need to be updated to accommodate this change.

Ensuring that the SONAME's change when the library ABI's change is
the main motivating reason for proposing this change. However,
I think in general it is also better for the project to be able to
distinguish between release candidates and final releases.

Note that clang does not currently include LLVM_VERSION_SUFFIX in its
--version output, and I'm not proposing change this as part of this
RFC.

- Tom

2

Just a note that this can be confusing because there would now have
builds reporting as `13.0.0-rc1` without any corresponding tag in the
repository. I think that instead of doing that, changing it to be
`13.90.0` or something to indicate "after 13, but not yet 14" while
avoiding any (plausible) `13.x` release number might be a better
strategy.

I learned this trick from KDE by the way if any prior art is wanted.

(Just an outside observer, FWIW.)

--Ben

3

This would mean that the so-called release candidate is no longer a candidate for release. Even if no problems are identified in 13.0.0-rc1, it is still guaranteed that 13.0.0 will be different from 13.0.0-rc1. In particular, this means if a distro were to do a rebuild against 13.0.0-rc1, and then no further changes are needed and 13.0.0 can be released, the distro will still need to rebuild everything that was already built against 13.0.0-rc1 against 13.0.0. The fact that the SONAME changes also means it's possible that other projects adjust to wrongly account for the SONAME change in a way that happens to work for the release candidates, but not for the actual release, so testing with the release candidate suggests that everything is fine when in fact it isn't.

I was working on changing my own testing procedures for my own system to handle the current release candidate structure, where I am in much the same boat as distros, except on a smaller scale. Currently, llvm-12.0.0rc5.src.tar.xz and llvm-12.0.0.src.tar.xz are different files, but the only difference is that the former extracts to an llvm-12.0.0rc5.src directory whereas the latter extracts to a llvm-12.0.0.src directory, the archives are otherwise 100% identical, down to the mtime of each individual file. I wanted to use this to create a build of LLVM+clang 12.0.1-rc3 that, if 12.0.1-rc3 turns out to be the final release candidate, will be bitwise identical to the same build of LLVM+clang 12.0.1 and there will be no reason to re-test anything that worked with 12.0.1-rc3 against 12.0.1, allowing me to avoid a further mass rebuild once 12.0.1 is released. Under your proposed scheme, this may continue to work for x.0.1 releases, I am not sure whether it would continue to work for x.1.0 releases, but it would definitely cease to be an option for x.0.0 releases. That seems a shame, because it means the release candidates will be less tested than they would otherwise be.

Cheers,
Harald van Dijk

4

How can it be; isn't at least the embedded `LLVM_VERSION_SUFFIX`
different? Or am I confused at how `LLVM_VERSION_SUFFIX` is encoded into
the build today?

--Ben

5

Currently, LLVM_VERSION_SUFFIX is cleared on release branches, which happens before the first release candidate, and it remains cleared once the release happens. See commit 7051e794 for when this was done for the 12.x branch. So llvm-12.0.0rc5.src/CMakeLists.txt contains:

if(NOT DEFINED LLVM_VERSION_SUFFIX)
   set(LLVM_VERSION_SUFFIX "")
endif()

And this remained the same in llvm-12.0.0.src/CMakeLists.txt.

Cheers,
Harald van Dijk

6

I would like to propose that we start using the LLVM_VERSION_SUFFIX
CMake variable for release candidates. For example:
after the release/13.x branch is created, instead of changing
LLVM_VERSION_SUFFIX from "git" to "", we would change it to "rc1",
then after the 13.0.0-rc1 release is tagged, we would update the
variable to "rc2", etc. Then right before the final release has been
tagged, we would set it to ""

Just a note that this can be confusing because there would now have
builds reporting as `13.0.0-rc1` without any corresponding tag in the
repository. I think that instead of doing that, changing it to be
`13.90.0` or something to indicate "after 13, but not yet 14" while
avoiding any (plausible) `13.x` release number might be a better
strategy.

This is the situation now, because the version of llvm in the release/13.x
branch is 13.0.0 before there is a 13.0.0 tag.

We use the minor release number for ABI breaking releases sometimes, so I
don't think we can use it to indicate 'almost' the next version.

-Tom

7

Hi,

I would like to propose that we start using the LLVM_VERSION_SUFFIX
CMake variable for release candidates. For example:
after the release/13.x branch is created, instead of changing
LLVM_VERSION_SUFFIX from "git" to "", we would change it to "rc1",
then after the 13.0.0-rc1 release is tagged, we would update the
variable to "rc2", etc. Then right before the final release has been
tagged, we would set it to ""
The library SONAME's currently include LLVM_VERSION_SUFFIX, so this
change will cause each release candidate to have a different SONAME
for libraries. This is correct for X.Y.0 releases, since it's possible
for a library's ABI to change between release candidates. However,
for X.Y.1 releases, we do not want to modify the SONAME's at all, so
the build system will need to be updated to accommodate this change.

This would mean that the so-called release candidate is no longer a candidate for release. Even if no problems are identified in 13.0.0-rc1, it is still guaranteed that 13.0.0 will be different from 13.0.0-rc1. In particular, this means if a distro were to do a rebuild against 13.0.0-rc1, and then no further changes are needed and 13.0.0 can be released, the distro will still need to rebuild everything that was already built against 13.0.0-rc1 against 13.0.0. The fact that the SONAME changes also means it's possible that other projects adjust to wrongly account for the SONAME change in a way that happens to work for the release candidates, but not for the actual release, so testing with the release candidate suggests that everything is fine when in fact it isn't.

As you point out, the disadvantage of my proposal is that the SONAME
will change between the last release candidate and the final release,
even though the ABI has not. I agree this is not ideal. However,
in my opinion, this is better than changing the ABI without changing
the SONAME, which is what can happen in some release candidates with
the current process. Changing the ABI without changing the SONAME is
incorrect, and I would really like to find a way to fix this.

I was working on changing my own testing procedures for my own system to handle the current release candidate structure, where I am in much the same boat as distros, except on a smaller scale. Currently, llvm-12.0.0rc5.src.tar.xz and llvm-12.0.0.src.tar.xz are different files, but the only difference is that the former extracts to an llvm-12.0.0rc5.src directory whereas the latter extracts to a llvm-12.0.0.src directory, the archives are otherwise 100% identical, down to the mtime of each individual file. I wanted to use this to create a build of LLVM+clang 12.0.1-rc3 that, if 12.0.1-rc3 turns out to be the final release candidate, will be bitwise identical to the same build of LLVM+clang 12.0.1 and there will be no reason to re-test anything that worked with 12.0.1-rc3 against 12.0.1, allowing me to avoid a further mass rebuild once 12.0.1 is released. Under your proposed scheme, this may continue to work for x.0.1 releases, I am not sure whether it would continue to work for x.1.0 releases, but it would definitely cease to be an option for x.0.0 releases. That seems a shame, because it means the release candidates will be less tested than they would otherwise be.

The fact that we have to produce new tarballs for the final release
even when nothing has changed since the last release candidate is
an inefficiency in our process and is something I would also like
to fix, but I'm not sure exactly how. I do acknowledge that this
proposal means that we are locking ourselves in to always doing
a separate final release build. Maybe there is some middle ground
where we can fix our SONAME usage without forcing unnecessary builds.

-Tom

8

Hi,

I would like to propose that we start using the LLVM_VERSION_SUFFIX
CMake variable for release candidates. For example:
after the release/13.x branch is created, instead of changing
LLVM_VERSION_SUFFIX from "git" to "", we would change it to "rc1",
then after the 13.0.0-rc1 release is tagged, we would update the
variable to "rc2", etc. Then right before the final release has been
tagged, we would set it to ""
The library SONAME's currently include LLVM_VERSION_SUFFIX, so this
change will cause each release candidate to have a different SONAME
for libraries. This is correct for X.Y.0 releases, since it's possible
for a library's ABI to change between release candidates. However,
for X.Y.1 releases, we do not want to modify the SONAME's at all, so
the build system will need to be updated to accommodate this change.

This would mean that the so-called release candidate is no longer a candidate for release. Even if no problems are identified in 13.0.0-rc1, it is still guaranteed that 13.0.0 will be different from 13.0.0-rc1. In particular, this means if a distro were to do a rebuild against 13.0.0-rc1, and then no further changes are needed and 13.0.0 can be released, the distro will still need to rebuild everything that was already built against 13.0.0-rc1 against 13.0.0. The fact that the SONAME changes also means it's possible that other projects adjust to wrongly account for the SONAME change in a way that happens to work for the release candidates, but not for the actual release, so testing with the release candidate suggests that everything is fine when in fact it isn't.

As you point out, the disadvantage of my proposal is that the SONAME
will change between the last release candidate and the final release,
even though the ABI has not. I agree this is not ideal. However,
in my opinion, this is better than changing the ABI without changing
the SONAME, which is what can happen in some release candidates with
the current process. Changing the ABI without changing the SONAME is
incorrect, and I would really like to find a way to fix this.

Changing the ABI without changing the SONAME is incorrect once we are at a stable version, and not an issue when we are not at a stable version. It is not an issue that the SONAME will be *.so.13git until the 13.x release branch is created, even if breaking changes go in, because nobody should be expecting *.so.13git shared objects to be stable.

On release branches, it gets a bit iffy, because the SONAME has already changed to no longer indicate that it is not stable, but it is not actually stable yet. Whether it is okay to merge breaking changes in that case without updating the SONAME is something that does not have a clear answer, some projects will say yes, others will say no. [...]

I was working on changing my own testing procedures for my own system to handle the current release candidate structure, where I am in much the same boat as distros, except on a smaller scale. Currently, llvm-12.0.0rc5.src.tar.xz and llvm-12.0.0.src.tar.xz are different files, but the only difference is that the former extracts to an llvm-12.0.0rc5.src directory whereas the latter extracts to a llvm-12.0.0.src directory, the archives are otherwise 100% identical, down to the mtime of each individual file. I wanted to use this to create a build of LLVM+clang 12.0.1-rc3 that, if 12.0.1-rc3 turns out to be the final release candidate, will be bitwise identical to the same build of LLVM+clang 12.0.1 and there will be no reason to re-test anything that worked with 12.0.1-rc3 against 12.0.1, allowing me to avoid a further mass rebuild once 12.0.1 is released. Under your proposed scheme, this may continue to work for x.0.1 releases, I am not sure whether it would continue to work for x.1.0 releases, but it would definitely cease to be an option for x.0.0 releases. That seems a shame, because it means the release candidates will be less tested than they would otherwise be.

The fact that we have to produce new tarballs for the final release
even when nothing has changed since the last release candidate is
an inefficiency in our process and is something I would also like
to fix, but I'm not sure exactly how. I do acknowledge that this
proposal means that we are locking ourselves in to always doing
a separate final release build. Maybe there is some middle ground
where we can fix our SONAME usage without forcing unnecessary builds.

[...] If the decision for LLVM is no, then a way around that would be to say that as soon as a release branch is created, the SONAME gets updated as usual, but any breaking change between release candidates results in a SONAME bump. This could mean that 13.0.0 would get SONAMEs that end in e.g. .so.13.2. It sort of decouples SONAME from the release version and avoids the incompatibilities you would like to avoid.

And in that case, the releases could be identical to the last release candidates: the builds would already naturally be identical as they are now, and the source tarballs could be made identical, if desired, just by changing the way they are produced so that they extract to a directory name that does not include the rc version.

Cheers,
Harald van Dijk

9

Hi,

I would like to propose that we start using the LLVM_VERSION_SUFFIX
CMake variable for release candidates. For example:
after the release/13.x branch is created, instead of changing
LLVM_VERSION_SUFFIX from "git" to "", we would change it to "rc1",
then after the 13.0.0-rc1 release is tagged, we would update the
variable to "rc2", etc. Then right before the final release has been
tagged, we would set it to ""
The library SONAME's currently include LLVM_VERSION_SUFFIX, so this
change will cause each release candidate to have a different SONAME
for libraries. This is correct for X.Y.0 releases, since it's possible
for a library's ABI to change between release candidates. However,
for X.Y.1 releases, we do not want to modify the SONAME's at all, so
the build system will need to be updated to accommodate this change.

This would mean that the so-called release candidate is no longer a candidate for release. Even if no problems are identified in 13.0.0-rc1, it is still guaranteed that 13.0.0 will be different from 13.0.0-rc1. In particular, this means if a distro were to do a rebuild against 13.0.0-rc1, and then no further changes are needed and 13.0.0 can be released, the distro will still need to rebuild everything that was already built against 13.0.0-rc1 against 13.0.0. The fact that the SONAME changes also means it's possible that other projects adjust to wrongly account for the SONAME change in a way that happens to work for the release candidates, but not for the actual release, so testing with the release candidate suggests that everything is fine when in fact it isn't.

As you point out, the disadvantage of my proposal is that the SONAME
will change between the last release candidate and the final release,
even though the ABI has not. I agree this is not ideal. However,
in my opinion, this is better than changing the ABI without changing
the SONAME, which is what can happen in some release candidates with
the current process. Changing the ABI without changing the SONAME is
incorrect, and I would really like to find a way to fix this.

Changing the ABI without changing the SONAME is incorrect once we are at a stable version, and not an issue when we are not at a stable version. It is not an issue that the SONAME will be *.so.13git until the 13.x release branch is created, even if breaking changes go in, because nobody should be expecting *.so.13git shared objects to be stable.

On release branches, it gets a bit iffy, because the SONAME has already changed to no longer indicate that it is not stable, but it is not actually stable yet. Whether it is okay to merge breaking changes in that case without updating the SONAME is something that does not have a clear answer, some projects will say yes, others will say no. [...]

I was working on changing my own testing procedures for my own system to handle the current release candidate structure, where I am in much the same boat as distros, except on a smaller scale. Currently, llvm-12.0.0rc5.src.tar.xz and llvm-12.0.0.src.tar.xz are different files, but the only difference is that the former extracts to an llvm-12.0.0rc5.src directory whereas the latter extracts to a llvm-12.0.0.src directory, the archives are otherwise 100% identical, down to the mtime of each individual file. I wanted to use this to create a build of LLVM+clang 12.0.1-rc3 that, if 12.0.1-rc3 turns out to be the final release candidate, will be bitwise identical to the same build of LLVM+clang 12.0.1 and there will be no reason to re-test anything that worked with 12.0.1-rc3 against 12.0.1, allowing me to avoid a further mass rebuild once 12.0.1 is released. Under your proposed scheme, this may continue to work for x.0.1 releases, I am not sure whether it would continue to work for x.1.0 releases, but it would definitely cease to be an option for x.0.0 releases. That seems a shame, because it means the release candidates will be less tested than they would otherwise be.

The fact that we have to produce new tarballs for the final release
even when nothing has changed since the last release candidate is
an inefficiency in our process and is something I would also like
to fix, but I'm not sure exactly how. I do acknowledge that this
proposal means that we are locking ourselves in to always doing
a separate final release build. Maybe there is some middle ground
where we can fix our SONAME usage without forcing unnecessary builds.

[...] If the decision for LLVM is no, then a way around that would be to say that as soon as a release branch is created, the SONAME gets updated as usual, but any breaking change between release candidates results in a SONAME bump. This could mean that 13.0.0 would get SONAMEs that end in e.g. .so.13.2. It sort of decouples SONAME from the release version and avoids the incompatibilities you would like to avoid.

And in that case, the releases could be identical to the last release candidates: the builds would already naturally be identical as they are now, and the source tarballs could be made identical, if desired, just by changing the way they are produced so that they extract to a directory name that does not include the rc version.

Thanks for your feedback. This does seem like the best approach to me.
I will write a patch to implement this.

-Tom