The LLVM 14 release cycle is over now, and I would like to do a retrospective to get feedback about how the release process went. If you want to provide feedback, please answer one of the following questions in the comments:
What went well?
What did not go well?
What can be improved?
As a reminder, we implemented two major changes in this cycle, the first was the new automated backport process with pre-commit CI and the second was the new stable release cycle with faster more frequent stable releases. Feedback on these changes or any other parts of the release process will be greatly appreciated.
My response is from a very packaging-centric perspective, adjust bias compensators accordingly.
Switching to the new release cycle
Responsiveness on discussing (though not necessarily fixing) issues by the community and RM
Automated backports (also dealt with conflicts )
Not so well:
I was trying to build all the RCs for LLVM 14 in conda-forge, and there was pretty substantial breakage between the last RC & GA around the whole tarball situation - this get not get fixed on the 14.x branch despite being milestoned since 14.0.1.
Changes around CLANG_SONAME were not in the release notes and not well documented or consistent across platforms.
Since 14.0.5 (and still on 14.0.6), we’re experiencing hangs on windows in compiler-rt, lld, openmp. Haven’t had time to debug or raise an issue.
Do not allow infrastructure changes between the last RC and GA - if the changes are necessary, add another RC.
Already the 4 RCs of LLVM 14 were on the lower side compared to the 4, 5, 6 RCs of LLVM 13, 12, 11 (respectively). I’m concerned that the 2 RCs proposed for LLVM 15 will not be enough to ensure good release quality (unless there are other procedural changes I’m not aware of), especially if people aren’t aware of this reduction and don’t prioritize testing from rc1 (rather than waiting for an rc3 that might not happen).
Currently, there seems to be no real “release blocker” category that actually blocks a release - perhaps I’m wrong in this perception, but everything unfixed at a specific time seems to just get pushed to the next (patch) release. I understand the advantages for the higher-frequency “ship-what-we-have” mindset, but it would IMO be worthwhile to consider something like that (to be used sparingly).