Hi LLVM Community,
We are working on moving our codebase to C++20 and LLVM is one the major libraries we depend on.
I wanted to add a buildbot that builds Clang and CTE with -std=c++20 to make sure we catch and fix any upstream breakages early.
We will fix any existing breakages first, but since it will introduce some burden on the community (emails to fix compile errors in C++20), I wanted to give an early warnings and see if everyone is supportive.
Any thoughts?
Having a staging build bot LGTM as llvm-project will eventually migrate to C++20 and building llvm-project itself gives us a good test suite to sort out compiler issues (we need dogfood to make users trust Clang C++20 mode more 
). Thanks for putting up resources for this!
IIUC some groups have been building llvm-project with C++20 for a while, so the maturity of Clang seems fine. If there are indeed compiler groups, your group is signing up to fix issues or assign relevant work items to appropriate contributors, right? Am I understanding it right that such a bot can be added to Buildbot first, then it appears mature enough, move it to LLVM Production Buildbot ?
There are some previous buildbot discussions which may be of interest, e.g. Buildbots roundtable notes US LLVM Dev 2022
If this is true, does that mean the host compiler will be some released version of clang? (or gcc, either)
My only worry here is that if we were finding a lot of issues in the host compiler, the bot would end up needing an unreleased version of clang to reproduce. So the committer has to build another copy of clang, which is a big ask for some.
Which wouldn’t block having a bot, but you’d have to do more of the work yourselves.
If you expect 99% of the issues to be in the commits themselves then great, one can always download the matching clang release and use that.
(and we could always carry workarounds for known c++20 issues until the next release of clang)
We already have bootstrapping bots that are maintained and may require a committer to build clang to reproduce an issue, I wouldn’t think that having a bot bootstrapping with a stage2 building C++20 is fundamentally different here?
I didn’t consider that, you would want 2 stages. First stage for code that is accepted as c++17 but doesn’t compile as c++20, second stage to catch commits that break c++20 support in clang.
So you’re right it’s the same burden as a 2 stage bot (like those focused on code generation). Assuming that is, that the stage 1 compiler is always a released version of clang.
Do you see a situation where the stage 1 host compiler is not a released clang? Meaning that to reproduce I would have to do a 3 stage build. Use my clang to build your host clang, build stage 2, build the final clang.
Or would stage 1 stick to released clang versions and we’d workaround c++20 issues present in that release? When there’s a fixed release, upgrade and remove the workaround.
(which is what we did with our scalable vector bots, and we got lucky that there was usually already a fixed release to upgrade to)
You could do either but highlighting that the first one is more unusual and just looking at the buildbot email I’m not sure it’d be clear how to reproduce one of those builds. So be prepared to follow up with committers on Phabricator.
Overall I think this bot is a great idea, early warning for the next standard change.
This would be very welcome. If patches are going to be asked to be reverted because they’re not C++20 compatible, then it stands to reason there should be a C++20 buildbot.
Thanks for the input everyone! I’ll start setting up the buildbot and update this thread when I make progress.
If there are indeed compiler groups, your group is signing up to fix issues or assign relevant work items to appropriate contributors, right?
We actually already aim to keep parts of LLVM building in C++20 mode and will continue doing it. At some point we want to make C++20 the default configuration used at Google, and we will be also be catching these failures during our integrate process. But that’s too far away at that point, for now we want to minimize the C++20-related breakages. The buildbot is a step in that direction.
Am I understanding it right that such a bot can be added to Buildbot first, then it appears mature enough, move it to LLVM Production Buildbot ?
Yes, that is the plan. From my initial research, changing from Staging to Production is just a matter of changing a port number after the buildbot is actually set up.
If this is true, does that mean the host compiler will be some released version of clang? (or gcc, either)
I initially plan to try building with released Clang 15. I have not checked how many breakages we will get. Since LLVM is not using any C++20 features, I expect the compiler is in a decent shape to not be the main source of issues. My hope is that most issues that the buildbot hits should be easily reproducible by setting CMAKE_CXX_STANDARD to 20 in the compiler that LLVM folks use daily or can easily get.
We already have bootstrapping bots that are maintained and may require a committer to build clang to reproduce an issue, I wouldn’t think that having a bot bootstrapping with a stage2 building C++20 is fundamentally different here?
Building in C++20 with bootstrapped compiler could also be interesting, but I would rather set up a separate buildbot for that. That is not also not at the top of my priority list, I would maybe only do it if we actually start noticing those issues. I expect it won’t catch most problems with C++20 support in Clang since LLVM code is C++17.
Just wanted to report on the progress we made.
The buildbot has been green in staging for almost a week. It builds with Clang 15, LLD 15 and libstdc++12.
We plan to ask for approvals to move into production Buildbot instance, please let us know if you have any concerns.
What’s the status of this? There don’t seem to be any recent builds on the bot.
The buildbot is working as expected, it moved out of staging into production. Could it be that you were looking at the staging link?
Ah yep I was. Thanks!