When answering a question in a buildbot review, it occurred to me that with the current state our CI is (highly distributed across Buildbot, Jenkins, Buildkite, Github), we really don’t want to be discussing how we’re going to “bring them all together”, but rather thinking how can we aggregate their results in a way that allows people to create whatever CI they’re more comfortable with, as long as they follow some minimal guidelines.
Because we had our own master, I added support for multiple servers, and now, it seems, that monitor also supports Buildkite. After all those years (since I wrote that script), the need for an aggregator hasn’t changed. In fairness, it’s probable more relevant today than it was then.
There’s a lot of talks about buildbot quality, pre-merge CI, noise etc, and these are all hard problems to solve. Trying to bring them all under a single CI technology doesn’t help.
At the very least, it stifles innovation because we’re not allowed to experiment with new builders. But it also forces people to use some CI technology they’re not used to, and end up creating sub-par quality tests (like I did many times over the last many years) because doing better involves spending weeks learning a new CI tool that I’ll never use again.
So I ask: Is there such a thing as a CI aggregator?
And can we just give up relying on a single Buildbot Master instance and a single person to manage it all?
About pre-merge CI, Github is good enough to register many sources (I’ve used DevOps, Buildkite and Github Actions, so far), so it doesn’t matter what we use once we move to PRs.
Is there a general discussion / work group looking into this?