I’d like to share my thoughts on this topic. Most of my experience with LLVM is through libc++ so, that’s what I’ll talk about but, I assume that most of this applies to other sub-projects as well.
There are four things that I think everyone would agree are important to allow us to succeed in collaborating on a shared codebase. These things are:
- A good reviewing tool.
- Issues tracking.
- A good testing framework.
- The ability for the maximum number of people to contribute high-quality patches.
It’s hard to compare the review tool. Lots of people have strong opinions about it, and it’s mostly subjective.
We have already moved some sub-projects to GitHub issues. This means we have four different places to look for the “status” of something. We have the status page, GitHub issues, Bugzilla, and Phabricator/GitHub. To check if something has been implemented, a person has to look through an undefinable combination of those sites. Imagine this instead: the status of issues in the current implementation, LWG issues, and papers would all be tracked on GitHub. Those issues could be linked to a patch made on GitHub, and both the issue and the patch could be assigned to someone. Currently, to check the status of something, you have to look through all four of those platforms above and maybe ask on the mailing list to check if someone’s working on it already. If we use GitHub, we eliminate both the difficulty of checking and the semi-frequent occurrence of two people implementing the same thing.
Libc++ has a great set of unit tests but, they have lots of different (usually inline) configurations. Often (especially for newer contributors), all the tests are run in several configurations, and then the patch is committed. Later the buildbots find something, and the patch has to be reverted. This is unnecessary. If the buildbots were run before the patch lands, we could catch the majority of the errors early.
When a patch is submitted to libc++ by anyone except the three maintainers, a few things happen. First, it’s reviewed. That process works well and shouldn’t change too much. Second, it often needs to be tested by one of the maintainers. While I’m not a maintainer of libc++, I can imagine that this process can be time-consuming, especially considering the volume of patches submitted. Last, the change needs to be committed. This involves downloading the raw patch, applying it, and committing on behalf of the person. Then, watching the bots and sometimes reverting the patch. This is also time-consuming. On GitHub, we can configure the PRs with requirements. These requirements can be things like, “x number of maintainers approve this patch.” Or, “the following buildbots succeed.” These requirements mean that the reviewers are no longer responsible for committing, running the tests, or reverting. GitHub will make sure the tests pass before anything is committed, then after it’s been approved, the contributor can commit and revert the code themself (without commit access).
TL;DR: There are only three maintainers and their time is very valuable. Currently, we waste lots of it by having them manually go through a series of steps that are error-prone and time-consuming, but easily automatable. We should automate them such that they have a higher success rate and don’t waste valuable time.