RFC: "RFC Shepherds"

LLVM Project
1

I would like to propose that we create a new role within the community called “RFC Shepherds”. “RFC Shepherds” will be responsible for ensuring that a definitive yes or no decision is made for every RFC proposed for the project.

Creating this new role will help solve the problem we have with some RFCs, where an RFC is proposed, discussed, but then goes nowhere because there is no definitive decision made about how to move forward. This is especially an issue for newer contributors who may not know who to ping to revive a discussion or to give a definitive yes or no decision on their RFC. A good example of this problem is the recent RFC to move the llvm vim utils into its own repo.

The process for “Shepherding” an RFCs will be:

  1. Monitor Discourse for new RFCs and add self as a watcher.
  2. After 1 week of discussion on an RFC, review the discussion and decide if an RFC has:
    a) Reached consensus (either yes or no).
    b) Needs another week of discussion.
    c) Needs to be promoted to the LLVM Decision Making Process.
  3. Explicitly communicate the decision by commenting on the Discourse thread.

Once an “RFC Shepherd” decides that there is a consensus ‘yes’ on an RFC, then and only then may the changes implementing the RFC be committed to the repository (after following the existing process for code review).

It is important to note that “RFC Shepherds” should not make decisions based on their own personal preferences, but should be making decisions based entirely on community feedback. An ideal person to “Shepherd” each proposal would be someone with some background knowledge on the topic, but without any personal stake in it. It is understood, though, that this ideal person may not exist for every RFC.

In most cases only one “RFC Shepherd” will be needed to review an RFC, but RFC authors or anyone who commented on the thread may request additional “RFC Shepherds” review the proposal in case of a disagreement.

There will be three “RFC Shepherds” and they will be chosen from community volunteers with preference given to those who volunteer first. (i.e. The first three people to volunteer will become “RFC Shepherds”). Each “RFC Shepherd” will commit to keeping the role for six months. After the six months is up, the “RFC Shepherd” may volunteer again for another six months if there are no new volunteers.

I think that our community has grown big enough that our lack of authoritative decision makers is starting to affect the quality of the project. When I first started working on LLVM 10+ years ago, there were a number of highly visible, very active contributors who were clearly the decision makers for the project and where able to fill this role. But today, I don’t think we really have anyone who feels empowered enough to make authoritative decisions in the same way. Or at least the people that do are much less visible and much less active then they used to be.

I think it’s very important that we change this and give more of the project experts the authority to evaluate and decide community consensus for the future direction of the project. The “RFC Shepherds” is how I think this problem could be solved, but I would be happy with any solution that solves this fundamental issue.

7 Likes
2

This is a good idea overall.

Some practical details:

seems at odd to me with:

In particular your current proposal seems to encompass all the subprojects, and I have a hard time imagining that we’ll find 3 people to cover all of them. Even in a single project there are enough components that this seem difficult.

Ultimately how does that articulate with the “code owners” role? Could we look into exploring some options that would consolidate the “code owner” role, for example having a small “team of code owners” for each component and have them be responsible of this “shepherding”?

1 Like
3

I started with a small number, because I wasn’t sure we would get many people to volunteer and wanted to have a goal that was achievable.

I really like the idea of selecting Code Owners to act as the “RFC Shepherds”. We could add this to the current Code Owner responsibility or tag certain Code Owners as Shepherds for a particular sub-project or component.

4

+1 to the general idea and that Code Owners should be RFC Shepherds, as they are already in theory responsible for ensuring patches to their area of code are “shepherded” through, although I think we need non-code-owner shepherds too, because otherwise who is the RFC Shepherd for an RFC proposed by the Code Owner of an area, or for areas (such as LLVM processes) that don’t have Code Owners?

5

I believe that code owners and shepards are orthogonal roles. For purely process-based RFCs, there are no code owners. For many RFCs there are obvious code owners, but not always.

6

I like the general idea here. I’m not sure using code owners for this is a good idea, for a couple of reasons:

  • At least for the LLVM subproject, the list of code owners is hopelessly outdated. Many code owners are inactive or inactive in the area they allegedly own. Conversely, people who de-facto own certain areas are not listed. We would need a process that actually makes sure this is kept reasonably up to date.
  • Code ownership tends to be pretty specific (e.g. individual passes), while RFCs tend to be higher level (e.g. IR changes). This means that there will usually not be a single obvious code owner who would feel responsible for driving the proposal through the process.
  • My understanding is that the “RFC shepherd” role is largely a procedural one. It should ensure that interested parties (e.g. code owners) are aware of the RFC and provide feedback, and a final decision is reached based on that feedback. While some degree of familiarity with the topic at hand is helpful, I don’t think the people providing the technical feedback, and the person driving the procedural aspects, need to be the same (or would even benefit from being the same).
7

I generally support the notion of a shepherd, but there’s a couple points worth noting.

First, and this is a really important one, LLVM has a long standing community norm that “silence means no”. Part of getting an RFC through is the (offline) coalition building to build agreement on an idea. There’s a lot of proposals that are just flat out bad ideas. Explaining why in a public written forum is often both time consuming, and exhausting for the person doing so.

From the perspective of this proposal, this shifts the definition of the shepherd role slightly. It goes from “counting the number of responses” to “did the right people reply to this? are they supportive?”. In order to be effective, a shepherd should assist an RFC author with identifying key stakeholders and doing the coalition building required.

Second, as a consequence of the former, I’m a bit leery about your “first three people” suggestion. I would suggest instead this needs to be “first three people who have demonstrated an ability to navigate our community in the past”.

Third, you’re right that this is a long standing problem, but it’s also one which got recently much worse due to the transition to discourse. We used to have a number of folks who would reply to an RFC with the “right” CC list to draw it attention of relevant parties. I can only speak for myself, but with discourse I’m much less likely to see a relevant proposal in a timely manner. One thing we should consider with the shepherd role is whether proactive outreach (email, discord, whatever) to key stakeholders should be a part.

Fourth, I worry that the time commitment involved in doing all the above will overwhelm any small group. My best suggestion on how to address this would be to allow shepherds to abandon RFCs they consider either uninteresting or a lost cause. That is, our expectation for a shepherd is that they pickup at least a few RFCs during their tenure, not that they handle all RFCs during their tenure.

3 Likes
8

Thank you for the proposal! I think there’s definitely a gap to be filled in the LLVM community here, and some kind of ‘shepherd’ is a good approach. A few points:

  • I think an explicit responsibility of the shepherd should be ensuring that the right people have had input (preames gave a great description of this - so I won’t elaborate further!). We should also acknowledge that there is a judgement call in terms of how much input is required based on the specifics of an RFC (some have wide-ranging long term impact, others don’t).

  • I don’t think shepherds need to map to code owners - to act as a shepherd someone needs to be familiar enough with the community norms and the active contributors to be able to make informed judgements about whether the right people have weighed in on a topic etc. But that’s not an ability limited to code owners, and although you need some familiarity I don’t think the ‘shepherd’ necessarily needs to be intimately familiar with the aspect of LLVM the RFC is written about.

  • A somewhat minor note on phrasing. If we go in this direction, I imagine this RFC will be used as the basis for the documentation on the role of the shepherd. I think I fully agree with “give more of the project experts the authority to evaluate and decide community consensus for the future direction of the project.” But the penultimate paragraph’s talk about “authoritative decision makers” threw me off a little bit, as I thought it was going down a different path.

1 Like
9

Do you have any suggestions for how determine who is eligible based on this (or some similar) criteria?

10

“Authoritative Decision Makers” was just me trying to describe who had been doing the shepherding work in the past, or at least where the definitive yes/no answers were coming from. Since we don’t really have people in those roles any more, we need someone else to do the shepherding. The quote “give more of the project experts the authority to evaluate and decide community consensus for the future direction of the project.” is really what I think the goal of the shepherds should be.

1 Like
11

This is a really good proposal.

Just to be clear, this feels like almost the definition of being unwelcoming via illegible (hidden) politics. And, based on private/offline conversations I’ve had about how LLVM-the-project works in practice, I know I’m not the only one to think so.

As a corollary, I agree with this:

FWIW, this is a good idea regardless of what the historical practice in LLVM has been, and I didn’t read Tom’s proposal as “counting the number of responses” in the first place.

2 Likes