LLVM Incubator + new projects draft

The idea of adding an “incubation” stage to projects in the LLVM world seems to be positively received. I also noticed that we don’t really document the new project policy in general in the LLVM Developer Policy. To help with both of these Stella and I worked together to draft up a new section for the LLVM developer policy (incorporating the existing “New Targets” section).

Ahead of proposing a Phabricator patch, we put it into this google doc, I’d love to get feedback on it from anyone who is interested in this:
https://docs.google.com/document/d/1ss4jGHywL0Y2KW_l4LqTo5CgJxx3i0_4-FkbXiPQMus/edit

Assuming no major concerns here, I’ll post the patch in the next couple days.

Thanks!

-Chris

The idea of adding an “incubation” stage to projects in the LLVM world seems to be positively received. I also noticed that we don’t really document the new project policy in general in the LLVM Developer Policy. To help with both of these Stella and I worked together to draft up a new section for the LLVM developer policy (incorporating the existing “New Targets” section).

Ahead of proposing a Phabricator patch, we put it into this google doc, I’d love to get feedback on it from anyone who is interested in this:
https://docs.google.com/document/d/1ss4jGHywL0Y2KW_l4LqTo5CgJxx3i0_4-FkbXiPQMus/edit

Currently it doesn't open, "You need access", sanity check: is viewing
allowed for everybody?

Assuming no major concerns here, I’ll post the patch in the next couple days.

Thanks!

-Chris

Roman

It says that “anyone on the internet is allowed to comment”, maybe this link will work better?:
https://docs.google.com/document/d/1lC7cOJ2Iiqdx62o81J5YP7RzFHi8k2Rkt0zla-Kh6no/edit?usp=sharing

In any case, if google docs isn’t cooperating, then you can check it out when it gets to Phabricator.

-Chris

This document is titled “The Great ML Language (Un)Debate: Swift”, did you mean to post a different link?

Dmitri

Hi Chris,

I’m also seeing an access denied error on the first link you shared, and although I can access the second document, it doesn’t look like the document you meant to share. It looks like a one pager on ML in Swift.

Thomas

Hah, whoops, sorry about that. This is the correct link:
https://docs.google.com/document/d/1ss4jGHywL0Y2KW_l4LqTo5CgJxx3i0_4-FkbXiPQMus/edit

-Chris

Looks like a good proposal to me as-is! Thanks for putting this together to both of you :slight_smile:

This looks to be a reasonable starting point.

A couple of nit picks, none are blockers.

  1. I’d hold off on handing out the sub-domain for the moment. This feels more official than we probably want for a random incubator. I reserve the right to change my mind here, but maybe we should delay this part until we see what actual incubators look like? As an alternative, maybe have a common incubator.llvm.org page which links to the docs defining the process and lists all active incubators with links to docs in their own repo?
  2. The must/should terminology should probably be factored out once and referenced. As written, it takes a little effort to be sure the definitions are the same between the two uses.
  3. I’m not sure I agree with the no-code standard. I agree with minimal code, but I think an incubator should be established enough to be discussed concretely (e.g. “what is” vs “ideals”).
  4. As I mentioned before, I’d advocate for the notion of a sponsor (an existing LLVM contributor) for each incubator. I’d have that a must on the incubator list.

Philip

One other thought, since it came up on the apple branch thread.

I think we need to be very specific about the fact that code in an incubator has been contributed to LLVM. I think this is implicit in the statements about license and policy, but having this be made explicit seems worthwhile. This would, for instance, make it obvious that code could be moved between repository without further involvement of the original authors.

Again, I think this is covered by the existing text, but leaving no room for ambiguity here seems worthwhile. :slight_smile:

Philip

One other thought, since it came up on the apple branch thread.

I think we need to be very specific about the fact that code in an incubator has been contributed to LLVM. I think this is implicit in the statements about license and policy, but having this be made explicit seems worthwhile. This would, for instance, make it obvious that code could be moved between repository without further involvement of the original authors.

Again, I think this is covered by the existing text, but leaving no room for ambiguity here seems worthwhile. :slight_smile:

+1 - This is actually a significant part of why I originally advocated for this approach. The reality is that many of us have various hurdles and ambiguities around collaborating with each other but are explicitly allowed/encouraged to do so in the context of contributions to LLVM and engaging in public as part of this community. Being explicit that a contribution to an incubator project is a contribution to LLVM is a bright line that is easy to point at (and hopefully understand). I’ll leave the opinion of whether it is fine to leave this implicit vs explicit to those more knowledgeable about the vagaries of such things.

One other thought, since it came up on the apple branch thread.

I think we need to be very specific about the fact that code in an incubator has been contributed to LLVM. I think this is implicit in the statements about license and policy, but having this be made explicit seems worthwhile. This would, for instance, make it obvious that code could be moved between repository without further involvement of the original authors.

Again, I think this is covered by the existing text, but leaving no room for ambiguity here seems worthwhile. :slight_smile:

+1 - This is actually a significant part of why I originally advocated for this approach. The reality is that many of us have various hurdles and ambiguities around collaborating with each other but are explicitly allowed/encouraged to do so in the context of contributions to LLVM and engaging in public as part of this community. Being explicit that a contribution to an incubator project is a contribution to LLVM is a bright line that is easy to point at (and hopefully understand). I’ll leave the opinion of whether it is fine to leave this implicit vs explicit to those more knowledgeable about the vagaries of such things.

Thanks Philip and Stella,

It is hard to be concrete about this with specificity, but I added a new line to address this at the end of the first paragraph of the Incubator section:

"This also allows collaboration between teams that have a special carveout for working with projects in the LLVM umbrella.”

In my opinion, incubator projects are unarguably “part of the LLVM umbrella”, but companies may have different policies - including limiting contributions to specific subproject, specific targets, etc. Lemme know if this seems like a reasonable middle path to you.

-Chris

This looks to be a reasonable starting point.

A couple of nit picks, none are blockers.

  1. I’d hold off on handing out the sub-domain for the moment. This feels more official than we probably want for a random incubator. I reserve the right to change my mind here, but maybe we should delay this part until we see what actual incubators look like? As an alternative, maybe have a common incubator.llvm.org page which links to the docs defining the process and lists all active incubators with links to docs in their own repo?

Sounds great, I’m happy to take this out - this avoids “promising” it, but we can still discuss it on a case-by-case basis. I changed this to "Other infrastructure integration can be discussed on a case-by-case basis.”, because there are bug tracker and other things as well.

  1. The must/should terminology should probably be factored out once and referenced. As written, it takes a little effort to be sure the definitions are the same between the two uses.

I’m not sure what you mean here. Do you have a recommended approach?

  1. I’m not sure I agree with the no-code standard. I agree with minimal code, but I think an incubator should be established enough to be discussed concretely (e.g. “what is” vs “ideals”).

I hear what you’re saying, but I think we can handle this as part of the approval process. We can bounce of things that qualitatively don’t feel credible and give guidance there, but can still be receptive if something seems like a promising direction.

  1. As I mentioned before, I’d advocate for the notion of a sponsor (an existing LLVM contributor) for each incubator. I’d have that a must on the incubator list.

Yes, this is a good idea. The problem here is “how do we decide who qualifies as a sponsor?”. I don’t know a good way to say that - someone with N years of LLVM experience, M patches, …? How does this get explained?

-Chris

> I'm not sure I agree with the no-code standard. I agree with minimal code, but I think an incubator should be established enough to be discussed concretely (e.g. "what is" vs "ideals").

I hear what you’re saying, but I think we can handle this as part of the approval process. We can bounce of things that qualitatively don’t feel credible and give guidance there, but can still be receptive if something seems like a promising direction.

I have to say that I agree with Philip here, and saying "we can handle
this as part of the process" feels like a cop-out to me. Part of the
point of writing up a document like this is to establish and document
expectations, and this is an important expectation to establish.

To clarify, the key underlying issue here is that there is no reason
to make incubation a free-for-all (people can always create repos on
GitHub), but if it's not a free-for-all, then a decision must be made
somehow. The question is: what grounds is that decision going to be
made on?

The fewer concrete artefacts we have to make a decision, the more
people will form their opinions based on soft, clique-ish factors such
as whether they like the person making the proposal, whether they
consider them to "already be part of the community", what company or
university they work for (if any), and so on. Clique-ishness is the
opposite of inclusivity, so I feel very strongly that the expectation
should be having at least _some_ basic artefacts here. There might be
the rare legitimate exception to this rule, but that is nicely
captured by making it a "Should" rule.

Along similar lines, having some sort of manifesto / development plan
/ whatever really should be a Must. Writing something like that up is
an incredibly low bar, and if people can't even be bothered to do
*that*, well... Having that Must next to "there Should be a prototype"
also helps clarify the expectations.

> As I mentioned before, I'd advocate for the notion of a sponsor (an existing LLVM contributor) for each incubator. I'd have that a must on the incubator list.

Yes, this is a good idea. The problem here is “how do we decide who qualifies as a sponsor?”. I don’t know a good way to say that - someone with N years of LLVM experience, M patches, …? How does this get explained?

Do we really need that? It feels to me like this is much more
something that could be handled as part of the approval process. If
nobody from "the community" (whatever that means) is in favor of the
incubation, then it's not going to happen. So kind of by definition
you need at least one person speaking in favor :slight_smile:

Cheers,
Nicolai

I'm not sure I agree with the no-code standard. I agree with minimal code, but I think an incubator should be established enough to be discussed concretely (e.g. "what is" vs "ideals").

I hear what you’re saying, but I think we can handle this as part of the approval process. We can bounce of things that qualitatively don’t feel credible and give guidance there, but can still be receptive if something seems like a promising direction.

I have to say that I agree with Philip here, and saying "we can handle
this as part of the process" feels like a cop-out to me. Part of the
point of writing up a document like this is to establish and document
expectations, and this is an important expectation to establish.

To clarify, the key underlying issue here is that there is no reason
to make incubation a free-for-all (people can always create repos on
GitHub), but if it's not a free-for-all, then a decision must be made
somehow. The question is: what grounds is that decision going to be
made on?

The fewer concrete artefacts we have to make a decision, the more
people will form their opinions based on soft, clique-ish factors such
as whether they like the person making the proposal, whether they
consider them to "already be part of the community", what company or
university they work for (if any), and so on. Clique-ishness is the
opposite of inclusivity, so I feel very strongly that the expectation
should be having at least _some_ basic artefacts here. There might be
the rare legitimate exception to this rule, but that is nicely
captured by making it a "Should" rule.

Along similar lines, having some sort of manifesto / development plan
/ whatever really should be a Must. Writing something like that up is
an incredibly low bar, and if people can't even be bothered to do
*that*, well... Having that Must next to "there Should be a prototype"
also helps clarify the expectations.

As I mentioned before, I'd advocate for the notion of a sponsor (an existing LLVM contributor) for each incubator. I'd have that a must on the incubator list.

Yes, this is a good idea. The problem here is “how do we decide who qualifies as a sponsor?”. I don’t know a good way to say that - someone with N years of LLVM experience, M patches, …? How does this get explained?

Do we really need that? It feels to me like this is much more
something that could be handled as part of the approval process. If
nobody from "the community" (whatever that means) is in favor of the
incubation, then it's not going to happen. So kind of by definition
you need at least one person speaking in favor :slight_smile:

Do we want to specifically say some member of the community (i.e., someone with commit access?) not associated with the proposed incubator project? Are we looking for some kind of community consensus, lack of broad concern, etc.?

-Hal

I'm not sure I agree with the no-code standard. I agree with minimal code, but I think an incubator should be established enough to be discussed concretely (e.g. "what is" vs "ideals").

I hear what you’re saying, but I think we can handle this as part of the approval process. We can bounce of things that qualitatively don’t feel credible and give guidance there, but can still be receptive if something seems like a promising direction.

I have to say that I agree with Philip here, and saying "we can handle
this as part of the process" feels like a cop-out to me. Part of the
point of writing up a document like this is to establish and document
expectations, and this is an important expectation to establish.

Sure, I can see that concern. The document does exactly this, by laying out requirements like "Must be generally aligned with the mission of the LLVM project to advance compilers, languages, tools, runtimes, etc.”

Anything in a policy has to be written to have the right level of rigidity. Too rigid and you bounce of potentially good ideas that could be molded into something workable with community discussion and coaching. Too loose and there is no value. The major problem of being too loose is that you can create scale problems: When we have hundreds of requests, each needs to be discussed individually. Writing things down in a more rigid way reduces the cost of that.

In this case, I don’t expect to have hundreds of requests or for scalability to be a problem. Furthermore, we don’t have a lot of experience here. Once we get a few requests and if we hit scale problems, we can codify the best practices we discover along the way into a more rigid description over time.

To clarify, the key underlying issue here is that there is no reason
to make incubation a free-for-all (people can always create repos on
GitHub), but if it's not a free-for-all, then a decision must be made
somehow. The question is: what grounds is that decision going to be
made on?

RFC’s as described in the writing. This is generally how the LLVM community makes decisions for all the things.

The fewer concrete artefacts we have to make a decision, the more
people will form their opinions based on soft, clique-ish factors such
as whether they like the person making the proposal, whether they
consider them to "already be part of the community", what company or
university they work for (if any), and so on. Clique-ishness is the
opposite of inclusivity, so I feel very strongly that the expectation
should be having at least _some_ basic artefacts here. There might be
the rare legitimate exception to this rule, but that is nicely
captured by making it a "Should" rule.

You seem very afraid of “abuse” here. Given the way the LLVM community dynamics work, I see no reason to design around the possibility of abuse. If a problem arises in practice, we can revise our approach.

Along similar lines, having some sort of manifesto / development plan
/ whatever really should be a Must. Writing something like that up is
an incredibly low bar, and if people can't even be bothered to do
*that*, well... Having that Must next to "there Should be a prototype"
also helps clarify the expectations.

I agree, upgraded to ‘must’ thanks!

As I mentioned before, I'd advocate for the notion of a sponsor (an existing LLVM contributor) for each incubator. I'd have that a must on the incubator list.

Yes, this is a good idea. The problem here is “how do we decide who qualifies as a sponsor?”. I don’t know a good way to say that - someone with N years of LLVM experience, M patches, …? How does this get explained?

Do we really need that? It feels to me like this is much more
something that could be handled as part of the approval process. If
nobody from "the community" (whatever that means) is in favor of the
incubation, then it's not going to happen. So kind of by definition
you need at least one person speaking in favor :slight_smile:

Yes, I don’t think this should be a requirement. It is too difficult to nail down as I tried to convey.

Thank you for the feedback Nicolai!

-Chris

In practice, commit access is pretty easy to get. I really think we can cross this bridge when we come to it. The two proposed projects so far already include experienced LLVM project contributors. If someone proposes a new project that is out of the blue, this concern can be addressed through qualitative concerns brought up in the discussion - it doesn’t have to be codified in the developer policy in my opinion.

-Chris

Thank you to everyone for the feedback. I moved the draft from the google doc below to a Phabricator patch here:
https://reviews.llvm.org/D83182

I’d appreciate any last feedback, thanks!

-Chris

Land yours, and if I still care, I’ll send a patch. :slight_smile:

  1. The must/should terminology should probably be factored out once and referenced. As written, it takes a little effort to be sure the definitions are the same between the two uses.

I’m not sure what you mean here. Do you have a recommended approach?

Land yours, and if I still care, I’ll send a patch. :slight_smile:

WFM! :slight_smile:

  1. As I mentioned before, I’d advocate for the notion of a sponsor (an existing LLVM contributor) for each incubator. I’d have that a must on the incubator list.

Yes, this is a good idea. The problem here is “how do we decide who qualifies as a sponsor?”. I don’t know a good way to say that - someone with N years of LLVM experience, M patches, …? How does this get explained?

You said elsewhere that we could let this evolve with experience. I would take that sentiment, and apply it here. I’m really more concerned about the expectations of the role (i.e. some human familiar with LLVM norms willing to invest non-trivial time), than I am the details of who is eligible.

Since I don’t want this to be blocking item, why don’t we land what you have and I can draft something as a patch? It seems like there’s some general agreement about a potential issue and we just need to find a way to address it.

Sounds great. I think this is something we can handle qualitatively as a concern raised in the RFC stage based on the details of the proposal. The LLVM Developer Policy doesn’t have to be a complete list of criteria, it just provides a general framework to set expectations.

Thank you for the feedback Philip!

-Chris