Policy on support tiers

Hi all,

It seems the BAZEL[1] discussion is going round in circles and many have suggested we encode the policy on the tiers of support we want from the community.

This is not what I think is the best, just what I have interpreted the “consensus” seems to think is the best. I may be completely wrong, please don’t assume anything other than a faulty attempt at getting it right.

I tried to use different words like “must”, “should”, “could”, “will”, etc. to convey the right meaning, but I’m not a native English speaker, so feel free to tweak that to get the right balance.

Here it goes.

*** Tier 1: the core compiler, which must work at all times.

  • Front-ends, back-ends, middle-ends, libraries, core APIs, official projects and tools.
  • The CMake build infrastructure.
  • Builds on all first class citizen combinations of targets x OSs. [2]
  • Vim support (kidding!! :slight_smile:

It must:

  • have noisy green bots
  • revert on failure policy
  • etc?

*** Tier 2: side projects that integrate with the compiler and that should work at all times.

  • Experimental targets, infrastructure and APIs, non-default cmd-line options
  • Alternative build systems (ex. GN, Bazel)

It must not:

  • Break or hinder tier 1
  • Increase validation noise beyond its scope

It should:

  • Have green buildbots, handled by the sub-community that cares about it
  • Have notifications only to those interested when things break (avoid bitrot)

Sub-communities that care about it must fix issues in them, but the rest of the community has no obligations to support it. Lack of maintenance could be subject to removal.

*** Tier 3: miscellaneous accessories

  • Helper scripts, editor files, debugger scripts, etc.
  • Whatever is detached enough that bit rot is irrelevant

It must not:

  • Break or hinder tiers 1 or 2
  • Increase validation noise beyond its scope

It should:

  • Have a sub-community that cares about it and maintain it

Sub-communities that care about it should fix issues in them, but the rest of the community has no obligations to support it. Lack of maintenance will be subject to removal.

cheers,
–renato

[1] http://lists.llvm.org/pipermail/llvm-dev/2020-October/146138.html
[2] Existing definition of “first-class”, could be updated / moved here.

+Geoffrey Martin-Noble

+Geoffrey Martin-Noble

Hi all,

It seems the BAZEL[1] discussion is going round in circles and many have suggested we encode the policy on the tiers of support we want from the community.

This is not what I think is the best, just what I have interpreted the “consensus” seems to think is the best. I may be completely wrong, please don’t assume anything other than a faulty attempt at getting it right.

I tried to use different words like “must”, “should”, “could”, “will”, etc. to convey the right meaning, but I’m not a native English speaker, so feel free to tweak that to get the right balance.

Here it goes.

*** Tier 1: the core compiler, which must work at all times.

  • Front-ends, back-ends, middle-ends, libraries, core APIs, official projects and tools.
  • The CMake build infrastructure.
  • Builds on all first class citizen combinations of targets x OSs. [2]
  • Vim support (kidding!! :slight_smile:

It must:

  • have noisy green bots
  • revert on failure policy
  • etc?

*** Tier 2: side projects that integrate with the compiler and that should work at all times.

  • Experimental targets, infrastructure and APIs, non-default cmd-line options
  • Alternative build systems (ex. GN, Bazel)

It must not:

  • Break or hinder tier 1
  • Increase validation noise beyond its scope

It should:

  • Have green buildbots, handled by the sub-community that cares about it
  • Have notifications only to those interested when things break (avoid bitrot)

Sub-communities that care about it must fix issues in them, but the rest of the community has no obligations to support it. Lack of maintenance could be subject to removal.

*** Tier 3: miscellaneous accessories

  • Helper scripts, editor files, debugger scripts, etc.
  • Whatever is detached enough that bit rot is irrelevant

I think bit rot is never really totally “irrelevant”. Perhaps “detached and small enough that bit rot is a minor concern”?

It must not:

  • Break or hinder tiers 1 or 2
  • Increase validation noise beyond its scope

It should:

  • Have a sub-community that cares about it and maintain it

Sub-communities that care about it should fix issues in them, but the rest of the community has no obligations to support it. Lack of maintenance will be subject to removal.

cheers,
–renato

[1] http://lists.llvm.org/pipermail/llvm-dev/2020-October/146138.html
[2] Existing definition of “first-class”, could be updated / moved here.


LLVM Developers mailing list
llvm-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Thanks for kicking this off Renato. This looks very similar to what I would’ve written after that discussion except that I’m not sure I see the clear distinction between when something should be considered tier 2 vs 3. It seems like if something is sufficiently big it’s not allowed to be tier 3 and must have a buildbot and remain green? And tier 2 seems to be largely an extension of the “experimental targets” policy, but encompassing things that aren’t targets.

Some additional constraints I would add:

  1. Something in a higher tier may never depend on something in a lower tier.
  2. Items in tiers <1 must always be simple to remove (rm -rf /path/to/thing ideally, although experimental backends also include some CMake options).
  3. Items in tiers <1 must have clear documentation accompanying them indicating their less-supported status.

So, I thought that the whole idea of the ‘incubator’ was to take up tiers 2/3 until they were sufficiently baked for tier 1. Is this intended to go against that for some things (that is, Bazel in this case)?

I personally see a difference between “subprojects” (like lld, clang, flang, circt …) which are conceptually either independent from LLVM or built on top from a layering point of view ; and on the other hand things like the IDE, debugger, build scripts which are about LLVM itself and intended to serve users of LLVM.
The incubator process as I understand it has been introduced to address the former more than the latter.

+1 :slight_smile:

-eric

Exactly!

I think bit rot is never really totally “irrelevant”. Perhaps “detached and small enough that bit rot is a minor concern”?

Good point. Minor concern is a good alternative.

Thanks for kicking this off Renato. This looks very similar to what I would’ve written after that discussion except that I’m not sure I see the clear distinction between when something should be considered tier 2 vs 3. It seems like if something is sufficiently big it’s not allowed to be tier 3 and must have a buildbot and remain green? And tier 2 seems to be largely an extension of the “experimental targets” policy, but encompassing things that aren’t targets.

Right, I think I’m trying to separate things that interact with the core code from the things that don’t. I think this is important because the evolution of tier 2 should be a lot closer to mainline, while tier 3 could be many commits behind and still be useful, even if it needs patches to reapply to mainline. For example, when new files are added, the Bazel files need to change, while an editor or debugger configuration will be largely the same, and if not, it won’t break anything other than the very specific usage that it was intended for.

Perhaps I’m going in circles, I hope someone has a better way of describing the difference. If not (ie it’s just in my head), then I’d be happy to merge the two tiers in one.

Some additional constraints I would add:

  1. Something in a higher tier may never depend on something in a lower tier.
  2. Items in tiers <1 must always be simple to remove (rm -rf /path/to/thing ideally, although experimental backends also include some CMake options).
  3. Items in tiers <1 must have clear documentation accompanying them indicating their less-supported status.

All good suggestions, thanks!

–renato

I think bit rot is never really totally “irrelevant”. Perhaps “detached and small enough that bit rot is a minor concern”?

Good point. Minor concern is a good alternative.

Thanks for kicking this off Renato. This looks very similar to what I would’ve written after that discussion except that I’m not sure I see the clear distinction between when something should be considered tier 2 vs 3. It seems like if something is sufficiently big it’s not allowed to be tier 3 and must have a buildbot and remain green? And tier 2 seems to be largely an extension of the “experimental targets” policy, but encompassing things that aren’t targets.

Right, I think I’m trying to separate things that interact with the core code from the things that don’t. I think this is important because the evolution of tier 2 should be a lot closer to mainline, while tier 3 could be many commits behind and still be useful, even if it needs patches to reapply to mainline. For example, when new files are added, the Bazel files need to change, while an editor or debugger configuration will be largely the same, and if not, it won’t break anything other than the very specific usage that it was intended for.

Perhaps I’m going in circles, I hope someone has a better way of describing the difference. If not (ie it’s just in my head), then I’d be happy to merge the two tiers in one.

Yeah I think I have a similar feeling about the two categories, but couldn’t really see logically what it was or why it necessarily implied a hierarchy. Like the hierarchy would suggest to me that something from tier3 could be “upgraded” to tier2, just as an experimental tier2 target could become tier1, but I’m not sure it really makes sense that there would actually be any movement between those tiers. Vim configs might be more widely used than gn build files, so it seems a bit strange that they’re “lower tier”, but I guess that may be true of experimental backends as well. I think the point you articulated makes sense. Something that degrades gracefully doesn’t need to have as much support commitment. I guess the tiers are not about the level of the support something receives so much as the level of support that is required for it to be allowed in the repo. Editor configs just need some people to care about them and keep them reasonably up to date, whereas experimental targets or build systems also need people to agree that they will keep them green (plus the requirements already described in https://llvm.org/docs/DeveloperPolicy.html#adding-a-new-target). I think code size is also a reasonable proxy for this.

Yeah I think I have a similar feeling about the two categories, but couldn’t really see logically what it was or why it necessarily implied a hierarchy. Like the hierarchy would suggest to me that something from tier3 could be “upgraded” to tier2, just as an experimental tier2 target could become tier1, but I’m not sure it really makes sense that there would actually be any movement between those tiers.

Ah, I see the confusion. I didn’t mean 2 is “better” than 3, just that it should be harder to remove stuff in tier 2 than stuff in tier 3, even if they bitrot a bit. I don’t see any routes of “promotion” between tiers.

For example, a (completely hypothetical) replacement of CMake with Bazel would need a whole new series of RFCs and years of work that doesn’t count like a promotion, but an entirely new work.

I think the point you articulated makes sense. Something that degrades gracefully doesn’t need to have as much support commitment.

There’s the term I was looking for: “degrades gracefully”. Thanks! :slight_smile:

I guess the tiers are not about the level of the support something receives so much as the level of support that is required for it to be allowed in the repo. Editor configs just need some people to care about them and keep them reasonably up to date, whereas experimental targets or build systems also need people to agree that they will keep them green (plus the requirements already described in https://llvm.org/docs/DeveloperPolicy.html#adding-a-new-target). I think code size is also a reasonable proxy for this.

Exactly!

–renato