Policy on support tiers

Project Infrastructure LLVM Dev List Archives
#1

Thanks for taking this initiative Renato! While this doesn’t completely alleviate my concerns with the addition of the Bazel build system, I think this proposal is valuable in and of itself. It’s worthwhile to have documented policies so that community members who are not in the in-group can point to a violation and state that it’s wrong without needing to have an excess level of clout or coalition to lend their words weight. I’ve suggested a few changes inline below, but I think this is largely reasonable.

I think talk of committing the Bazel build system to the repo should be tabled until we can ratify this policy, and then it should be re-proposed in terms of how it fits in with this policy. If Bazel is accepted into the repository in conformance with an existing policy that can be enforced, my misgivings would be lessened.

Thanks,

Christopher Tetreault

*** Tier 1: the core compiler, which must work at all times.

  • Front-ends, back-ends, middle-ends, libraries, core APIs, official projects and tools.
  • The CMake build infrastructure.
  • Builds on all first class citizen combinations of targets x OSs. [2]

It must:

  • have noisy green bots
  • revert on failure policy
  • etc?

*** Tier 2: side projects that integrate with the compiler and that should work at all times.

  • Experimental targets, infrastructure and APIs, non-default cmd-line options
  • Alternative build systems (ex. GN, Bazel)

I would suggest: that it must have a subcommunity that cares about it

It must not:

  • Break or hinder tier 1
  • Increase validation noise beyond its scope

It should:

  • Have green buildbots, handled by the sub-community that cares about it
  • Have notifications only to those interested when things break (avoid bitrot)

Sub-communities that care about it must fix issues in them, but the rest of the community has no obligations to support it. Lack of maintenance could be subject to removal.

I would suggest: that lack of maintenance will be cause for removal. Perhaps a timetable should be stated? (“if it’s red for more than 1 month, it is subject to removal?”)

*** Tier 3: miscellaneous accessories

  • Helper scripts, editor files, debugger scripts, etc.
  • Whatever is detached enough that bit rot is irrelevant

It sounds to me like the basic difference between tier 2 and 3 is that tier 2 needs to have a (quiet) build bot, and tier 3 does not need a build bot? If so, we should state the requirement for a public build bot under tier 2.

#2

Dangerous statement!!!

In case non-English speakers are unaware, "to table" means the EXACT OPPOSITE depending on whether you speak English or American.

Okay, I know you mean American, but in English it means "let's discuss it *now*".

Cheers,
Wol

#3

Hi Chris,

Before I start to reply inline, your editor is doing something funky and it’s breaking the threading on GMail. I’m not sure other clients have the same problem, but it seems other people are finding the same issue. Can you check what’s going wrong?

I think talk of committing the Bazel build system to the repo should be tabled until we can ratify this policy, and then it should be re-proposed in terms of how it fits in with this policy. If Bazel is accepted into the repository in conformance with an existing policy that can be enforced, my misgivings would be lessened.

With the warning about what this means (to wait), I agree and that’s my intent. To discuss this now and then the introduction of Bazel files will become trivial.

We’ve done the same for new back-ends, sub-projects, incubator projects, etc. It’s so much easier that way.

*** Tier 1: the core compiler, which must work at all times.

I would suggest: that it must have a subcommunity that cares about it

Good point. And they are responsible for maintaining it, like existing policies on code.

Sub-communities that care about it must fix issues in them, but the rest of the community has no obligations to support it. Lack of maintenance could be subject to removal.

I would suggest: that lack of maintenance will be cause for removal. Perhaps a timetable should be stated? (“if it’s red for more than 1 month, it is subject to removal?”)

Right, the “could” here is to sound weaker than tier 3. Weaker not in that it may not remove and let it rot, but in that we’ll need more rot before we remove.

I agree the idea is “will” with a different time frame / necessary effort.

It sounds to me like the basic difference between tier 2 and 3 is that tier 2 needs to have a (quiet) build bot, and tier 3 does not need a build bot? If so, we should state the requirement for a public build bot under tier 2.

Not exactly. In another response I tried to make the difference explicit: 2 interacts with code (use it, build it, is used by it) while 3 is accessories (scripts, config files, etc). The need for silent buildbots is a consequence of that, not the main difference.

It’s not clear we need that difference, and if not, I’m happy to drop it.

But I wanted to express: “3 can’t break 2 can’t break 1”, and make it easier to drop silly scripts or config files if they’re no longer valid or maintained, more easily than dropping entire build systems.

cheers,
–renato

#4

Renato,

By your earlier requirements for tier 1, not seeing how tier 2 could break tier 1. If a file was in tier 2, it would not be in tier 1. There would be no action in tier 1 on tier 2 files to cause a break.

What this may mean is that those working in tier 2 do not wander into tier 1, obscuring that distinction. Or that there may be some confusion about what was in tier 1 vs. 2. That in some manner the distinction becomes not well defined.

Neil Nelson

#5

Examples of tier 2 breaking 1 are when a change in its code need & warrant a change in common apis (in tier 1), and those changes break unrelated tier 1 code.

I agree it’s not super well defined, but I really think we shouldn’t try to list what is in/out of each tier and I’m hoping people can help me define the scope of each.

Questions like yours are super helpful to do just that.

Thanks!
Renato

#6

Reading this, I'm not suggesting any change to policy, but simply to the "official" wording.

Let's do something along the lines of

Tier 1 is LLVM.
Tier 2 is Clang, and Cmake, and all the other stuff that was mentioned that is a first class citizen and supported as part of the project.
Tier 3 is maybe incubator projects heading for tier 2
Tier 4 is projects like Bazel which we have no intention of supporting but are best supported as part of the project ...

Then we define all our policies along the lines of

No tier should trigger warnings or errors in a higher tier.
If a tier requires changes in a higher tier, they are responsible for making those changes (bit like if you change an API in the kernel, you have to fix everywhere else).

And so on. That way, the rules are consistent everywhere, and if some other class of project appears, we can just slot them straight in by creating a new tier.

Cheers,
Wol

#7

Tier 1 is LLVM.

It’s hard to define what LLVM is. There are things in the “llvm” directory that would definitely belong to lower tiers (helper scripts, build files, experimental code). Experimental back ends shouldn’t break core LLVM, but they remain in “llvm/lib/Target” together with others.

Tier 2 is Clang, and Cmake, and all the other stuff that was mentioned

I’d say clang and lld, lldb, RT, libcxx are all tier 1. Flang, once it reaches production quality, will also, but not because of the tier system, because of the experimental policy.

I want to make sure it’s clear I’m not (yet) proposing a tiered system to extend the experimental or incubator policies.

that is a first class citizen and supported as part of the project.
Tier 3 is maybe incubator projects heading for tier 2

I may be wrong, but I personally don’t think we should include incubator projects here. My main concern is the mono-repo and whatever wants to “get in” needs to follow the same rules.

Incubator projects are by definition user or LLVM and we already have a (similar) policy for those.

Tier 4 is projects like Bazel which we have no intention of supporting
but are best supported as part of the project …

I think this is where it gets fuzzy. I personally have no intention of supporting Bazel, but there are many people in the community that very much do. I would be sad if Vim support broke, and would probably try to fix it as soon as possible, but I’m aware that not everyone is enlightened in their choice of editors like me. :smiley:

In summary, I don’t think the message is “we have no intention of supporting”, but “only part of the community will support, and therefore, that part of the community is responsible for not making it break other parts”.

Perhaps having a numbered tier structure sends the wrong message. Perhaps we should have one core tier that everyone must care about, and satellite tiers that are just different subsets of the community.

And so on. That way, the rules are consistent everywhere, and if some
other class of project appears, we can just slot them straight in by
creating a new tier.

Right. If we have only two tiers, but the second tier covers all subsets and all new stuff that is not core would be a “satellite”, the policy would be simpler to state and keep.

If much later on we extend the satellite concept to experimental and incubator projects, this could be the thread that leads to a unified policy.

But only after we reach a consensus on what we think a tier is. Probably even after we write a tier policy and let it simmer for a while. Definitely not in this thread.

–renato