RFC: Removing the "Require Pull Request" bypass

This is a follow up to RFC: Require Pull Requests for all llvm-project commits. Following the Project Council decision, we have enabled the “Require a pull request before merging” ruleset and granted everyone the ability to bypass this rule.

The next step is to remove the bypass option. Based on the discussion in Project Council and Infrastructure Area Team, our proposal is:

  • 3 March 2026: Removing the bypass option for 24 hours as a trial period. The goal is to uncover any remaining issues before we remove the bypass option permanently.
  • 31 March 2026: Removing the bypass option permanently. This date could be moved back if we uncover issues during the trial period and need more time to address them.

Note that this change means you will need to create a PR rather than pushing to main directly, but we do not require approval prior merging the PR. If you rely on pushing changes to main directly, please try the new git-llvm-push script which automates the workflow of creating and merging the PR. If you run into any problems with the script, please file an issue and we’ll look into it as soon as possible.

@petrhosek on behalf of the @infra-area-team.

6 Likes

Can a summary of the follow-up actions taken to address concerns such as excessive notifications of reviewer groups be posted or linked here?

Also, is 24 hours necessarily enough of a window for all impacts to be apparent?

This is the summary of the changes we have made so far to address the concerns raised in RFC: Require Pull Requests for all llvm-project commits:

The primary goal of the trial period is to identify bots and other kinds of automated workflows that rely on direct commit access and we hope that 24 hours would be sufficient.

2 Likes

Since there were no objections, I removed the bypass rule for the main branch and will restore it after 24 hours. Please let me know if you encounter any issues.

1 Like

It looks like this change now prevents landing PRs with failing premerge CI (where there was previously an opt-out option for that).

[CI] Test failing fast by boomanaiden154 · Pull Request #184423 · llvm/llvm-project · GitHub is a test PR for that.

I can only enable and disable automerge whereas previously one was able to click a checkbox that had a warning and merge anyways.

2 Likes

Thanks for the report, this should be resolved now.

I noticed that we can’t now merge PR that a member of LLVM marked as “Request Changes”.
Could this change affected it?

I believe this is expected: whoever raised concerns should clear them first and accept PR, no?

A lot of long-lived PRs will have one reviewer request changes at the start and then never come back. Is the new expectation that we hunt them down before we can merge?

1 Like

Absolutely not. The reviewer bandwidth is way oversaturated to expect that in most situations

This is not expected. The only change in behavior that was expected as the result of this change was that directly pushing to main would now error.

I think it’s a major technical vulnerability that any of 3k+ llvm members can block PR indefinitely.
A reviever can suddenly dissapear due to any personal reasons and at least maintainers should be able to “bypass merge”. It looks like a topic for another RFC.

Since this behavior was not anticipated and not discussed previously, we need to do that and get consensus/understanding on it, prior to rolling this change out again.

However, after I’ve looked into how this works a bit: I actually think it’ll be a good change for LLVM. It is not a hard block on submit (that would, indeed, be terrible for us!). Rather, anyone who has write access may dismiss the review, if they wish, without waiting for the original reviewer to re-review – but you must dismiss it with an explicit action, writing a comment as to why you’re doing so, before you can submit.

See the GitHub docs. I’ve just tested it to make sure it does work this way, by adding a blocking review on [CI] Test failing fast by boomanaiden154 · Pull Request #184423 · llvm/llvm-project · GitHub, and having @boomanaiden154-1 “dismiss” my review; it indeed did unblock the ability to submit the PR.

Typically, I would expect someone to do so if believe they’ve resolved the issues the reviewer had with the change, and the original reviewer has not responded back in a reasonable period of time. We might need to add that expectation to our developer policy – without an explicit permission, people might be unnecessarily scared of pressing the “dismiss review” button, or might never find the button!

2 Likes

Thanks for investigating this @jyknight! I have also experimentally confirmed using [CI] Test failing fast by boomanaiden154 · Pull Request #184423 · llvm/llvm-project · GitHub that this was indeed caused by the change to the ruleset bypass which was not expected.

There’s a potential solution: I re-introduced the bypass but changed it to only apply to pull requests.

In my local testing, this does seem to have the desired effect.

I agree with @jyknight thought that having to explicitly “dismiss” the blocking review with a reason seems preferable as a project policy.

Is it possible this is not working as expected? For example, I created [VPlan] Add disable-output to tests using vplan-print-after. by fhahn · Pull Request #184586 · llvm/llvm-project · GitHub with the skip-precommit-approval label, but it still got labels auto-assigned.

The label is still assigned, but it does not trigger the comment with the ping to @llvm/pr-subscribers-llvm-transforms.

1 Like

I used git-llvm-push for the first time. My branches are origin and local. I had to pass this in manually, but I feel like the tool should at least be able to detect the LLVM repository. Detecting the user’s branch would be ambiguous, but you could let the user select one of multiple in the default case. The current failure mode for this isn’t very intuitive, also I’m thinking it would be nicer if the tool queried you for an auth token instead of requiring and environment variable. If gh is in the path you could probably just use that directly.

Error running command: git rev-list --reverse upstream/main..HEAD
--- stderr ---
fatal: ambiguous argument 'upstream/main..HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'

Returning to original branch: TestFix
Traceback (most recent call last):
  File "llvm/utils/git-llvm-push", line 704, in <module>
    main()
    ~~~~^^
  File "llvm/utils/git-llvm-push", line 698, in main
    automator.run()
    ~~~~~~~~~~~~~^^
  File "llvm/utils/git-llvm-push", line 542, in run
    commits = self._get_commit_stack()
  File "llvm/utils/git-llvm-push", line 436, in _get_commit_stack
    result = self.runner.run_command(
        ["git", "rev-list", "--reverse", f"{target}..HEAD"],
    ...<2 lines>...
        read_only=True,
    )
  File "llvm/utils/git-llvm-push", line 109, in run_command
    raise e
  File "llvm/utils/git-llvm-push", line 91, in run_command
    return subprocess.run(
           ~~~~~~~~~~~~~~^
        command,
        ^^^^^^^^
    ...<4 lines>...
        env=env,
        ^^^^^^^^
    )
    ^
  File "/usr/lib/python3.14/subprocess.py", line 577, in run
    raise CalledProcessError(retcode, process.args,
                             output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['git', 'rev-list', '--reverse', 'upstream/main..HEAD']' returned non-zero exit status 128

Also, merging this gave me a “[llvm/llvm-project] PR run cancelled: CI Checks” email which I’d prefer not to need to deal with.

By branch here you mean remote? Given we can’t know statically in the case of multiple non llvm/llvm-project remotes which remote to choose, I’m not sure we want to get into the business of choosing remotes. I’ve been planning to make the tool read from a git config section that I think would solve this problem though.

The failure messages could definitely be improved in this case. Automatic token querying also makes me a bit nervous, but pulling in from gh could maybe be another config option.

I don’t think there is anything we can do about this.

What I mean is that we should be able to detect what the script currents assumes is upstream/ by checking for llvm/llvm-project. The others could be a drop-down selection similar to how gh pr create will do it, but I don’t know how much effort people want to put into making this script user friendly.

This is quite unfortunate if so, I tended to use direct merges to avoid email spam like this.

I understand what you meant.

A git config approach that lets you specify the options once I still think is a better solution. It’s one and done versus needing to select every time you want to push a commit. Of course we could look at supporting both, but that’s additional effort that might be better spent elsewhere. If documentation/failure messages can point at the git config solution, I think that would probably suffice.