FYI: LLVM Phabricactor notifications.

Hi,

I think some of contributors to the Clang received a notifications about some commits done in the past.

I wanted to share my thoughts on why it might has happened.

I think the commits from this PR https://github.com/llvm/llvm-project/pull/13 were pulled by Phabricator (probably with aim to review GitHub pull requests in https://reviews.llvm.org/ environment).

The PR has 2000+ commit, most of which are some old commits from the master branch, and when Phabricator pulled the commits from PR, it sent a notification to the commit author (or committer).

Probably we can do something to avoid this situation in the future.

Sorry for the inconvenience.

Alexey

+llvm-dev

+llvm-dev

*From:* cfe-dev [mailto:cfe-dev-bounces@lists.llvm.org] *On Behalf Of *Bader, Alexey via cfe-dev
*Sent:* Thursday, May 30, 2019 7:31 PM
*To:* clang-dev developer list <cfe-dev@lists.llvm.org>
*Subject:* [cfe-dev] FYI: LLVM Phabricactor notifications.
*Importance:* Low

Hi,

I think some of contributors to the Clang received a notifications about some commits done in the past.

I wanted to share my thoughts on why it might has happened.

I think the commits from this PR https://github.com/llvm/llvm-project/pull/13were pulled by Phabricator (probably with aim to review GitHub pull requests in https://reviews.llvm.org/ environment).

The PR has 2000+ commit, most of which are some old commits from the master branch, and when Phabricator pulled the commits from PR, it sent a notification to the commit author (or committer).

Probably we can do something to avoid this situation in the future.

+ Manuel, Chandler

Any idea what happened here? Is phabricator setup to automatically import
pull requests?

-Tom

I believe (and I believe it was James who pointed this out on IRC) that Phabricator pulls in all refs, and GitHub stores PR commits under refs/pull.

    > +llvm-dev
    >
    >
    >
    > *From:* cfe-dev [mailto:cfe-dev-bounces@lists.llvm.org] *On Behalf Of *Bader, Alexey via cfe-dev
    > *Sent:* Thursday, May 30, 2019 7:31 PM
    > *To:* clang-dev developer list <cfe-dev@lists.llvm.org>
    > *Subject:* [cfe-dev] FYI: LLVM Phabricactor notifications.
    > *Importance:* Low
    >
    >
    >
    > Hi,
    >
    >
    >
    > I think some of contributors to the Clang received a notifications about some commits done in the past.
    >
    > I wanted to share my thoughts on why it might has happened.
    >
    >
    >
    > I think the commits from this PR https://github.com/llvm/llvm-project/pull/13were pulled by Phabricator (probably with aim to review GitHub pull requests in https://urldefense.proofpoint.com/v2/url?u=https-3A__reviews.llvm.org_&d=DwIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=AD-hZ9rLcmVgzhl1TFTAvKbX-nrgHD75y9sqN5zTEsg&s=KaCscc77Z-fBuYo2dCuykCeHg35yKQXRwp6uVuIsPos&e= environment).
    >
    >
    >
    > The PR has 2000+ commit, most of which are some old commits from the master branch, and when Phabricator pulled the commits from PR, it sent a notification to the commit author (or committer).
    >
    >
    >
    > Probably we can do something to avoid this situation in the future.
    >
    >
    
    + Manuel, Chandler
    
    Any idea what happened here? Is phabricator setup to automatically import
    pull requests?
    
    -Tom

We should be able to control the noise of these, but I also wonder whether we can switch to github reviews as part of the git move :slight_smile:
(we just lost our long time phab maintainer on the team)

We should be able to control the noise of these

, but I also wonder whether we can switch to github reviews as part of the git move :slight_smile:

(Provocative offtopic. (I personally hope that never ever happens.))

We should be able to control the noise of these

, but I also wonder whether we can switch to github reviews as part of the git move :slight_smile:
(Provocative offtopic. (I personally hope that never ever happens.))

Strong +1 to not switching away from phab.

~Aaron

We should be able to control the noise of these

, but I also wonder whether we can switch to github reviews as part of the git move :slight_smile:
(Provocative offtopic. (I personally hope that never ever happens.))

Strong +1 to not switching away from phab.

Aaron, do you have admin on phab yet? If yes, would you mind to config the notifications?
(honestly, part of my comment was trying to figure out who wants it so badly that they’d be willing to help with maintenance - I have admin bits to hand out :slight_smile:

I actually run a in-house Phabricator instance with 400+ users and multiple git repo including custom extension developments

I also follow the Phabricator development quite closely because we upgrade every couple of weeks

I’d be more than happy to help maintain the phab instance for LLVM

MyDeveloperDay

I actually run a in-house Phabricator instance with 400+ users and multiple git repo including custom extension developments

I also follow the Phabricator development quite closely because we upgrade every couple of weeks

I’d be more than happy to help maintain the phab instance for LLVM

Congrats. You’re now an admin :slight_smile:
(and thanks!!)

I’ll figure out who easy it is to give access to our GCE to external folks, given everything on it is OSS stuff.

I tweaked the notifications yesterday, hopefully we don’t see more spam storms.

For filtering git refs: this is something that has changed substantially in Phabricator over the past couple of months, but I think I’ve managed to restrict notifications to only “master.” (Although, again, this has changed a lot… getting this right depends on my own archaeological skills for digging through old revisions of the Phab documentation.)

There are some complicating factors in deciding what the notifications should actually be doing… one major source of complexity stems from the fact that we re-import most commits multiple times. This makes it hard to reason about how different notifications will fire, and which source will actually autoclose differential revisions.

The current state is that most commits should notify the committer twice: once for the Subversion repo (repo callign rL), and once for git master (rG). I think this may actually be the best state while we’re living in both worlds: the SVN commit is the source of truth, but a lot of folks care about re-syncing from Git, so that also seems like something folks will want. Of course, this is a transitional pain point, and there’s not going to be a single configuration that can satisfy every possible thing folks would want.

One particular change: I’ve disabled notifications for the duplicate Subversion meta-repos… so, for example, a commit to Clang will still get 2 notifications (rL and rG). Before yesterday, this should have sent 3: one each for rL, rG, and rC. Projects not in the monorepo will get notifications for rL only.

One particular change: I’ve disabled notifications for the duplicate Subversion meta-repos… so, for example, a commit to Clang will still get 2 notifications (rL and rG). Before yesterday, this should have sent 3: one each for rL, rG, and rC. Projects not in the monorepo will get notifications for rL only.

I am curious how this plays with what I reported in PR41996. It would be sad if Clang commits were reported to the llvm-commits list, and not cfe-commits.

–paulr

>
> We should be able to control the noise of these

>, but I also wonder whether we can switch to github reviews as part of the git move :slight_smile:
(Provocative offtopic. (I personally hope that never ever happens.))

Strong +1 to not switching away from phab.

Aaron, do you have admin on phab yet? If yes, would you mind to config the notifications?
(honestly, part of my comment was trying to figure out who wants it so badly that they'd be willing to help with maintenance - I have admin bits to hand out :slight_smile:

I don't have admin on Phab but I would be happy to help in any way
needed. I'm only mildly familiar with the app (mostly had to deal with
basic config and setup), however, so I may not be the best admin.

~Aaron

One particular change: I’ve disabled notifications for the duplicate Subversion meta-repos… so, for example, a commit to Clang will still get 2 notifications (rL and rG). Before yesterday, this should have sent 3: one each for rL, rG, and rC. Projects not in the monorepo will get notifications for rL only.

I am curious how this plays with what I reported in PR41996. It would be sad if Clang commits were reported to the llvm-commits list, and not cfe-commits.

–paulr

I’m guessing that the commit mails you’re thinking of are not from Phabricator, but rather directly from the SVN post-commit hook. Those are not affected.

The ones that would be different now are the ones with something like “[PATCH] D123456” in the subject line, or “[Diffusion] rL123456”, where rL is the callsign. You’ll still get emails for callsigns rG and rL, but those (already) generally only go to the committer, and will also get the replies if you comment on the commit through the Phabricator UI.

Hi David,

No, these are Phabricator emails, for a recent example see D62616 which was originally subscribed only to cfe-commits, but then in the final Phabricator email after committing we see:

This revision was automatically updated to reflect the committed changes.
Closed by commit rL362363: [CodeComplete] Add a bit more whitespace to completed patterns (authored by ibiryukov, committed by ).
Herald added a project: LLVM.
Herald added a subscriber: llvm-commits.

There’s no reason for Herald to be adding project LLVM/subscriber llvm-commits at the last second here.

–paulr

Sorry if this is duplicated knowledge, this seem related to:

https://discourse.phabricator-community.org/t/is-there-a-way-to-stop-phabricator-pulling-in-certain-refs/2274
which points to https://secure.phabricator.com/T11314

There has been a recent change (April 2019), This may be somewhat fixed by

https://secure.phabricator.com/rP1cda1402c7c29170167e28e598bc9bf350e96ac5

The changes are outline here in their release notes
https://secure.phabricator.com/w/changelog/2019.17/

But beware this changes the AutoClose behavior (where commits autocloses reviews), most of the LLVM repos have AutoClose on, but rC and rCTE for example do not

As this hit my own Phabricator installation where we had AutoClose off, and we ended up having to make a custom change to match our “post commit” review workflow, it is worth thinking carefully about before upgrading, because I’m not sure of the effect on the 3 repo models + 1 observed (Mono)

MyDeveloperDay (Paul)

PaulR

(sorry again if this is known knowledge)

There’s no reason for Herald to be adding project LLVM/subscriber llvm-commits at the last second here.

Its possible the rL (LLVM) had be added as the repository in the review on creation rather than rCFE, if thats the case then the herald rule “H270” is going to fire because it see the repository in the review, so add LLVM project and llvm-commits as a subscriber automatically. It won’t care that this has gone into rCFE and not rL (I mean it does go into rL via the cfe/trunk but I’m not sure if you want to notify for that)

As there is no mention of the repository being change the revisions feed (https://reviews.llvm.org/D62616) I suspect it was created that way, and its only as the commit fires that it gets added. (it might be clearer if a herald rule so these are added at review creation, although anyone then removing them will get them readded at commit if they still have the incorrect repository.)

MyDeveloperDay (Paul)

Related to the recent Phabricator changes (post April 2019) mentioned above

https://secure.phabricator.com/book/phabricator/article/diffusion_managing/

In the section on Fetch Refs…

Fetch Refs: In Git, if you are observing a remote repository, you can specify that you only want to fetch a subset of refs using “Fetch Refs”.

This may be useful if the remote is on a service like GitHub, GitLab, or Gerrit and uses custom refs (like refs/pull/ or refs/changes/) to store metadata that you don’t want to bring into Phabricator.

I suspect we could use this mechanism to prevent such a problem from happening.

MyDeveloperDay (Paul)

As there is no mention of the repository being change the revisions feed (https://reviews.llvm.org/D62616) I suspect it was created that way, and its only as the commit fires that it gets added. (it might be clearer if a herald rule so these are added at review creation, although anyone then removing them will get them readded at commit if they still have the incorrect repository.)

Note the highlighted part of this quote from the revision-closed email:

This revision was automatically updated to reflect the committed changes.
Closed by commit rL362363: [CodeComplete] Add a bit more whitespace to completed patterns (authored by ibiryukov, committed by ).
Herald added a project: LLVM.
Herald added a subscriber: llvm-commits.

The LLVM project was added at the time Phabricator saw the closing commit, not when the revision was created.

If it had been added at the time the revision was created, all review emails would have gone to both lists. They did not.

–paulr

Sorry maybe I didn’t explain what I saw…(again sorry if this is known already and I’m stating the obvious to people who know better than me.)

The Revision contains rL in the repository field, as such the herald rule will add the LLVM project.and the llvm-commit subscriber whenever the revision is updated.

When the commit AutoCloses, it updates the Revision, as such if the revision still has the rL repository field then I believe the rule will fire again and add them. (post commit)

This revision was automatically updated to reflect the committed changes.

H270 Tag SVN LLVM revisions as LLVM

Conditions
Passed Repository is any of rCRT, rL, rLLD, rPLO, rT
Passed Revision title does not contain [private]
Passed Rule passed.
Action: Add projects
Added Projects Added a project: LLVM.
Action: Add subscribers
Added Subscribers Added a subscriber: llvm-commits.

Its true I’m a little surprised that the Herald rule hadn’t fired earlier and added the LLVM project and llvm-commit subscriber already, unless somehow it had, but they were removed manually, but I see no trace of that in the history.

Potentially you could extend the rule to say “Revision status is not any of Accepted”, but I guess the whole point of that rule is to tell people watching rL that a commit for a revision marked as being part of rL has been changed

Unfortunately there isn’t any connection between the Revision repository and the actual repo its committed to.

I do notice LLVM doesn’t use the “Owners” application in Phabricator, this is an great way of ensuring code is automatically channeled to code owners/reviews for a particular area via the “Affected packages” in a hearld rule, and can be used to automate the adding of reviewers, blocking reviews, adding projects (but alas it doesn’t allow the setting of the repository from what I can tell)

MyDeveloperDay (Paul)