Mailing list changes this week

Hi,

We are going to start to switching from SVN commit emails to GitHub commit
emails this week. The only real change you should notice is that
the revision number in the subject will be replaced with a git hash and
the diff links in the email will point to GitHub. Otherwise the
content and format of the email should be the same.

We are going to start by rolling this out for the openmp-commits list
and then once that's working begin migrating the rest of the lists. If you
notice any issues with the new emails, please file a bug and mark it
as a blocker of the github meta-bug (llvm.org/PR39393).

Thanks,
Tom

Hi Tom.

One issue with this is that we don’t have a clear “ordering” from linear revision numbers from these emails. Have we looked into continuing to generate our own emails per commits instead so that we control the format?

Thanks,

Hi Tom.

One issue with this is that we don't have a clear "ordering" from linear revision numbers from these emails. Have we looked into continuing to generate our own emails per commits instead so that we control the format?

This actually what we are doing, we are listening for github commit events and
then generating our own emails based on the data in the event. We can format
the emails how ever we want, and we tried to match the current SVN format exactly.
Is the some other information you would like to have in the emails to show the
ordering?

-Tom

Hi Tom.

One issue with this is that we don’t have a clear “ordering” from linear revision numbers from these emails. Have we looked into continuing to generate our own emails per commits instead so that we control the format?

This actually what we are doing, we are listening for github commit events and
then generating our own emails based on the data in the event. We can format
the emails how ever we want, and we tried to match the current SVN format exactly.

Ah great!

Is the some other information you would like to have in the emails to show the
ordering?

The only thing I was looking to get was to continue to have a monotonic incrementing integer for the revision instead of the git hash alone: I don’t know if git llvm has this feature yet but this was discussed a while ago (I don’t remember if we just mentioned counting the commits in the repo from the beginning or using an invocation of git describe or something derived).

    > Hi Tom.
    >
    > One issue with this is that we don't have a clear "ordering" from linear revision numbers from these emails. Have we looked into continuing to generate our own emails per commits instead so that we control the format?
    >

    This actually what we are doing, we are listening for github commit events and
    then generating our own emails based on the data in the event. We can format
    the emails how ever we want, and we tried to match the current SVN format exactly.

Ah great!

    Is the some other information you would like to have in the emails to show the
    ordering?

The only thing I was looking to get was to continue to have a monotonic incrementing integer for the revision instead of the git hash alone: I don't know if `git llvm` has this feature yet but this was discussed a while ago (I don't remember if we just mentioned counting the commits in the repo from the beginning or using an invocation of `git describe` or something derived).

We talked about using `git describe` for this, but this would require that we
add tags to the master branch each time the version number was bumped. We
discussed this[1] last year, but deferred the decision, since we couldn't get
consensus on the tag name.

-Tom

[1] http://lists.llvm.org/pipermail/llvm-dev/2018-December/128484.html

I thought we were just going to count commits on a particular branch and use the (branch name, commit count) tuple as our monotonic incrementing identifier? https://llvm.org/docs/Proposals/GitHubMove.html#on-managing-revision-numbers-with-git

Hi,

Is this monotonic incrementing identifier part of the email? I think
it would really complicate buildbot maintenance if we didn't have it
in obvious places. For instance, we have a (hacky) monitoring page for
our ARM/AArch64 buildbots [1] and it really helps a lot to just look
at the commit range for the last build to know if a fix actually
worked or not ("is the fix commit in that range? if not wait a bit
more before investigating"). Using just a git hash and having to
manually count commits would be a waste of time.

Just my 2c,
Diana

[1] http://llvm.validation.linaro.org/

+1. And put it in the email (subject?). While it’s possible to derive a count from a hash manually, better to have it in the email in the first place. You can’t rely on order-of-email-delivery to reflect order-of-commit.

–paulr

+1, please.

Also, putting a tag on the *first* commit in the repo,
and doing `git describe --match FIRST_COMMIT_TAG` will be *great*!

Roman.

+1, please.

Also, putting a tag on the *first* commit in the repo,
and doing `git describe --match FIRST_COMMIT_TAG` will be *great*!

Do we need to add a tag or is `git rev-list --count HEAD`
good enough?

-Tom

> +1, please.
>
> Also, putting a tag on the *first* commit in the repo,
> and doing `git describe --match FIRST_COMMIT_TAG` will be *great*!
>

Do we need to add a tag or is `git rev-list --count HEAD`
good enough?

Ah, interesting, that works too, better than nothing.
But to be noted that number is different from the `llvm-svn: <>` for
the current commit.

-Tom

Roman

+1, please.

Also, putting a tag on the *first* commit in the repo,
and doing `git describe --match FIRST_COMMIT_TAG` will be *great*!

Do we need to add a tag or is `git rev-list --count HEAD`
good enough?

Ah, interesting, that works too, better than nothing.
But to be noted that number is different from the `llvm-svn: <>` for
the current commit.

This is expected. There were a few commits that were filtered
out in the SVN to git conversion:

https://github.com/jyknight/llvm-git-migration/blob/master/llvm-svn2git-monorepo.rules

-Tom

From: Tom Stellard <tstellar@redhat.com>
Sent: Wednesday, October 16, 2019 3:14 PM
To: Roman Lebedev <lebedev.ri@gmail.com>
Cc: Robinson, Paul <paul.robinson@sony.com>; Shoaib Meenai
<smeenai@fb.com>; Mehdi AMINI <joker.eph@gmail.com>; llvm-
dev@lists.llvm.org; cfe-dev <cfe-dev@lists.llvm.org>; openmp-dev (openmp-
dev@lists.llvm.org) <openmp-dev@lists.llvm.org>; LLDB Dev <lldb-
dev@lists.llvm.org>
Subject: Re: [llvm-dev] [cfe-dev] Mailing list changes this week

>>
>>> +1, please.
>>>
>>> Also, putting a tag on the *first* commit in the repo,
>>> and doing `git describe --match FIRST_COMMIT_TAG` will be *great*!
>>>
>>
>> Do we need to add a tag or is `git rev-list --count HEAD`
>> good enough?
> Ah, interesting, that works too, better than nothing.
> But to be noted that number is different from the `llvm-svn: <>` for
> the current commit.
>

This is expected. There were a few commits that were filtered
out in the SVN to git conversion:

https://github.com/jyknight/llvm-git-migration/blob/master/llvm-svn2git-
monorepo.rules

Does the same tactic work for the release branches? Or are we going to
see some duplicate count numbers on master and the branches? That might
be confusing. (FYI at Sony, for release we count revs since the branch,
so the monotonic numbers on branches are generally small.)
--paulr

From: Tom Stellard <tstellar@redhat.com>
Sent: Wednesday, October 16, 2019 3:14 PM
To: Roman Lebedev <lebedev.ri@gmail.com>
Cc: Robinson, Paul <paul.robinson@sony.com>; Shoaib Meenai
<smeenai@fb.com>; Mehdi AMINI <joker.eph@gmail.com>; llvm-
dev@lists.llvm.org; cfe-dev <cfe-dev@lists.llvm.org>; openmp-dev (openmp-
dev@lists.llvm.org) <openmp-dev@lists.llvm.org>; LLDB Dev <lldb-
dev@lists.llvm.org>
Subject: Re: [llvm-dev] [cfe-dev] Mailing list changes this week

+1, please.

Also, putting a tag on the first commit in the repo,
and doing git describe --match FIRST_COMMIT_TAG will be great!

Do we need to add a tag or is git rev-list --count HEAD
good enough?
Ah, interesting, that works too, better than nothing.
But to be noted that number is different from the llvm-svn: <> for
the current commit.

This is expected. There were a few commits that were filtered
out in the SVN to git conversion:

https://github.com/jyknight/llvm-git-migration/blob/master/llvm-svn2git-
monorepo.rules

Does the same tactic work for the release branches? Or are we going to
see some duplicate count numbers on master and the branches? That might
be confusing. (FYI at Sony, for release we count revs since the branch,
so the monotonic numbers on branches are generally small.)
–paulr

If we always display the revisions as a tuple number+branch (r1234-master vs r1234-release9) is it good enough?

+1. And put it in the email (subject?). While it’s possible to derive a count from a hash manually, better to have it in the email in the first place. You can’t rely on order-of-email-delivery to reflect order-of-commit.

I spent some time today looking into what it would take to add the commit
number into the email. Implementing this will add some extra complexity to the
emailer script and add another point of failure for us. We also
can't guarantee to always have it since at some point we may want to start using
github's standard commit emails.

I think we should just wait and see how things go without having
a commit number in the email. It's easy to generate the number
locally from a git hash if needed. If it becomes a major inconvenience
to not have it in the email, we can always look at adding it in later.

-Tom

Having to get an up-to-date local clone and run commands to be able to reason about the logical relationship between commits (does this build failure email from a slow bot comes from before or after I landed my revert?) seems to me like a non-trivial workflow regression. I would personally be OK to increase the tooling complexity to preserve this.

The best way to prove or disprove this is likely do what you suggest though, and live through this for some time :slight_smile:

+1 on this, but it’s worth clarifying this is definitely not a blocker. Just a nice to have which can easily be done after the switch if needed.

    > +1. And put it in the email (subject?). While it’s possible to derive a count from a hash manually, better to have it in the email in the first place. You can’t rely on order-of-email-delivery to reflect order-of-commit.
    >

    I spent some time today looking into what it would take to add the commit
    number into the email. Implementing this will add some extra complexity to the
    emailer script and add another point of failure for us. We also
    can't guarantee to always have it since at some point we may want to start using
    github's standard commit emails.

    I think we should just wait and see how things go without having
    a commit number in the email. It's easy to generate the number
    locally from a git hash if needed. If it becomes a major inconvenience
    to not have it in the email, we can always look at adding it in later.

Having to get an up-to-date local clone and run commands to be able to reason about the logical relationship between commits (does this build failure email from a slow bot comes from before or after I landed my revert?) seems to me like a non-trivial workflow regression. I would personally be OK to increase the tooling complexity to preserve this.

There are other ways to solve this, though, for example we could have
the bots pass/fail the status checks for commits, so you could answer
the question just by clicking the link in the email and looking at
which checks have passed or failed. And I think overall this would
be better than what we have now.

I think there may be other cases like this were problems solved using
the commit numbers may be able to be solved in different and better ways.
Part of the reason to move to GitHub is to be able to take advantage
of features like this, and I think continuing to use the commit numbers
may hold us back a little from trying new things.

The best way to prove or disprove this is likely do what you suggest though, and live through this for some time :slight_smile:

Right, and I'm not sure we would even be able to get the changes done in
time for the switch over anyway.

-Tom