Project Release Branches

Dear Pawel and All,

Pawel (and other release managers), can you ask the project maintainers for each project whether they want a release branch before creating it next time? Alternatively, can you leave branch creation up to the project maintainers as in the past?

Pawel, you created a release_32 branch of SAFECode last night without my consultation. That commit itself is fine since we just happen to be tracking LLVM mainline right now; I'll just tell SAFECode developers to move development to the release_32 branch. However, in general, you can't assume that SAFECode mainline is tracking LLVM mainline; if you do, and we're not, you may fork a release branch before all changes to the previous release branch have been folded back into mainline.

Thanks.

-- John T.

John,

Dear Pawel and All,

Pawel (and other release managers), can you ask the project maintainers
for each project whether they want a release branch before creating it
next time? Alternatively, can you leave branch creation up to the
project maintainers as in the past?

I think asking project maintainers for a go/no-go for branch creation
is a very good idea that should be incorporated into the LLVM release
process.

Pawel, you created a release_32 branch of SAFECode last night without my
consultation. That commit itself is fine since we just happen to be
tracking LLVM mainline right now; I'll just tell SAFECode developers to
move development to the release_32 branch. However, in general, you
can't assume that SAFECode mainline is tracking LLVM mainline; if you
do, and we're not, you may fork a release branch before all changes to
the previous release branch have been folded back into mainline.

John, it was not my intention to disrupt your work in any way.
However, branching for release is a bit of a grey area and I
have made a judgment call to branch SAFECode in sync with llvm.
This could have been a bit overzealous but I am used to
"release at any time" approach or agile project management.

Since LLVM/Clang has grown so much I think the release process
will require some adjustment to keep up with the sheer size of it.

From my experience it is hard to make progress without accepting

changes while at the same time making changes usually leads to
some unintentional breakages.

As for the SAFECode, perhaps I have made assumptions
on how you track mainline that are not valid.
It would be constructive to have your perspective on how
to manage release of a project dependent on the LLVM.

Thanks.

-- John T.

Pawel

John,

Dear Pawel and All,

Pawel (and other release managers), can you ask the project maintainers
for each project whether they want a release branch before creating it
next time? Alternatively, can you leave branch creation up to the
project maintainers as in the past?

  I think asking project maintainers for a go/no-go for branch creation
is a very good idea that should be incorporated into the LLVM release
process.

What I think needs to be done is to add details like this to the document describing the release management process. In the past, the release manager has just sort of known what to branch and what not to branch. I can see how that information fails to transmit between release managers.
:slight_smile:

In the past, I recall the release manager made branches of LLVM, Clang, and test-suite because all 3 of those projects followed LLVM mainline. Other projects simply followed whatever branching policy they wanted.

Pawel, you created a release_32 branch of SAFECode last night without my
consultation. That commit itself is fine since we just happen to be
tracking LLVM mainline right now; I'll just tell SAFECode developers to
move development to the release_32 branch. However, in general, you
can't assume that SAFECode mainline is tracking LLVM mainline; if you
do, and we're not, you may fork a release branch before all changes to
the previous release branch have been folded back into mainline.

  John, it was not my intention to disrupt your work in any way.
However, branching for release is a bit of a grey area and I
have made a judgment call to branch SAFECode in sync with llvm.
This could have been a bit overzealous but I am used to
"release at any time" approach or agile project management.

  Since LLVM/Clang has grown so much I think the release process
will require some adjustment to keep up with the sheer size of it.
From my experience it is hard to make progress without accepting
changes while at the same time making changes usually leads to
some unintentional breakages.

Okay. Just know that SAFECode is undergoing evaluation by one of our funding agencies. There have been and will be times in which we can't tolerate breakage. Fortunately, the next evaluation is about 12 months away, IIRC.

  As for the SAFECode, perhaps I have made assumptions
on how you track mainline that are not valid.
It would be constructive to have your perspective on how
to manage release of a project dependent on the LLVM.

I think there are two general ways to do it:

1) When someone makes a change to LLVM that breaks the C++ API, they make the corresponding changes to a set of projects that opt-in. I would certainly love that for SAFECode; one of the reasons why we sometimes opt not to track mainline is because the C++ API is simply too volatile.

If the community helped us track mainline, then having the release manager branch the opted-in projects makes a lot of sense.

2) If people aren't going to update projects consistently when C++ API changes are made, then the only thing that the release manager should branch is LLVM, Clang, test-suite, and any project that commits to tracking LLVM mainline. Every other project is on its own.

I would love it if option #1 were adopted, but I'm skeptical if the LLVM community as a whole is interested in fixing up sub-projects when the main projects make changes. However, if my skepticism is misplaced, please let me know.
:slight_smile:

So, assuming no changes in C++ API change policy, I'd prefer to make SAFECode branches myself. I'd rather focus on SAFECode development than chasing LLVM's tail, so I don't think we can guarantee that our mainline development is always going to track LLVM's.

-- John T.

Why not make a "release" branch for these sorts of things?

-Chris

Chris, can you clarify what you are suggesting? Your statement is sufficiently ambiguous that it could mean one of several things.

Just so that everyone knows, SAFECode branching and development works a little differently than LLVM. With LLVM, nearly all development happens on the trunk, and then branches are created for each release. Changes are selectively folded into the release branch until the release is blessed, and then (generally speaking) the release branch isn't touched again.

SAFECode works differently. With SAFECode, there is a release_xx branch for each release of LLVM that we support. For example, the SAFECode release_30 branch works with the release_30 branch of LLVM. The idea is to make it easy for people to grab updates to SAFECode that work with a particular LLVM version.

The real difference is that SAFECode usually sticks with a single release of LLVM for awhile, so while LLVM is developed on trunk, we often develop SAFECode on the branches. When we decide to start following mainline LLVM again (e.g., because we want to work with the upcoming next release of LLVM), we first merge the release branch changes back to SAFECode trunk, update SAFECode trunk to work with LLVM trunk, and then track LLVM trunk until the next release. When the next release of LLVM is made, we make a corresponding release branch and continue development there.

Note that the merges here are usually conflict free. When a SAFECode branch is made, development stops on SAFECode trunk until we're ready to start chasing LLVM mainline again.

With this process, having a person who isn't doing SAFECode development creating release branches is problematic. This person may create a release branch before changes from the previous release branch have been folded into trunk or create a release branch for a version of LLVM that we don't even support.

While SAFECode's branching behavior is different, I think it's logical, simple, and effective. However, since it's different, I'd rather handle the branching myself or, at the very least, have the release manager talk to me about what I need.

-- John T.

John, it was not my intention to disrupt your work in any way.
However, branching for release is a bit of a grey area and I
have made a judgment call to branch SAFECode in sync with llvm.
This could have been a bit overzealous but I am used to
"release at any time" approach or agile project management.

Since LLVM/Clang has grown so much I think the release process
will require some adjustment to keep up with the sheer size of it.
From my experience it is hard to make progress without accepting
changes while at the same time making changes usually leads to
some unintentional breakages.

Okay. Just know that SAFECode is undergoing evaluation by one of our funding agencies. There have been and will be times in which we can't tolerate breakage. Fortunately, the next evaluation is about 12 months away, IIRC.

Why not make a "release" branch for these sorts of things?

Chris, can you clarify what you are suggesting? Your statement is sufficiently ambiguous that it could mean one of several things.

I'm suggesting that you could change your development approach so that mainline safecode always follows mainline llvm, and that you do a majority of your work on branches.

Just so that everyone knows, SAFECode branching and development works a little differently than LLVM. With LLVM, nearly all development happens on the trunk, and then branches are created for each release. Changes are selectively folded into the release branch until the release is blessed, and then (generally speaking) the release branch isn't touched again.

SAFECode works differently. With SAFECode, there is a release_xx branch for each release of LLVM that we support. For example, the SAFECode release_30 branch works with the release_30 branch of LLVM. The idea is to make it easy for people to grab updates to SAFECode that work with a particular LLVM version.

The real difference is that SAFECode usually sticks with a single release of LLVM for awhile, so while LLVM is developed on trunk, we often develop SAFECode on the branches. When we decide to start following mainline LLVM again (e.g., because we want to work with the upcoming next release of LLVM), we first merge the release branch changes back to SAFECode trunk, update SAFECode trunk to work with LLVM trunk, and then track LLVM trunk until the next release. When the next release of LLVM is made, we make a corresponding release branch and continue development there.

Note that the merges here are usually conflict free. When a SAFECode branch is made, development stops on SAFECode trunk until we're ready to start chasing LLVM mainline again.

With this process, having a person who isn't doing SAFECode development creating release branches is problematic. This person may create a release branch before changes from the previous release branch have been folded into trunk or create a release branch for a version of LLVM that we don't even support.

While SAFECode's branching behavior is different, I think it's logical, simple, and effective. However, since it's different, I'd rather handle the branching myself or, at the very least, have the release manager talk to me about what I need.

You're describing the old LLDB development model (with the absence of "llvm.zip" :). It has a number of problems: 1) it discourages community contribution, 2) it makes it difficult to set up a buildbot, 3) it is different than everything else in the project.

I don't have a strong opinion - you can do what you want, just saying that there are other options.

-Chris