Local svn strategy for future LLVM release updates

Project Infrastructure LLVM Dev List Archives
1

We are starting a local LLVM backend project (using svn) and are looking for a mechanism to periodically update to later LLVM releases.
Certainly, this seems like it would be a relatively common use case and I expected a google search would reveal several options. However, I didn’t really find anything. Perhaps I’m not looking for the right keywords, if so please help me out and point me to any relevant docs.
Could others share their experiences with maintaining local LLVM projects?

Without any further guidance, the path we will follow comprises of the following flow:

Create a local repo with trunk/llvm_60
Copy trunk/llvm_60 to trunk/myproj60

Make project changes on trunk/myproj60

When LLVM 7.0 is released, create trunk/llvm_70 and copy this to trunk/myproj70

Diff trunk/llvm_60 with trunk/myproj60, identify and fix any conflicts, and apply resulting patch to trunk/myproj70

Continue project development on trunk/myproj70

I’m pretty sure this will work, what I don’t like about it is that every 6 months we are rebasing our tree. Assuming a successful project, over the years we will have many rebased trees and finding a particular change/check-in will require knowledge of which base a change was made. I don’t know of any good alternatives.

thanks
Doug

For completeness, I’ve included specific svn commands for starting with a LLVM 6.0 release and later upgrading to LLVM 7.0 below.

svnadmin create repos
svn mkdir file://`pwd`/repos/llvm<file:///%60pwd%60/repos/llvm>
svn mkdir file://`pwd`/repos/llvm/trunk<file:///%60pwd%60/repos/llvm/trunk>
svn mkdir file://`pwd`/repos/llvm/branches<file:///%60pwd%60/repos/llvm/branches>

svn export http://llvm.org/svn/llvm-project/llvm/branches/release_60llvm_60
svn import llvm_60 file://`pwd`/repos/llvm/trunk/llvm_60<file:///%60pwd%60/repos/llvm/trunk/llvm_60>

svn copy trunk/llvm_60 file://`pwd`/repos/llvm/trunk/myproj60<file:///%60pwd%60/repos/llvm/trunk/myproj60>

//Make project changes to trunk/myproj60.

//When next llvm release is available (say llvm 7.0)

svn diff file://`pwd`/repos/llvm/trunk/llvm_60<file:///%60pwd%60/repos/llvm/trunk/llvm_60> file://`pwd`/repos/llvm/trunk/myproj<file:///%60pwd%60/repos/llvm/trunk/myproj> > /tmp/diffs60.patch
// Identify and fixup any conflicts, repeat above until no conflicts

// Repeat same steps as above to make a trunk/llvm_70 and trunk/myproj70
svn co file://`pwd`/repos/llvm/trunk/llvm_70<file:///%60pwd%60/repos/llvm/trunk/llvm_70> wc70

cd wc70
svn patch /tmp/diffs60.patch
svn commit

// Start using trunk/myproj70 as project base

2

Hi Doug,
This is a common problem but little gets published about it, at
least little that I found helpful when trying to deal with it.
You can look here for links to "Living Downstream Without Drowning"
http://llvm.org/devmtg/2015-10/
and notes from a follow-up BoF "Surviving Downstream"

I have some additional remarks inline below.

From: llvm-dev [mailto:llvm-dev-bounces@lists.llvm.org] On Behalf Of Doug
Vanesko (dvanesko) via llvm-dev
Sent: Saturday, March 17, 2018 8:13 AM
To: llvm-dev@lists.llvm.org
Subject: [llvm-dev] Local svn strategy for future LLVM release updates

We are starting a local LLVM backend project (using svn) and are looking
for a mechanism to periodically update to later LLVM releases.
Certainly, this seems like it would be a relatively common use case and I
expected a google search would reveal several options. However, I didn’t
really find anything. Perhaps I’m not looking for the right keywords, if
so please help me out and point me to any relevant docs.
Could others share their experiences with maintaining local LLVM projects?

Without any further guidance, the path we will follow comprises of the
following flow:

Create a local repo with trunk/llvm_60
Copy trunk/llvm_60 to trunk/myproj60

Make project changes on trunk/myproj60

When LLVM 7.0 is released, create trunk/llvm_70 and copy this to
trunk/myproj70

Diff trunk/llvm_60 with trunk/myproj60, identify and fix any conflicts,
and apply resulting patch to trunk/myproj70

Continue project development on trunk/myproj70

I’m pretty sure this will work, what I don’t like about it is that every 6
months we are rebasing our tree. Assuming a successful project, over the
years we will have many rebased trees and finding a particular
change/check-in will require knowledge of which base a change was made. I
don’t know of any good alternatives.

thanks
Doug

You've correctly identified the problem with this approach, which is that
you'll lose all your individual patch comments and history. This can be
tolerable if you have a small number of changes, and I've heard some
people talk about how they actually maintain their work as a series of
patch files that they just keep reapplying. But it sounds like you have
a much bigger project than that.

Another tactic would be to use 'svn merge' to move all your stuff to the
new base. This will preserve patch history to some extent but you will
have the same piles of conflicts to resolve. The first time I did this
it took me 3 months to resolve all the differences; admittedly I was new
to LLVM at the time but it's still not what you'd call efficient. :slight_smile:

I would, nevertheless, suggest that you try it at least once and see how
it goes. If you're adding a new target, LLVM tries to make that not too
onerous, and if you don't have lots of patches scattered all through the
codebase it might not be too painful to take the merge hit twice a year.

If it is too painful, then probably your best bet is to go with a more
continuous-integration plan, which is what we have evolved to at Sony
and the tactic we mainly describe in the Downstream talk. This does
require some investment in automation, but if you have the resources it
is definitely the easiest on the team as a whole.

Let us know what you think, and how things are going. You might also
find the git hooks to be a more tractable way to handle merges, but
that's for your organization to decide.

Good luck!
--paulr