Hi,
Has there been discussion about having svn mirror git instead of vice versa? I’ve been happy with the git+cmake+ninja combo for a while now, but the online docs suggests this configuration is second-class to svn+autotools+make. Is the community transitioning to any particular configuration?
Thanks,
Greg
This has been discussed many times. Might want to take a look in the mailing list archives. 
cmake and autotools are on roughly equal footing at this point with chandler and I handling each of them respectively and fairly quickly. git vs. svn is a different story though.
-eric
I think svn works better than git as an authoritative upstream (monotonic version numbers being the most obvious advantage).
A few months ago, people were talking about removing the autotools build system in favour of cmake, although I don't think that a consensus was reached. I've also now switched to using cmake+ninja, and it's a lot better as a development platform (incremental builds for minor changes take 1-2 seconds instead 12-15 as recursive make discovers it doesn't have anything to do, which makes a huge difference when you do 10-20 edit-build-test cycles in quick succession). I think there are still a few things that the cmake system can't do, but I'd definitely be in favour of making it the authoritative build system. It's also vastly easier to build out-of-tree components when you are using cmake. That said, Ninja is still pretty experimental. The latest CMake to ship on OS X, for example, doesn't support it out of the box, and clock skew (which is easy to get in a VM) can confuse Ninja in a way that doesn't give helpful errors.
David
David Chisnall <David.Chisnall@cl.cam.ac.uk> writes:
clock skew (which is easy to get in a VM) can confuse Ninja in a way
that doesn't give helpful errors.
This is a significant issue. I would not want to transition to
cmake+ninja before this is fixed.
-David
clock skew (which is easy to get in a VM) can confuse Ninja in a way
that doesn’t give helpful errors.This is a significant issue. I would not want to transition to
cmake+ninja before this is fixed.
Do you mean in the same way that clock skew would affect Make? Or just that Ninja doesn’t provide that nice “clock skew detected” message that Make does? Or are you looking for a build system that uses file hashes instead of last-modified times?
I think svn works better than git as an authoritative upstream
Would you mind expanding on this? What problem specifically is being solved? Linus and Guido both use DVCS’s and the authoritative upstream is whatever URL the BDFL says it is.
Thanks,
Greg
Do you mean in the same way that clock skew would affect Make? Or just that Ninja doesn't provide that nice "clock skew detected" message that Make does? Or are you looking for a build system that uses file hashes instead of last-modified times?
Ninja complains about circular dependencies when you have your clock set to the wrong time. There is no easy way of fixing this other than to delete the entire build tree and start again.
> I think svn works better than git as an authoritative upstream
Would you mind expanding on this? What problem specifically is being solved? Linus and Guido both use DVCS's and the authoritative upstream is whatever URL the BDFL says it is.
Monotonic version numbers are the biggest advantage. It is easy to see that r1234432 contains the bug fix introduced in r1234430. It is very hard to see if version 23bef194ac contains the bug fix added in 23bef19412. This makes interaction with bugzilla and so on much easier. If someone says 'please test r1245145 - should be fixed' you can easily check whether you are running r1245145 or newer.
David
git branch --contains 23bef19412
This will tell you which of your branches have that commit and
highlight the current branch you are on.
Git also has monotonically increasing identifiers for each commit. The
time stamp. Which I find much more informative than a revision number
split between multiple repositories.
As for actually switching to git. I see no benefit to justify the cost
of switching unless we actually take advantage of git's features. And
I've yet to see anyone propose this. So for now, git-svn works for me.
- Michael Spencer
Hi Michael,
As for actually switching to git. I see no benefit to justify the cost
of switching unless we actually take advantage of git’s features. And
I’ve yet to see anyone propose this.
Then I’ll be the first.
The benefit is that the review process would require no file copies or email attachments, shorter email conversations, no copying code during reviews to simulate inline comments, and no need to “git rebase” to push to the top of svn. I wouldn’t be surprised if the difference was so significant that folks would stop using the llvm-commits list altogether. To see what changed, you’d check the github mirror, and to contribute you could post a link to llvmdev (not too noisy).
For example, say github’s llvm-mirror was a contributor’s fork. The review process might look like this:
Contributor:
Please review my patch: https://github.com/llvm-mirror/llvm/commit/4823be3be1d87632fbd51ce8e51a58ee5e44b115
Maintainer:
Adds inline comments with online tool. Then when patch is looking good:
$ git fetch https://github.com/llvm-mirror/llvm.git
$ git cherry-pick 4823be3be1d87632fbd51ce8e51a58ee5e44b115
$ git push
Thanks,
Greg
Hi Michael,
As for actually switching to git. I see no benefit to justify the cost
of switching unless we actually take advantage of git's features. And
I've yet to see anyone propose this.Then I'll be the first.
I'd suggest you/others read the previous thread discussing this issue.
It's a bit tricky to link to a whole thread in the llvm-dev mail
archive, but here's one part of it:
http://lists.cs.uiuc.edu/pipermail/llvmdev/2011-July/041738.html
The benefit is that the review process would require no file copies or email
attachments, shorter email conversations, no copying code during reviews to
simulate inline comments, and no need to "git rebase" to push to the top of
svn. I wouldn't be surprised if the difference was so significant that
folks would stop using the llvm-commits list altogether. To see what
changed, you'd check the github mirror, and to contribute you could post a
link to llvmdev (not too noisy).
Essentially that's precisely what we want to avoid. The intention is
to keep discussion & review in the shared public view & keep the
codebase in a (mostly) singular state. The transition to git would
have to be justified on its merits while still preserving that
workflow, not while working against it.
(I'm not sure how folks would stop using llvm-commits all together, if
we still have as much shared development there would still be a push
email for every shared commit, an email for every review request, and
if the reduction in email was because review happened off-list, that
would be a loss, not a benefit)
Greg Fitzgerald <garious@gmail.com> writes:
I've yet to see anyone propose this.
Oh, many people have proposed using git's features.
Then I'll be the first.
The benefit is that the review process would require no file copies or
email attachments, shorter email conversations, no copying code during
reviews to simulate inline comments, and no need to "git rebase" to
push to the top of svn.
I'm not sure this is actually true. Yes, you can ask people to pull
your changes and review them, but that misses the value of other people
chiming in when they see your code on the e-mail list.
And for practicality, "git rebase" will almost certainly be necessary
before pushing to upstream master.
I wouldn't be surprised if the difference was so significant that
folks would stop using the llvm-commits list altogether. To see what
changed, you'd check the github mirror, and to contribute you could
post a link to llvmdev (not too noisy).
I think a possibly more productive way to do reviews would be through
something like Gerrit.
I believe there are far more important git advantages for llvm, many
of which I have listed before. These include but are not limited to:
- A natural system for small, incremental commits. The svn model
generally discourages this kind of development while the git model
encourages it.
- Tools for responding to reviews. These include git rebase -i and git
add -i.
- Easier third-party merging. Merging via git merge/rebase is MUCH
simpler than merging a tarball from svn. I know the llvm leaders
don't seem to care about this but it is a real issue for many users.
- Ease of fork/merge. git makes it easy to experiment with long-lived
development branches. svn forces all development into trunk which
limits experimentation and innovation.
-David
Neither of these are true. There is nothing stopping people from working in branches in svn and merging from branches is easy. The advantage with git is that it's very easy to have your fork somewhere where no one else sees it and your changes get lost and forgotten.
Your point about small incremental commits is also untrue. svn encourages small incremental commits and, more importantly, encourages small incremental pushes upstream. This is something we want to encourage: we don't want to make it easier for people to have private forks than it is to push them upstream.
David
I'd suggest you/others read the previous thread discussing this issue.
It's a bit tricky to link to a whole thread in the llvm-dev mail
archive, but here's one part of it:
http://lists.cs.uiuc.edu/pipermail/llvmdev/2011-July/041738.html
Maybe I should add this to the FAQ? Does this look good?:
Why don't you move from svn to git?
For example, say github's llvm-mirror was a contributor's fork. The review
process might look like this:Contributor:
Please review my patch:
https://github.com/llvm-mirror/llvm/commit/4823be3be1d87632fbd51ce8e51a58ee5e44b115Maintainer:
Adds inline comments with online tool. Then when patch is looking good:
$ git fetch GitHub - llvm-mirror/llvm: Project moved to: https://github.com/llvm/llvm-project
$ git cherry-pick 4823be3be1d87632fbd51ce8e51a58ee5e44b115
$ git push
Most development is not "some contributor's fork". What you suggest
works great for the occasional drive-by contributors, but most
development is not from drive-by's.
-- Sean Silva
cmake and autotools are on roughly equal footing at this point with chandler
and I handling each of them respectively and fairly quickly. git vs. svn is
a different story though.
Would you be willing to put this commitment to autotools in
CODE_OWNERS.TXT? Sometimes I have needed to make changes to Makefiles
and didn't know who was the right person to CC for it, so having an
owner for that would be good.
-- Sean Silva
Oh, sure. Been doing it anyhow.
-eric
I'd suggest you/others read the previous thread discussing this issue.
It's a bit tricky to link to a whole thread in the llvm-dev mail
archive, but here's one part of it:
http://lists.cs.uiuc.edu/pipermail/llvmdev/2011-July/041738.htmlMaybe I should add this to the FAQ? Does this look good?:
Why don't you move from svn to git?
Please read the following discussions about the issue, and think
carefully before bringing it up on the list:* http://lists.cs.uiuc.edu/pipermail/llvmdev/2011-July/041738.html
* ... others ...
Possible. I'm not sure it comes up quite often enough to draw
attention to it in the FAQ (drawing attention to it might increase the
number of conversations where someone thinks they've got some idea
that we didn't cover last time), but maybe (& I'm not sure how
many/which people read the FAQ anyway).
Side note (perhaps for another email thread): am I missing something,
or is the mailing list archive really that hard to link to the root of
a thread & navigate the /whole/ thread over all time? Every time I go
to the archive I find the fragment of a thread that occurred in a
given month & I can't seem to find where to easily view/navigate the
whole thread across multiple months.
Sean Silva <silvas@purdue.edu> writes:
For example, say github's llvm-mirror was a contributor's fork. The review
process might look like this:Contributor:
Please review my patch:
https://github.com/llvm-mirror/llvm/commit/4823be3be1d87632fbd51ce8e51a58ee5e44b115Maintainer:
Adds inline comments with online tool. Then when patch is looking good:
$ git fetch GitHub - llvm-mirror/llvm: Project moved to: https://github.com/llvm/llvm-project
$ git cherry-pick 4823be3be1d87632fbd51ce8e51a58ee5e44b115
$ git pushMost development is not "some contributor's fork".
Well, in the git model every developer has a fork.
What you suggest works great for the occasional drive-by
contributors, but most development is not from drive-by's.
Not sure what you mean by "drive-by" but in git this is really
irrelevant, I think.
-David
Sean Silva <silvas@purdue.edu> writes:
I'd suggest you/others read the previous thread discussing this issue.
It's a bit tricky to link to a whole thread in the llvm-dev mail
archive, but here's one part of it:
http://lists.cs.uiuc.edu/pipermail/llvmdev/2011-July/041738.htmlMaybe I should add this to the FAQ? Does this look good?:
Why don't you move from svn to git?
Please read the following discussions about the issue, and think
carefully before bringing it up on the list:* http://lists.cs.uiuc.edu/pipermail/llvmdev/2011-July/041738.html
* ... others ...
That particular message is a poor link to the thread. Best to start at
the top.
-David
David Chisnall <David.Chisnall@cl.cam.ac.uk> writes:
- Easier third-party merging. Merging via git merge/rebase is MUCH
simpler than merging a tarball from svn. I know the llvm leaders
don't seem to care about this but it is a real issue for many users.- Ease of fork/merge. git makes it easy to experiment with long-lived
development branches. svn forces all development into trunk which
limits experimentation and innovation.Neither of these are true. There is nothing stopping people from
working in branches in svn
Yes there is. I have asked about it many times and have always been
refused.
and merging from branches is easy.
That is absolutely not true for any significant merge operation.
The advantage with git is that it's very easy to have your fork
somewhere where no one else sees it and your changes get lost and
forgotten.
Actually that's true today with svn. We have a ton of changes here we'd
like to submit but the process is really painful.
Your point about small incremental commits is also untrue. svn
encourages small incremental commits and, more importantly, encourages
small incremental pushes upstream.
An svn commit is an expensive operation. A git commit is a cheap
operation. That's why I'm referring to. Development inertia under svn
tends to steer the developer toward making larger changes before
committing. With git committing becomes an almost unnoticed part of the
process. At least that's been my experience.
This is something we want to encourage: we don't want to make it
easier for people to have private forks than it is to push them
upstream.
There is value in those private forks. As I said, having places to
experiment is important.
-David
Most development is not "some contributor's fork".
Well, in the git model every developer has a fork.
What you suggest works great for the occasional drive-by
contributors, but most development is not from drive-by's.Not sure what you mean by "drive-by" but in git this is really
irrelevant, I think.
Sorry for the lack of clarity. I was trying to contrast with the
status quo. By "drive-by" I *roughly* mean "people without commit
access" (i.e. that aren't consistently delivering patches). In his
scenario there is "contributor" and "maintainer". What I was trying to
get across is that most development (in the status quo) is actually
done by people that are closer to the "maintainer" role, so making it
easy for "contributor" is not a gigantic priority; although, (and I
speak from my own experience from before I had commit access), when
you're not a committer, it really *seems* like the "contributor" role
is the only one that matters.
Also, "contributor" is not going to be sending in huge changes, so the
overhead of git-fetch'ing from github or wherever is probably going to
be greater than git-apply of an attached patch.
-- Sean Silva