'git llvm push' not working for me on Windows

Project Infrastructure LLVM Dev List Archives
1

I am consistently getting:

`git apply -p2 -` returned 1
error: include/llvm/IR/DIBuilder.h: No such file or directory
[[ etc ]]
Patch doesn't apply; maybe you should try `git pull -r` first?

My usual response to a problem from git-llvm (which is most often
an anti-virus issue) is to blow away .git\llvm-upstream-svn but
that doesn't help this time.

I see James Knight did a "performance improvement" on Friday,
perhaps that doesn't work so well on Windows? The main
performance cost is one-time, and as long as my anti-virus isn't
trashing things behind my back the performance is just fine;
so it's not clear this is really a necessary improvement.
If it's interfering with developing the new git repo, then I'd
suggest making it optional or platform-dependent.

As a workaround I fetched the previous version of git-llvm
and put it earlier in my PATH, that worked.

--paulr

2

It does sound like I must’ve broken something, but I believe zturner used it on windows successfully. Perhaps you could help me debug the issue? Possibly running it with --verbose would show something useful.

3

Usually every time I’ve seen that error message, it’s been related to line ending normalization. But James is right, I did use it successfully this morning as well as yesterday.

4

Hm. Just now it worked for me for r347271. If it happens again I’ll try verbose mode and attach a log.

Thanks,

–paulr

5

FTR the commit where it failed for me had both llvm and clang changes. I wonder if that contributed.

–paulr

6

As a test case, try committing a change to clang/test/SemaCXX/sourceranges.cpp?

I believe most editors force Windows-style line endings for that file (they show up in my vim as “^M”). I recently tried to commit a change to that file using “git llvm push” on macOS, but hit what I think is the same issue. At least, the error message was the same.

vedant (sent from my iPhone)

7

I’ve verified that none of the files I tried to check in had Windows-style line endings. It’s something else.

–paulr

8

I’m facing this now on Linux.

Any idea how to fix it?

9

What I did was retrieve an older copy of the git-llvm script, and put it on my PATH somewhere with another name (e.g., git-llvm-old). Then use git llvm-old push.

Question for you: Are you trying to push a patch that has changes in multiple projects? (Multiple top-level directories in the mono repo clone, e.g., both llvm and clang) Or is it in just one project?

–paulr

10

Can you please run “git llvm --verbose push” and send me the output?

BTW, I’ve just committed a fix for the handling of binary patch data when you’re running it under python3.X. I don’t believe that was a regression from my recent changes, but if you’re using python3, you could see if that fixes it for you.

11

Hi,

The patch only changes one file in clang. Here’s the patch: https://reviews.llvm.org/D54974

Attached the log. It seems to complain about the git apply -p2.

Maybe it’s something related to arc?

out.log (11.1 KB)

12

OK, I’ve managed to do it:

I was trying to push it from a build/ directory, maybe that’s why the git apply was failing.

Pushing the commit from the root of the repo worked.

13

And now it’s failing for me also… complaining about “no such file or directory” on a file that plainly exists.

Verbose log attached.

–paulr

git-llvm-push-fail.txt (48.5 KB)

14

Poking around in the .git\llvm-upstream-svn tree, I find that llvm\trunk\test\DebugInfo\Generic is empty, as are all the other subdirectories of test\DebugInfo that I tried. I have other files in the checkin that are in leaf directories but those files all exist.

I hacked git-llvm to add a –verbose option:

git apply --verbose -p2 - returned 1

Checking patch include/llvm/IR/DebugInfoFlags.def…

Checking patch include/llvm/IR/DebugInfoMetadata.h…

Checking patch lib/AsmParser/LLLexer.cpp…

Checking patch lib/AsmParser/LLParser.cpp…

Checking patch lib/AsmParser/LLToken.h…

Checking patch lib/Bitcode/Reader/MetadataLoader.cpp…

Checking patch lib/Bitcode/Writer/BitcodeWriter.cpp…

Checking patch lib/IR/AsmWriter.cpp…

Checking patch lib/IR/DebugInfoMetadata.cpp…

Checking patch test/Assembler/disubprogram.ll…

Checking patch test/Assembler/invalid-disubprogram-uniqued-definition.ll…

Checking patch test/Bindings/llvm-c/debug_info.ll…

Checking patch test/Bitcode/DISubprogram-distinct-definitions.ll…

Checking patch test/Bitcode/DISubprogram-v4.ll…

Checking patch test/Bitcode/DISubprogram-v4.ll.bc…

Checking patch test/DebugInfo/Generic/invalid.ll…

error: test/DebugInfo/Generic/invalid.ll: No such file or directory

Checking patch test/DebugInfo/debugify.ll…

Checking patch test/Linker/replaced-function-matches-first-subprogram.ll…

Checking patch test/Transforms/GCOVProfiling/three-element-mdnode.ll…

Patch doesn’t apply: maybe you should try git pull -r first?

15

Aha! From your output I figured out what I screwed up. Sorry about the trouble…

Illustration of the issue:
svn checkout --depth=empty https://llvm.org/svn/llvm-project

svn update --depth=immediates --parents llvm/trunk/test/DebugInfo

svn update --depth=immediates --parents llvm/trunk/test/DebugInfo/Generic

After these commands, I expected Generic/ to be populated, but it is not, because the “–depth=immediates” creates the directory, but then it marks it as depth=empty. And then, later, calling svn update --depth=immediates doesn’t override that. This is apparently not a bug in subversion, that’s is how it is “supposed” to work. It’s just super confusing. If I use --depth=files instead of --depth=immediates, that fixes this issue, since in that case, it won’t create the subdirs and mark them as depth=empty.

Also, the --parents flag causes the same issue for parent directories. Ugh. After the following, I’d expected llvm/trunk/ to get populated, but it doesn’t get populated, because the first ‘svn update’ created it with sticky depth=empty:

svn checkout --depth=empty https://llvm.org/svn/llvm-project

svn update --depth=files --parents llvm/trunk/test/DebugInfo

svn update --depth=files --parents llvm/trunk/

Subversion has a --set-depth= argument which “fixes” this, but unfortunately, that argument also recursively deletes any subdirs which have already been populated. (I just want to add things, not delete already-downloaded data.)

In summary: subversion’s sparse-checkout command-line UI is really confusing and annoying! Ugh!

I’ve pushed a commit (r347883) to hopefully fix this issue. This time for sure.

You will probably need to remove the .git/llvm-upstream-svn directory, one last time, for the fix to work.

16

Excellent, thanks!

–paulr

17

I just hit this failure as well. I’ve uploaded the log to https://reviews.llvm.org/P8126. The error message is:

git apply -p2 - returned 1
error: patch failed: lib/Basic/Version.cpp:14
error: lib/Basic/Version.cpp: patch does not apply
Patch doesn’t apply: maybe you should try git pull -r first?

18

The usual workaround is to do rm –rf .git/llvm-upstream-svn and try again.

You could also patch llvm/utils/git-svn/git-llvm to add the --verbose option to the git apply command, which would provide better diagnostic output.

HTH,

–paulr

19

I already figured it out. Turned out some sources like clang/lib/Basic/Version.cpp contain Subversion keywords (in this particular case it’s $URL$). Subversion would automatically substitute these in the internal checkout that’s used by git-llvm-push, but Git wouldn’t, so when trying to apply the patch generated by Git you’ll get and error because while Git patch has $URL$, the Subversion checkout would have something like $URL: https://llvm.org/svn/llvm-project/cfe/trunk/$. The workaround I came up with is to do the keyword substitution in the generated patch (https://github.com/llvm/llvm-project/blob/master/llvm/utils/git-svn/git-llvm#L321).

20

Hm, there’s only a few affected files (files that both have svn:keywords set, and contain a $keyword$).

We’ll need to remove or replace all of these anyhow, since the expansion doesn’t actually function with a git checkout.

clang/lib/Basic/Version.cpp: StringRef SVNRepository(“$URL$”);

clang/tools/scan-build/man/scan-build.1:." $Id$
clang/www/cxx_dr_status.html:

Last updated: $Date$


clang/www/cxx_status.html:

Last updated: $Date$


llvm/utils/vim/syntax/llvm.vim:" Version: $Revision$

llvm/utils/vim/syntax/tablegen.vim:" Version: $Revision$
llvm/utils/vim/vimrc:" $Revision$