Need Python help in TestRunner.py

In running the test Driver/ast.c on Windows:

2> File “C:\Tools\llvm\utils\lit\TestRunner.py”, line 79, in executeShCmd
2> raise NotImplementedError,“Unsupported redirect: %r” % (r,)
2>NotImplementedError: Unsupported redirect: ((’>>’,), ‘C:\Tools\llvm\tools\clang\test\Driver\Output\ast.c.tmp’)

It appears that ‘>>’ is not one of the conditions in the if/else series, and I don’t know Python well enough to fix it.

My guess of adding:

elif r[0] == (’>>’,):
redirects[1] = [r[1], ‘a’, None]

resulted in the redirected output being put at the front of the output file.

So if this is a quick fix for someone, I’d appreciate the help.

-John

Hey John,

I'll try to take a look at this soon. File a bug or email me if I seem
to forget. :slight_smile:

- Daniel

Daniel,

Here’s a wierd one regarding running the test/Frontend/ast-main.c test on Windows. The “diff” utility seems to be crashing when run from the test scripts in . Running the same command line from the command shell doesn’t crash:

diff C:\Tools\llvm\tools\clang\test\Frontend\Output\ast-main.c.tmp1.ll C:\Tools\llvm\tools\clang\test\Frontend\Output\ast-main.c.tmp2.ll

By default, on my system, I’m using the diff from the gnuwin32 utils. I also substituted the msys diff, but it crashes also, but only when run under the test scripts.

Are you seeing this too?

Any thoughts?

-John

In running the test Driver/ast.c on Windows:

2> File "C:\Tools\llvm\utils\lit\TestRunner.py", line 79, in executeShCmd
2> raise NotImplementedError,"Unsupported redirect: %r" % (r,)
2>NotImplementedError: Unsupported redirect: (('>>',),
'C:\\Tools\\llvm\\tools\\clang\\test\\Driver\\Output\\ast.c.tmp')
It appears that '>>' is not one of the conditions in the if/else series, and
I don't know Python well enough to fix it.

My guess of adding:

        elif r\[0\] == \('>>',\):
            redirects\[1\] = \[r\[1\], 'a', None\]

Looks like this should work, to me. I checked it on Darwin using
internal-execution of the test scripts and checked it in. Let me know
if it seems not to work on Windows.

- Daniel

Hi John,

Daniel,

Here's a wierd one regarding running the test/Frontend/ast-main.c test on
Windows. The "diff" utility seems to be crashing when run from the test
scripts in . Running the same command line from the command shell doesn't
crash:

diff C:\Tools\llvm\tools\clang\test\Frontend\Output\ast-main.c.tmp1.ll
C:\Tools\llvm\tools\clang\test\Frontend\Output\ast-main.c.tmp2.ll

By default, on my system, I'm using the diff from the gnuwin32 utils. I
also substituted the msys diff, but it crashes also, but only when run under
the test scripts.
Are you seeing this too?

Nope, I've never seen diff crashing.

Any thoughts?

No clue. :slight_smile:

One major difference about stuff run under the test script is it tries
to limit the things that are in the environment. Maybe that makes your
diff unhappy somehow?

- Daniel

Daniel,

I’m afraid the result of this change is that instead of appending to the file, it overwrites the beginning of the file. This is the result in ast.c.tmp (on Windows):

END
put, “C:\Tools\llvm\tools\clang\test\Driver\ast.c”, c
1: preprocessor, {0}, cpp-output
2: compiler, {1}, ast

Compared to Linux:

0: input, “/home/john/llvm/tools/clang/test/Driver/Output/ast.c.tmp.ast”, ast
1: compiler, {0}, assembler
2: assembler, {1}, object
END

-John

Daniel,

I'm afraid the result of this change is that instead of appending to the
file, it overwrites the beginning of the file. This is the result in
ast.c.tmp (on Windows):

END
put, "C:\Tools\llvm\tools\clang\test\Driver\ast.c", c
1: preprocessor, {0}, cpp-output
2: compiler, {1}, ast
Compared to Linux:

0: input, "/home/john/llvm/tools/clang/test/Driver/Output/ast.c.tmp.ast",
ast
1: compiler, {0}, assembler
2: assembler, {1}, object
END

Can you file a bug on this -- I'll try to take a look soon. I didn't
expect it to be platform dependent and only tested on Unix.

- Daniel

Daniel,

Here's a wierd one regarding running the test/Frontend/ast-main.c test on
Windows. The "diff" utility seems to be crashing when run from the test
scripts in . Running the same command line from the command shell doesn't
crash:

diff C:\Tools\llvm\tools\clang\test\Frontend\Output\ast-main.c.tmp1.ll
C:\Tools\llvm\tools\clang\test\Frontend\Output\ast-main.c.tmp2.ll

By default, on my system, I'm using the diff from the gnuwin32 utils. I
also substituted the msys diff, but it crashes also, but only when run under
the test scripts.
Are you seeing this too?

I see this now when using GnuWin32's diff. I presume its a bug in
diff, but r86436 works around it on my system.

- Daniel

Daniel,

I'm afraid the result of this change is that instead of appending to the
file, it overwrites the beginning of the file. This is the result in
ast.c.tmp (on Windows):

Looks like a bug in Win32 or the subprocess module. r86437 adds a
workaround that works for me.

- Daniel

Have you considered using the Python difflib[1] module? It might be more reliable than the external dependancy…

[1] <http://docs.python.org/library/difflib.html>

Daniel,

I'm afraid the result of this change is that instead of appending to the
file, it overwrites the beginning of the file. This is the result in
ast.c.tmp (on Windows):

Looks like a bug in Win32 or the subprocess module. r86437 adds a
workaround that works for me.

Have you considered using the Python difflib[1] module? It might be more
reliable than the external dependancy…

We don't execute individual commands in the RUN lines. While useful
for other reasons, if we wanted to do this we would need to find a
nice way so that things are easy to reproduce on the command line. I
don't have a motivating reason to do this work (yet).

- Daniel

Daniel,

What I discovered last week (and noted in the Bugzilla 5318) is that I have the diff problem only if I run the tests from Visual Studio. If I run them from the command line, I don’t see the problem. Wierd.

I updated this morning to 86549, but still have the problem.

The redirection problem is gone, though, meaning we’re just down to these two failing tests (plus the CIndex one, which I have work pending approval).

Thanks!

-John

Daniel,

What I discovered last week (and noted in the Bugzilla 5318) is that I have
the diff problem only if I run the tests from Visual Studio. If I run them
from the command line, I don't see the problem. Wierd.

For me the tests failed when run from lit, which passes a very
restricted subset of the environment. Including PATHEXT fixed it for
me, but if that isn't working for you I suspect some kind of buffer
overflow or other problem in the gnuwin32 'diff'.

I updated this morning to 86549, but still have the problem.

Did you also update LLVM?

- Daniel