Problem with regression tests using stderr

I was attempting to write a test that involves grepping though the
stderr produced by opt -analyze, but found that my test was passing
even before I fixed the bug I was writing the test for!

I found that this one-line sure-fail test:

; RUN: echo hi |& false

passes. I also tried 2>&1, because I've seen some tests do that,
though that doesn't appear to work in this context either. It didn't
seem to create a file nameed &1 though, as the LLVM test-writing
documentation says.

After some investigation, I believe the problem is due to the regexp
on this line in test/lib/llvm.exp:

      } elseif {[regexp {RUN: *([^&]+)(&&)?} $line match oneline suffix]} {

I don't understand what this is trying to do. What is intended by
the special handling of && here?

I found only one test that uses && on a RUN line,
test/CodeGen/X86/2007-07-03-GR64ToVR64.ll, and it's not clear to
me what it's doing there.

Dan

I think that it's a hold-over to how things used to be done. IIRC, you had to have the && at the end of the RUN line if you had another RUN line that needed to be executed. That's no longer the case, of course.

If you want to catch stderr, there is one test doing that:

Analysis/ScalarEvolution/2007-07-15-NegativeStride.ll:

; RUN: llvm-as < %s | opt -analyze -scalar-evolution 2>&1 | grep "Loop bb: 100 iterations"

-bw

I think that it's a hold-over to how things used to be done. IIRC,
you had to have the && at the end of the RUN line if you had another
RUN line that needed to be executed. That's no longer the case, of
course.

Thanks. I'll remove the && from the one test that still has it then.

Then we can look at removing the associated code from llvm.exp and fixing
the bug. This will take a while though, at it exposes a large number of
failures that are currently hidden.

If you want to catch stderr, there is one test doing that:

Analysis/ScalarEvolution/2007-07-15-NegativeStride.ll:

; RUN: llvm-as < %s | opt -analyze -scalar-evolution 2>&1 | grep
"Loop bb: 100 iterations"

2>&1 doesn't evoke the desired behavior; the RUN line is unfortunately
interpreted by TCL rather than a regular shell. The llvm.exp bug has
allowed this and a variety of other common bourne-shell-isms to slip by
unnoticed.

Dan

Can you please file a bug for this (i havent seen one yet).

Thanks,
Tanya

Ok, it's PR1826.

Dan

Ok, it's PR1826.

Thanks Dan. I've commented on the bug. We can continue the conversation there.

-Tanya