[RFC] [lit] Report line numbers for failed RUN commands

Project Infrastructure LLVM Dev List Archives
1

When some lit tests fail, I find it time-consuming to locate the source line of the failing RUN command. Running lit with -vv (or just -v in the case of the internal shell) makes it easier because it prints an expanded version of each RUN command as it executes, but it can still be challenging because lit doesn’t provide the RUN command’s source line number.

While I haven’t personally seen failures in the following tests, grep helped me discover them as extreme examples where finding the RUN source line should prove challenging:

  1. clang’s test/Driver/linux-ld.c has 892 total RUN lines

  2. clang’s test/Driver/arm-cortex-cpus.c has 424 RUN lines after concatenation for line continuations

To address this issue, I wrote the following patch to introduce RUN source line numbers into lit’s -vv output (or -v for the internal shell), but the patch needs a review:

https://reviews.llvm.org/D44598

I’ve been using this patch for a couple of weeks, and it’s a big improvement for me. If there’s some existing feature that people already use to handle this issue, please let me know so I can improve my usage of lit.

Thanks.

Joel

2

Seems useful, but the current output is quite verbose.
Wouldn’t just

RUN:1; ls

be sufficient in place of

'RUN: at line 1’; ls

?

I’m also curious about the implementation, I recall there was a discussion about forcing all tests to use internal shell?
In that case a trick with a no-op “:” command would not be necessary.

George

3

Hi George,

Seems useful, but the current output is quite verbose.
Wouldn’t just

RUN:1; ls

be sufficient in place of

'RUN: at line 1’; ls

?

My only argument against your shorter version is that its meaning might not
be obvious to someone seeing it for the first time. Nevertheless, it would
certainly suffice for me.

I’m also curious about the implementation, I recall there was a discussion
about forcing all tests to use internal shell?

Do you have a link?

In that case a trick with a no-op “:” command would not be necessary.

Agreed. Without having seen the discussion, my guess it that usage of the
external shell isn't going away immediately, and this patch seems
immediately useful. It's also fairly small, so it should be easy to
migrate to an implementation tailored to the internal shell if usage of the
external shell goes away.

Thanks.

Joel