Changing behavior of lit.py's -v flag

Project Infrastructure LLVM Dev List Archives
#1

Hi,

I think lit.py’s current behavior is somewhat unintuitive in the presence of -v (and not
using -vv), so I’m proposing that it be changed.

Current behavior:
-v: Prints all the substituted lines, not clear which line failed :frowning:
-vv: Prints all the lines up to and including the line that failed :slight_smile:

Option 1:
-v: Prints only the failing line :slight_smile:
-vv: Same as today. :slight_smile:
-vvv (new): Prints all the substituted lines, highlighting* the line that failed :slight_smile:

Option 2 (less deviation from status quo):
-v: Adopts behavior of -vvv from Option 1. :slight_smile:

-vv: Same as today. :slight_smile:

See also: https://twitter.com/brentdax/status/1154955239620956160 (note: my opinion has
changed slightly since my comment there)

Varun

  • highlighting — could be bold and/or underlined or something else. Based on some simple
    testing on a Mac terminal, this doesn’t break copy-pasting even if you’re using the default
    formatting-aware copy-paste.
#2

Hi Varun,

I'm definitely in favour of making -v more useful like this.

Option 2 (less deviation from status quo):
-v: Adopts behavior of -vvv from Option 1. :slight_smile:
-vv: Same as today. :slight_smile:

I think I prefer this one. There's a significant minority of tests
where a the single failing line isn't enough to work out what's
actually happened, if the FileCheck is a separate RUN for example.
Under this scheme no-one even needs to know -vv exists (which is good,
because I've been working with LLVM for many years and didn't until
this thread).

Cheers.

Tim.

#3

Tim Northover via llvm-dev <llvm-dev@lists.llvm.org> writes:

Hi Varun,

I'm definitely in favour of making -v more useful like this.

Option 2 (less deviation from status quo):
-v: Adopts behavior of -vvv from Option 1. :slight_smile:
-vv: Same as today. :slight_smile:

I think I prefer this one. There's a significant minority of tests
where a the single failing line isn't enough to work out what's
actually happened, if the FileCheck is a separate RUN for example.
Under this scheme no-one even needs to know -vv exists (which is good,
because I've been working with LLVM for many years and didn't until
this thread).

+1.

                      -David

#4

Tim Northover via llvm-dev <llvm-dev@lists.llvm.org> writes:

Hi Varun,

I’m definitely in favour of making -v more useful like this.

Option 2 (less deviation from status quo):
-v: Adopts behavior of -vvv from Option 1. :slight_smile:
-vv: Same as today. :slight_smile:

I think I prefer this one. There’s a significant minority of tests
where a the single failing line isn’t enough to work out what’s
actually happened, if the FileCheck is a separate RUN for example.
Under this scheme no-one even needs to know -vv exists (which is good,
because I’ve been working with LLVM for many years and didn’t until
this thread).

+1.

+1 as well.

Perhaps just alias -v to -vv. The -v option is pretty much worthless.

Making -v behave like -vv would be huge win for anyone trying to debug a failure on a bot you don’t own, and wouldn’t require bot owners to change anything.

I’ll be happy to review the patch, or write it myself if that helps.

thanks…
don

#5

In the mean time, how about changing "ninja check" to run lit with -vv
instead of -v by default?
https://reviews.llvm.org/D72389

This gives me output like:

******************** TEST 'LLVM :: CodeGen/AMDGPU/fneg-combines.ll'
FAILED ********************
Script: