should we stop using llvm-as/llvm-dis in tests?

A recent commit added the ability to opt and llc to read .ll files directly. Should we go through and update the existing tests?

   llvm-as < %s | opt ... | llvm-dis

would become:

   opt %s ... -print-module

and

   llvm-as < %s | llc

would become:

   llc < %s

The pro of this is that it would remove the bitcode write and read from the tests, making them faster. The con of this is that it would remove the bitcode write and read from the tests, making them less well tested.

Should we not make this change at all? Should we create some sort of 'exhaustive bitcode tests' first, and require that new language features are added there? Or should we just do it and create "llvm-as < %s | llvm-dis" tests under test/Bitcode as problems are found?

Also, has anyone else already started doing this?

Nick

A recent commit added the ability to opt and llc to read .ll files
directly. Should we go through and update the existing tests?

Yes, I think that Dan is planning to do this.

-Chris

... and it is definitely worth doing. On my system a 'time' of make
check reports that about 50% of the real time running make check is
spent in the OS. This probably also limits the efficacy of attempts to
parallelize the test suite, if someone was crazy enough to do that.

Using opt -S for tests that end up going to llvm-dis would also be nice.

- Daniel

Daniel Dunbar <daniel@zuster.org> writes:

A recent commit added the ability to opt and llc to read .ll files
directly. Should we go through and update the existing tests?

Yes, I think that Dan is planning to do this.

... and it is definitely worth doing. On my system a 'time' of make
check reports that about 50% of the real time running make check is
spent in the OS. This probably also limits the efficacy of attempts to
parallelize the test suite, if someone was crazy enough to do that.

Why would be crazy to run the test suite on parallel? Because DejaGNU
limitations?

My compiler's test suite is driven by a poor-man's version of DejaGNU
(also written on Tcl, but not requiring Expect) that supports parallel
runs (with a -j command-line parameter, á la make) and works on Linux
and Windows.

Since long time ago I'm thinking on expanding its functionality for
accepting DejaGNU test cases and add it to the cmake build, but I'm
afraid of corner cases. You know, the 99% of the complexity on the 1% of
the instances.

Would you consider parallel runs and Windows support enough motivation
for rewriting those (hypothetical) corner cases as equivalent but
simpler ones?

It was a joke, I'm actively working on this. Stay tuned. :slight_smile:

- Daniel