2 experimental projects

1) Parallel "make check".

>> Modelled after the successful clang "make test -jX" experiment, I am
>> now bold enough to sink my teeth into the LLVM test suite.
>> I have a prototype implementation, along the same lines, ready.
>
>Can you please explain this in detail.
>
>Thanks,
>Tanya

Sure.

This is how "make test -j4" in clang works now:

The makefile locates all relevant test files inside
the specified test directories, that is files ending
with ".cpp", ".c" and ".m", synthesizes the targets
by appending ".testresults".

These targets are then built in parallel by a recursive
invocation of $(MAKE).

A build rule of form

Output/%.testresults: %

invokes the TestRunner.sh script.

Depending on the verbosity asked for, more or less
summary and error information is shown to the user.

My measurements on a 4-processor machine have
shown test time reduction of 60% when the tests were
invoked as "make test -j8".

In case of the llvm tests the situation is somewhat more
complicated, since dejagnu is driving the process, and
dejagnu hinders us in a direct $(MAKE) invocation, so
it falls back to -j1. My approach is thus using dejagnu
to build up makefiles in test/.../Output/... and when
these makefiles are executed by $(MAKE) we will
get the same output as before. Each of the makefiles
shall create the .testresults files just like clang tests do.

This is the plan.

Cheers,

  Gabor

In case of the llvm tests the situation is somewhat more
complicated, since dejagnu is driving the process, and
dejagnu hinders us in a direct $(MAKE) invocation, so
it falls back to -j1. My approach is thus using dejagnu
to build up makefiles in test/.../Output/... and when
these makefiles are executed by $(MAKE) we will
get the same output as before. Each of the makefiles
shall create the .testresults files just like clang tests do.

To be clear... do you plan to use the exisiting mechanism to execute the "RUN" lines? This is much preferred over the TestRunner script and there are many reasons why we transition away from using TestRunner.

Also, will you preserve the ability to run a subset of tests via TESTSUITE=X?

Will dejagnu still produce one log file or will there be multiples?

How will this impact the nightly testers?

While its cool to run them in parallel, I don't want to lose the functionality that we have now with dejagnu/tcl.

Thanks,
Tanya

My testing shows that -j4 is the sweet spot on my MacPro with 4 CPUs for cfe.

   17.334 --> 7.681

Nice, thanks. Anyway, := evaluates the variable once, = merely substitutes it, causing it to be run multiple times. With the below patch we go from 4 evaluations to 2.

test.patch (564 Bytes)