Note to buildbot/testbot runners

Hi all,

TL;DR: if you call dosep.py directly, you’ll need to modify your flow to call dotest.py.

Details:

dotest.py now runs in parallel test runner mode by default

Starting with lldb svn revision 246794, if you run buildbots or testbots and you directly called dosep.py as a build step, you’ll need to replace that with dotest.py. You will continue to get the parallel test execution speed improvements. You no longer need to wrap dotest.py parameters in a -o options block.

dotest.py now takes an optional --no-multiprocess argument

If you really want to run exactly like the old dotest runner, you can still get that behavior by specifying --no-multiprocess on the dotest.py command line.

dosep.py’s unnamed argument (test-subdir) has moved to --test-subdir

dosep.py’s arguments have moved into dotest.py under the Parallel test execution argument group. The only difference is that the unnamed final argument available in dosep.py was moved to a named argument in dotest.py as dotest.py already used unnamed arguments for something and the collision needed to be resolved. The new argument that dotest.py takes, and is only used when the test is running in parallel execution mode, is the --test-subdir argument, handled the same way dosep.py’s unnamed argument used to work. (test subdirectory relative to the lldb/test root, that limits all parallel test running to the specified subdirectory tree).

While dosep.py is still around as an implementation detail, calling it directly will now result in a stderr message and will return with a non-zero exit code. This is for two reasons: (1) we are no longer supporting dosep.py as a callable test driver, and (2) the methods for calling it have changed to support dotest.py’s implementation of running the parallel test runner and having args parsed already and passed to it. Thus, any buildbot/testbot that directly called dosep.py is going to need to change to call over to dotest.py.

Thanks, all!

-Todd

A few notes to dotest.py users:

  • If you want to limit a test run to a test subdirectory tree, you can use the new --test-subdir flag. It covers what used to be the supported unnamed argument in dosep.py. It is specified relative to the lldb/test dir.

  • If you want to use the unnamed arguments to specify single directory levels in which to execute tests, you’ll either want to convert to using --test-subdir, or add --no-multiprocess, until I fix a bug I found just recently (and filed a bugzilla bug on it).

  • dosep.py’s --threads used to also have a ‘-t’ abbreviation. Since all of dosep’s (3-ish) arguments moved over to dotest.py and now live in dotest.py’s argument namespace, I had to avoid collisions with existing dotest.py single-letter argument abbreviations. ‘-t’ already belonged to “Turn on tracing of lldb command and other detailed test executions”, so -t no longer specifies thread count. Use the longer --threads or the environment variable for it.

  • dosep.py’s --output-on-success had a ‘-s’ abbreviation. This collides with dotest.py’s already-existing session-dir-setting command. So, you must now specify the --output-on-success long form of the argument if you want the parallel test runner to do that.

  • The supported nameless arg in dosep.py has been moved to the dotest.py --test-subdir arg.

Other than that, I expect none of the behavior to differ aside from the bug I just filed on dosep.py’s inconvenient failure to handle ^C correctly. (This existed forever in dosep.py but it is more annoying for it to be broken for the default usage of dotest.py, so I’ll use this opportunity to fix it).

Thanks!

-Todd