Changing the way dotest reports passes and failures

Currently when you run dotest.py, at the end it prints a summary of the test run. It prints the number of failures, the total number of tests, and then for each failing test, it shows you what file failed.

I’m interested in changing this so that for the purposes of reporting, it treats every function in every file as a separate test.

Imagine the case where you’ve got 5 python files in your test suite, and each one has 12 individual tests. In each one, a single test fails, and the other 4 pass. Currently, dotest will make it look like you’ve got a 0% pass rate, when in truth you’ve got a 93% pass rate.

So I would propose changing the output of dotest from this:

FAILED (errors=6, skipped=3)
Ran 377 tests.

Failing Tests (146)
FAIL: LLDB (suite) :: Test-rdar-10449092.py (Windows zturner-win81 8 6.2.9200 AMD64 Intel64 Family 6 Model 45 Stepping 7, GenuineIntel)

To something like this:

FAILED (errors=6, skipped=3)
Ran 1,527 tests.

Failing Tests (356)
FAIL: LLDB (suite) :: Test-rdar-10449092.py (2 tests failed) (Windows zturner-win81 8 6.2.9200 AMD64 Intel64 Family 6 Model 45 Stepping 7, GenuineIntel)

Any thoughts on this?

No problem here.