[RFC] Running libc tests unconditionally

Runtimes C
1

Hello all,

There have been a few requests in the past to run libc untitests unconditionally (like how it is done in the rest of LLVM) as opposed to running the tests only when the deps change. If everyone agrees that this is a desirable change, I would like to prepare patches making this change. I would like to keep the granularity of running the tests intact post this change also. That is, one should be able to run an individual test by running the target corresponding to that test. Overall, I propose to make the following changes in order:

  1. Move all test related pieces, including the UnitTest and IntegrationTest frameworks over to the test directory. This will make it easy to exclude all test related pieces when LLVM_INCLUDE_TESTS is OFF.
  2. For libc level targets like check-libc:
    1. check-libc will run all tests which, depending on the build mode, will include unit tests, integration tests and api-test. The tests will be run unconditionally.
    2. A new target to run unit tests will be introduced: libc-unit-tests. Invoking ninja libc-unit-tests will run the unit tests unconditionally.
    3. The current target libc-integration-tests will run all integration tests unconditionally.
    4. The current target libc-api-test will run the API test unconditionally.
  3. The individual test targets, for example libc.src.string.strlen_test will run the corresponding tests unconditionally.

All of the above targets will bring the build up-to-date before running the corresponding test(s). Feel free to share your concerns if you have any.

I think the first item can be taken up as a change not directly related to this RFC. However, it involves a widespread mechanical change which touches almost all test sources making arcanist and phabricator unhappy. So, I have included it here instead of creating a separate RFC for it. If everyone agrees, I plan to submit that change without review (to avoid the arcanist and phabricator problems).

Thanks,
Siva Chandra

2

SGTM.

Regarding Bazel, I suppose you want to keep the current behavior?
It is easy to make the test run every time anyways with --cache_test_results=no

3

+1

Will this be done using the standard add_unittest like the LLVM unittests?

4

@gchatelet - Yes, we do not change anything wrt Bazel.
@jhuber6 - No, we cannot LLVM’s add_unittest as it uses gtest. My plan is to make a small modification to add_libc_unittest.

5

I see, it would be ideal if running ninja check-libc worked as closely as possible to ninja check-llvm-unit or similar.

6

The main items proposed by this RFC have landed. A follow up task could be to print a message with statistics of the test run at the end of the run. It can be taken up separately.