Why does llvm-lit use bash?

Beginners
1

When I run ninja check-llvm on Ubuntu Linux, especially towards the end of the run when it is running the fastest tests, perf top --sort comm,dso shows that bash is consuming over 10% of my total CPU time. This seems pretty wasteful.

pstree -lp shows that llvm-lit is running tools like llc via bash:

-ninja(4100463)---sh(4113978)---python3.10(4113979)-+-python3.10(4114115)
                                                    |-python3.10(4114116)---bash(4149685)---clang(4150056)
                                                    |-python3.10(4114117)---bash(4150032)
                                                    |-python3.10(4114118)---bash(4149937)
                                                    |-python3.10(4114119)---bash(4149848)-+-FileCheck(4150040)
                                                    |                                     `-llc(4150039)
                                                    |-python3.10(4114120)---bash(4149820)-+-FileCheck(4150065)
                                                    |                                     |-clang(4150063)
                                                    |                                     `-opt(4150064)
                                                    |-python3.10(4114121)---bash(4149933)
                                                    |-python3.10(4114122)---bash(4150069)-+-FileCheck(4150072)
                                                    |                                     |-clang(4150070)
                                                    |                                     `-opt(4150071)
                                                    |-python3.10(4114123)
                                                    |-python3.10(4114124)---bash(4149916)
                                                    |-python3.10(4114125)---bash(4150052)-+-FileCheck(4150059)
                                                    |                                     |-clang(4150057)
                                                    |                                     `-opt(4150058)
                                                    |-python3.10(4114126)---bash(4150041)
                                                    |-python3.10(4114127)---bash(4149793)-+-FileCheck(4150046)
                                                    |                                     `-clang(4150044)
                                                    |-python3.10(4114128)---bash(4149734)---clang(4150066)
                                                    |-python3.10(4114129)
                                                    |-python3.10(4114130)
                                                    |-python3.10(4114131)
                                                    |-python3.10(4114132)---bash(4149907)-+-FileCheck(4150051)
                                                    |                                     |-clang(4150049)
                                                    |                                     `-opt(4150050)
                                                    |-python3.10(4114133)---bash(4149766)
                                                    |-python3.10(4114134)---bash(4149890)
                                                    |-python3.10(4114135)---bash(4149773)
                                                    |-python3.10(4114136)---bash(4149755)
                                                    |-python3.10(4114137)
                                                    |-python3.10(4114138)---bash(4149816)-+-FileCheck(4150055)
                                                    |                                     |-clang(4150053)
                                                    |                                     `-opt(4150054)
                                                    |-{python3.10}(4114139)
                                                    |-{python3.10}(4114140)
                                                    `-{python3.10}(4114141)

Questions:

  1. Why does it do this? Is it using bash to execute the entire RUN: line? I thought llvm-lit used some internal code to do that, not a real shell.
  2. Is there any chance of using a faster shell like /bin/sh (which is typically dash on Debian-like systems) instead, or not using a shell at all?

Thanks!

2

I haven’t benchmarked it, but LIT checks for the LIT_USE_INTERNAL_SHELL environment variable, and will switch to its own implementation if it’s present.

3

Test suites can also be configured to use lit’s internal shell. More info is here: RFC: Improving lit's debug output - #3 by MaskRay

4

In my Release build, setting LIT_USE_INTERNAL_SHELL=1 speeds up check-llvm from about 98 (+/- 1) seconds to 91 (+/- 1) seconds. This seems worth having.

I guess the down side is that it skips tests which have REQUIRES: shell, but that is only 16 out of 59895 tests. Is there a better way to handle them? Could lit use a shell only for the tests that require it?

5

lit’s internal shell doesn’t yet support all the features we use in tests such as nested shell expression, but I hope this is something we can address and make the internal shell the default, eventually even deprecating the use of external shell. We have interns on our team that will be working on this project over the summer.

6

That’s good to hear, thanks.

Just brainstorming a bit:

  • Could lit switch to using any POSIX shell (e.g. /bin/sh) as its external shell, instead of bash specifically?
  • Could tests that require advanced shell features use an explicit sh -c "..." in their RUN: lines, instead of REQUIRES: shell?
7

The question is what to do on platforms that don’t provide a POSIX shell. For example, we’re porting LLVM to Fuchsia and Fuchsia is not POSIX and does not support any POSIX shells, but we still want to be able to run LLVM tests. This issue is not unique to Fuchsia though, we have the same exact problem on other non-POSIX platforms like Windows. Our goal is to improve lit so the external shell won’t be needed at all.

1 Like