Impact of default PIE on llvm testing times

I have just upgraded a machine from Ubuntu 20.04 to 22.04 and found that the time taken to run ninja check-llvm has increased by around 1.75x:

before:
Testing Time: 126.30s

after:
Testing Time: 223.98s

The reason is that I use clang as the host compiler, and Ubuntu 22.04 defaults to clang-14 with PIE enabled by default, which seems to increase the startup time of tools like llc rather a lot.

I can make it fast again by running cmake . -DCMAKE_EXE_LINKER_FLAGS=-no-pie. Is there a better way? Should we even do this by default, since it makes such a big difference to the testing time?

For reference my full cmake invocation for these timings was:
CC=clang CXX=clang++ cmake -G Ninja ~/git/llvm-project/llvm -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_LLD=ON

You are not the first one: LLVM 15 default PIE issue - #5 by Blis

I saw that discussion but I did not understand the point of it.

My message was intended more as a warning to LLVM developers: default PIE will slow down your testing cycle so you might want to disable it.

I assume this is because testing LLVM involves lots of very short runs of very large executables like llc, so it is dominated by the time taken to run fixups on the executable.

In the enable diff, there was problem noticed with increasing compile time: ⚙ D120305 [Driver] Default CLANG_DEFAULT_PIE_ON_LINUX to ON

For example, would a patch like this be acceptable?
https://reviews.llvm.org/D140880
It improves my life as an LLVM developer, but I guess it may not be appropriate for other (non-developer) users of LLVM.

I’ve also thought that we should change how tests are invoked. Instead of launching a process for every run line and every file, lit should recycle the same process (e.g. have some kind of llc/opt server consume multiple different invocations)

I’ve put up a stack of patches culminating in ⚙ D142219 [MC] Store operand info immediately after the TargetInsts table. NFC. to remove pointers from the <Target>Insts tables so they do not need any relocs. On my machine this reduces the time taken for a Release build check-llvm from 204 seconds to 131 seconds with PIE enabled. With PIE disabled it takes 113 seconds.

1 Like