Separate preprocess and compile: some performance numbers

Hi,

I have implemented the separate preprocess and compile setup in build2.
For Clang it is using -frewrite-includes (thanks to everyone's suggestions
in the earlier thread). I've also done some benchmarking:

https://build2.org/article/preprocess-compile-performance.xhtml

TL;DR for Clang:

Cost of a separate preprocessor run is less than 5% of a non-optimized
C++ build.

Interestingly, for GCC (with -fdirectives-only which does more than
-frewrite-includes) a separate preprocessor run is actually faster
(probably because of the time-localization of filesystem access).

Boris

Did you just build once for each number? If so, disk cache effects will make your measurements unreliable. I recommend doing 3 consecutive builds for each number and then recording the smallest of these 3 numbers.

Nico Weber <thakis@chromium.org> writes:

Did you just build once for each number? If so, disk cache effects will
make your measurements unreliable. I recommend doing 3 consecutive builds
for each number and then recording the smallest of these 3 numbers.

I did 4 consecutive builds discarding the first one as well as re-doing
any significant outliers. Then averaging the last three.

Interestingly, in all the cases except the straight compilation (the first
column), all the measurements were pretty close without any outliers. In
the straight compilation case I had a significant (10% or more) outlier
for each compiler. If I didn't discard them, the other numbers would have
looked even better.

Boris

I didn’t see your page mention what operating system you gathered your numbers on, but I suspect that also has an influence.

– Bob

Bob Haarman writes:

I didn't see your page mention what operating system you gathered your
numbers on,

It was Linux. More specifically, Ubuntu 16.04, ext4 partition. I've updated
the doc, thanks.

but I suspect that also has an influence.

I think it is true to the extent that the OS doesn't do anything stupid.
Given that, it's all the hardware (CPU, RAM, SSD) and the toolchain.

Boris