Can we make libcxx headers more modular for faster compilation?

Clang Frontend
#1

For example, including iostream also includes:

iostream:38:
ios:216:
__locale:15:
string:439:
algorithm:627:
memory:603:
tuple

That’s tons of baggage which slows down compilation significantly.
I guess that using std::cout does not really requires tuples.

Is it possible to break up the header files into smaller parts so that including a header file only include the other parts it really needs rather than the entire chain of includes as in above?

Yaron

#2

Have you tried to verify your assumption before this post?
It is easy to see that basic_string uses tuples and therefore, iostream
including stream will need.

Joerg

#3

Have you measured it? Especially for non-trivial translation units? I have
tried, and been unable to measure significant (> 5%) changes by minimizing
standard library includes.

Also, we're working on C++ modules which will largely eliminate *any* cost
associated with "bundling" together N different headers as part of a
library.

#4

Joerg, I am ashmed to admit I have re-read the code and still can not understand how basic_string uses tuple. How basic_string uses tuple?

Yaron

#5

Hi Chandler,

First let me say your GoingNative 2013 talk was very interesting. There is lots of potential in improving the efficiency and reliability of C++ programming, especially with the new feature in C++ 11. One has to de-program old habits to use these features.

To the subject, I did not meaure times exactly since the effect was very noticable for a trivial test program, three times as slow!

Here are the combinations I tested:

clang + libcxx iostream = about 3 seconds
clang + libstdc++ iostream = about 1 second
clang + libc stdio.h = about 1 second

gcc + libcxx iostream = about 3 seconds
gcc + libstdc++ iostream = about 1 second
gcc + libc stdio.h = about 1 second

clang.exe is from http://llvm.org/builds/, gcc is MingW-builds 4.8.1 32 bit.

For clang + libcxx the command was:

clang main.cpp -nostdinc -I…/libcxx -I…/mingw-4.8.1 -target i686-pc-mingw32 -Wno-deprecated-register -Wno-ignored-attributes libc++.dll.a libc++abi.dll.a -o main.exe

For clang + libstdc++ the command was:

clang main.cpp -nostdinc -I…/libstdc -I…/libstdc/i686-w64-mingw32 -I…/mingw-4.8.1 -target i686-pc-mingw32 -Wno-deprecated-register -Wno-ignored-attributes libstdc++.dll.a -o main.exe

For gcc + libstdc++ the command was:

gcc main.cpp -lstdc++

For gcc + libcxx the command was:

gcc main.cpp -std=c++11 -nostdinc -I…/libcxx -I…/libstdc/i686-w64-mingw32 -I…/mingw-4.8.1 -lstdc++ libc++.dll.a libc++abi.dll.a

These results were repeatable when running mutiple times.

The iostream code is:

#include

int main() {
std::cout<<“hello cout\n”;
return 0;
}

The stdio.h code is:

#include <stdio.h>

int main() {
printf(“hello printf\n”);
return 0;
}

The headers used with gcc are the defaults.
The headers used for clang are libcxx or libstdc++ for C++ and MingW 4.8.1 for the C library.

So when compiling small modules, libcxx slows down compilation considerably compared with including libstdc++.

Yaron

#6

basic_string -> __compressed_pair -> __libcpp_compressed_pair_imp
-> tuple.

Joerg

#7

Thanks, I see that now. It wasn’t trivial for me…

How can we understand why compiling the trivial program with libcxx is 3x slower vs with libstcd++ ?

Yaron

#8

support.h includes windows.h on win32 platform.
Does libstdc++ also include windows.h. I think this single include may be enough to explain the difference.

#9

support.h includes windows.h on win32 platform.
Does libstdc++ also include windows.h. I think this single include may be
enough to explain the difference.

It's a problem on Mac (and probably Linux) as well,
http://llvm.org/PR14587, so maybe it's not just windows.h.

Looking forward to modules!

#10

Although I just posted how windows.h is not required, in this copy it was still included.
So I repeated all measurments exactly using the ptime utility to get more accurate times for all combinations. The compiled code was:

#include

int main() {
std::cout<<“hello cout\n”;
return 0;
}

I repeated every run few times and got consistent results. Here are the results:

table.jpg

#11

Hello,

support.h includes windows.h on win32 platform.
Does libstdc++ also include windows.h. I think this single include may be enough to explain the difference.

I just looked into libc++, and it seems to include windows.h without first defining
WIN32_LEAN_AND_MEAN. From <http://support.microsoft.com/kb/166474>:

WIN32_LEAN_AND_MEAN: Windows Headers use this symbol to exclude rarely-used header files

Maybe that's a way to speed things up a bit, at least on Windows.

HTH,
Jonathan

#12

Jonathan,

It’s true that using WIN32_LEAN_AND_MEAN will speed compilation, but deleting the include <windows.h> will speed even more. It’s not required in libcxx. If the programmer needs it he can include it regardless of libcxx.

Yaron

#13

No. NO. NEVER DO THIS.

Defining that macro forces every single user of the Windows API to
always include the Windows.h header themselves before you do, lest they
will forever lose all ability to use very fundamental types in the
Windows headers.

That macro is for end user use only, and shall only be defined when all
participating users agree on its use.

"rarely-used" doesn't help the bloke that actually wanted to use HDROP
or niche libraries like the ATL or WTL in that TU.

If it's in your private libc++ library TUs, sure. But if it has _any_
chance of leaking out into other code? No. Just don't.

#14

OK. :slight_smile: We should avoid including windows.h from public headers anyway.