PCH: separation from source header

Clang Frontend
#1

Hi,

I am trying to adapt Clang' PCH to my build system from example here,
http://clang.llvm.org/docs/UsersManual.html#precompiledheaders . Here is
the test case:

$ ls -R
.:
build src

./build:
test.h.pch

./src:
test.c test.h

$ cat test.c
#include <test.h>

$ clang -x c-header -Ibuild -Isrc -o build/test.h.pch src/test.h
$ clang -c -Ibuild -Isrc -include test.h -o build/test.o src/test.c

It seems that clang fails to use build/test.h.pch in such situations (I
checked that for some non-trivial C++ sources - no performance gain).

So, here is the questions:
1) How can I make sure that PCH header is being used, not test.h? For
gcc, I use special "fake" build/test.h with contents like so:
#error PCH' using is failed, that shouldn't be done :frowning:

then, if PCH cannot be used than compilation will break. But this is not
working for clang.

2) Is it possible to use Clang' PCH in the test case above? If not then
what is a solution?

3) What is the reason against using .h.pch automatically like gcc does it?

Thanks in advance,
Ilya

#2

Hi,

I am trying to adapt Clang' PCH to my build system from example here,
http://clang.llvm.org/docs/UsersManual.html#precompiledheaders . Here is
the test case:

$ ls -R
.:
build src

./build:
test.h.pch

./src:
test.c test.h

$ cat test.c
#include <test.h>

$ clang -x c-header -Ibuild -Isrc -o build/test.h.pch src/test.h
$ clang -c -Ibuild -Isrc -include test.h -o build/test.o src/test.c

It seems that clang fails to use build/test.h.pch in such situations (I
checked that for some non-trivial C++ sources - no performance gain).

This happens because the driver doesn't look into the include directories (-Ibuild) for a PCH file, at your example it will only look into the current directory.
Daniel, should the driver look into the include dirs ?

So, here is the questions:
1) How can I make sure that PCH header is being used, not test.h? For
gcc, I use special "fake" build/test.h with contents like so:
#error PCH' using is failed, that shouldn't be done :frowning:

then, if PCH cannot be used than compilation will break. But this is not
working for clang.

It should, can you post a test case where this doesn't work ?

2) Is it possible to use Clang' PCH in the test case above? If not then
what is a solution?

$ clang -c -Isrc -include build/test.h -o build/test.o src/test.c

-Argiris

#3

Argyrios Kyrtzidis пишет:

So, here is the questions:
1) How can I make sure that PCH header is being used, not test.h? For
gcc, I use special "fake" build/test.h with contents like so:
#error PCH' using is failed, that shouldn't be done :frowning:

then, if PCH cannot be used than compilation will break. But this is not
working for clang.

It should, can you post a test case where this doesn't work ?

Well, I mean that that technique always fails now (fake header always
takes priority). Ok, here http://clang.llvm.org/docs/PCHInternals.html I
found out about -print-stats option for Clang compiler frontend, clang
-cc1 . But Clang doesn't produce any statistics about PCH use as the doc
states:

clang -cc1 -triple i386-pc-linux-gnu -S -disable-free
-disable-llvm-verifier -main-file-name test.cpp -mrelocation-model
static -mdisable-fp-elim -mconstructor-aliases -target-cpu pentium4
-target-linker-version 2.20 -v -resource-dir
/opt/llvm-28dbg/lib/clang/2.8 -include-pch build/test.h.pch -I src
-ferror-limit 19 -fmessage-length 165 -fexceptions -fgnu-runtime
-fdiagnostics-show-option -fcolor-diagnostics -o /tmp/cc-acnTys.s -x c++
-print-stats src/test.cpp 2&>1 | grep PCH

P.S. I just want to say that I don't like that "Clang falls back to
directly processing the content of test.h" *silently* if it cannot use
.h.pch file. Some way to turn that behaviour into an error/warning would
be sufficient.

2) Is it possible to use Clang' PCH in the test case above? If not then
what is a solution?

$ clang -c -Isrc -include build/test.h -o build/test.o src/test.c

-Argiris

Ok, now build/test.h.pch is used somehow. But there plenty of errors if
I use PCH for C++, e.g.:

$ cat src/test.h
#include <sstream>
$ cat src/test.cpp
#include <test.h>

int main()
{
    std::stringstream strm;
    return 0;
}

$ clang++ -x c++-header -Isrc -o build/test.h.pch src/test.h
$ clang++ -c -Isrc -o build/test.o src/test.cpp <---- no errors
$ clang++ -c -Isrc -include build/test.h -o build/test.o src/test.cpp
In file included from
/home/ilya/opt/programming/atom-project/Temp/pch/src/test.h:1:

In file included from /usr/include/c++/4.4/sstream:38:

In file included from /usr/include/c++/4.4/istream:39:

In file included from /usr/include/c++/4.4/ios:42:

In file included from /usr/include/c++/4.4/bits/ios_base.h:42:

In file included from /usr/include/c++/4.4/bits/locale_classes.h:41:
In file included from /usr/include/c++/4.4/string:52:
/usr/include/c++/4.4/bits/basic_string.h:417:7: error: constructor for
'std::basic_string<char>' must explicitly initialize the member
'_M_dataplus' which does not
      have a default constructor
      basic_string();
      ^
In file included from
/home/ilya/opt/programming/atom-project/Temp/pch/src/test.h:1:
/usr/include/c++/4.4/sstream:91:7: note: in instantiation of member
function 'std::basic_string<char, std::char_traits<char>,
std::allocator<char> >::basic_string'
      requested here
      basic_stringbuf(ios_base::openmode __mode = ios_base::in |
ios_base::out)
      ^
/usr/include/c++/4.4/sstream:509:7: note: in instantiation of member
function 'std::basic_stringbuf<char, std::char_traits<char>,
std::allocator<char>
      >::basic_stringbuf' requested here
      basic_stringstream(ios_base::openmode __m = ios_base::out |
ios_base::in)
      ^
src/test.cpp:5:23: note: in instantiation of member function
'std::basic_stringstream<char, std::char_traits<char>,
std::allocator<char> >::basic_stringstream'
      requested here
    std::stringstream strm;
                      ^
In file included from
/home/ilya/opt/programming/atom-project/Temp/pch/src/test.h:1:
In file included from /usr/include/c++/4.4/sstream:38:
In file included from /usr/include/c++/4.4/istream:39:
In file included from /usr/include/c++/4.4/ios:42:
In file included from /usr/include/c++/4.4/bits/ios_base.h:42:
In file included from /usr/include/c++/4.4/bits/locale_classes.h:41:
In file included from /usr/include/c++/4.4/string:52:
/usr/include/c++/4.4/bits/basic_string.h:268:28: note: member is
declared here
      mutable _Alloc_hider _M_dataplus;
                                ^
/usr/include/c++/4.4/bits/basic_string.h:251:14: note:
'std::basic_string<char, std::char_traits<char>, std::allocator<char>

::_Alloc_hider' declared here

      struct _Alloc_hider : _Alloc
             ^
1 error generated.

############### end of console output

Has Clang PCH support only for C, not for C++? Should I file in a bug
about that?

Regards,
Ilya

#4

Argyrios Kyrtzidis пишет:

So, here is the questions:
1) How can I make sure that PCH header is being used, not test.h? For
gcc, I use special "fake" build/test.h with contents like so:
#error PCH' using is failed, that shouldn't be done :frowning:

then, if PCH cannot be used than compilation will break. But this is not
working for clang.

It should, can you post a test case where this doesn't work ?

Well, I mean that that technique always fails now (fake header always
takes priority). Ok, here http://clang.llvm.org/docs/PCHInternals.html I
found out about -print-stats option for Clang compiler frontend, clang
-cc1 . But Clang doesn't produce any statistics about PCH use as the doc
states:

clang -cc1 -triple i386-pc-linux-gnu -S -disable-free
-disable-llvm-verifier -main-file-name test.cpp -mrelocation-model
static -mdisable-fp-elim -mconstructor-aliases -target-cpu pentium4
-target-linker-version 2.20 -v -resource-dir
/opt/llvm-28dbg/lib/clang/2.8 -include-pch build/test.h.pch -I src
-ferror-limit 19 -fmessage-length 165 -fexceptions -fgnu-runtime
-fdiagnostics-show-option -fcolor-diagnostics -o /tmp/cc-acnTys.s -x c++
-print-stats src/test.cpp 2&>1 | grep PCH

It's actually under "AST File Statistics:" now. We should either revert it back to 'PCH' or update the doc.

P.S. I just want to say that I don't like that "Clang falls back to
directly processing the content of test.h" *silently* if it cannot use
.h.pch file. Some way to turn that behaviour into an error/warning would
be sufficient.

It doesn't "fall back silently", it didn't find the PCH file in the first place so "-include test.h" was treated as a normal [#include "test.h"].
If it finds a PCH file, there will be an error if it cannot use it.

If you'd like to specify the path of a PCH file and get an error if it doesn't exist, you may find useful this "low level" way (although I wouldn't recommend it since it's clang-specific):

$ clang -c -Isrc -Xclang -include-pch -Xclang build/t.h.pch -o build/test.o src/test.c

If the file after the second -Xclang doesn't exist you will get an error.

2) Is it possible to use Clang' PCH in the test case above? If not then
what is a solution?

$ clang -c -Isrc -include build/test.h -o build/test.o src/test.c

-Argiris

Ok, now build/test.h.pch is used somehow. But there plenty of errors if
I use PCH for C++, e.g.:

$ cat src/test.h
#include <sstream>
$ cat src/test.cpp
#include <test.h>

int main()
{
   std::stringstream strm;
   return 0;
}

$ clang++ -x c++-header -Isrc -o build/test.h.pch src/test.h
$ clang++ -c -Isrc -o build/test.o src/test.cpp <---- no errors
$ clang++ -c -Isrc -include build/test.h -o build/test.o src/test.cpp

[snip errors]

############### end of console output

Has Clang PCH support only for C, not for C++? Should I file in a bug
about that?

Could you test with the latest clang from trunk and if the problem persists file a bug with a preprocessed file ?

-Argiris

#5

Argyrios Kyrtzidis пишет:

Ok, now build/test.h.pch is used somehow. But there plenty of errors if
I use PCH for C++, e.g.:

$ cat src/test.h
#include <sstream>
$ cat src/test.cpp
#include <test.h>

int main()
{
   std::stringstream strm;
   return 0;
}

$ clang++ -x c++-header -Isrc -o build/test.h.pch src/test.h
$ clang++ -c -Isrc -o build/test.o src/test.cpp <---- no errors
$ clang++ -c -Isrc -include build/test.h -o build/test.o src/test.cpp

[snip errors]

############### end of console output

Has Clang PCH support only for C, not for C++? Should I file in a bug
about that?

Could you test with the latest clang from trunk and if the problem persists file a bug with a preprocessed file ?

-Argiris

I am with clang 2.8, clang++ -v
clang version 2.8 (branches/release_28 114073)
Target: i386-pc-linux-gnu
Thread model: posix

Is that enough or still I need trunk clang?

Regards,
Ilya

#6

Argyrios Kyrtzidis пишет:

Ok, now build/test.h.pch is used somehow. But there plenty of errors if
I use PCH for C++, e.g.:

$ cat src/test.h
#include <sstream>
$ cat src/test.cpp
#include <test.h>

int main()
{
  std::stringstream strm;
  return 0;
}

$ clang++ -x c++-header -Isrc -o build/test.h.pch src/test.h
$ clang++ -c -Isrc -o build/test.o src/test.cpp <---- no errors
$ clang++ -c -Isrc -include build/test.h -o build/test.o src/test.cpp

[snip errors]

############### end of console output

Has Clang PCH support only for C, not for C++? Should I file in a bug
about that?

Could you test with the latest clang from trunk and if the problem persists file a bug with a preprocessed file ?

-Argiris

I am with clang 2.8, clang++ -v
clang version 2.8 (branches/release_28 114073)
Target: i386-pc-linux-gnu
Thread model: posix

Is that enough or still I need trunk clang?

Should be enough, please file a bug with a preprocessed file and I'll take a look at it.

-Argiris

#7

Argyrios Kyrtzidis пишет:

I am with clang 2.8, clang++ -v
clang version 2.8 (branches/release_28 114073)
Target: i386-pc-linux-gnu
Thread model: posix

Is that enough or still I need trunk clang?

Should be enough, please file a bug with a preprocessed file and I'll

take a look at it.

Ok, http://llvm.org/bugs/show_bug.cgi?id=8187 . I attach the
preprocessed one but I think it is of no use,- no errors if I compile it:

$ clang++ -E -Isrc -include build/test.h -o build/test.cpp src/test.cpp
$ clang++ -c -o build/test.o build/test.cpp

As for the bug, it can be reproduced on *any decent* C++ header (not
only #include <sstream>): at first I get the error on #define
<boost/shared_ptr.hpp> .

Regards,
Ilya

#8

Argyrios Kyrtzidis пишет:

2) Is it possible to use Clang' PCH in the test case above? If not then
what is a solution?

$ clang -c -Isrc -include build/test.h -o build/test.o src/test.c

-Argiris

3) What is the reason against using .h.pch automatically like gcc

does it?

Ok, with Clang 2.9 I succeeded in building my project with PCH, thanks.
I have 2 things to say.

1. I have to say that "-include" scheme is not so flexible as using PCH
automatically, really. Here is my use case: let's say I have a project
(a bunch of sources), and I want to use one PCH-file for it (and so on,
one PCH-file per project). So I set that "-include" option for the
project and now it is used for every source file in the project, without
an exception,- that is no so good! In several cases I do want to avoid it:
- my projects have C++ and C sources together; I can't use C++ PCH
  header for C sources (so the hack is done - "-include" goes to CXX
  options, not CPP ones).
- in some C++ sources I still need to set #defines at first lines (e.g.
BOOST_AUTO_TEST_MAIN for Boost.Test library); now it is impossible.

That said, I need to create additional projects (without PCH) just for
such exceptional sources to make things to work.

2. I have got some benchmarks (compiler' speed in debug mode) gcc vs
clang for my sources (http://gitorious.org/bombono-dvd). I use Boost *a
little*, and it makes a compiler be somewhat slow. Here are compilation
times (-g -O0) for src/mgui/editor/select.cpp (39kb, not so big!):

a) without PCH:
clang++ 2.9: 5.35 seconds
g++ 4.2: 5.05 seconds

b) with PCH:
clang++ 2.9: 3.47 seconds
g++ 4.2: 2.88 seconds

So I am sure clang has some room for optimizing like
http://clang.llvm.org/features.html#performance states it. :slight_smile:

Regards,
Ilya

P.S. Anyway, I was surprised when clang managed to build all my projects
and all tests are working! (except just a trivial one -
http://llvm.org/bugs/show_bug.cgi?id=8182).

#9

Argyrios Kyrtzidis пишет:

2) Is it possible to use Clang' PCH in the test case above? If not then
what is a solution?

$ clang -c -Isrc -include build/test.h -o build/test.o src/test.c

-Argiris

3) What is the reason against using .h.pch automatically like gcc

does it?

Ok, with Clang 2.9 I succeeded in building my project with PCH, thanks.
I have 2 things to say.

1. I have to say that "-include" scheme is not so flexible as using PCH
automatically, really. Here is my use case: let's say I have a project
(a bunch of sources), and I want to use one PCH-file for it (and so on,
one PCH-file per project). So I set that "-include" option for the
project and now it is used for every source file in the project, without
an exception,- that is no so good! In several cases I do want to avoid it:
- my projects have C++ and C sources together; I can't use C++ PCH
header for C sources (so the hack is done - "-include" goes to CXX
options, not CPP ones).

Automatically using PCH seems like an architectural change with little benefit.
Most of your woes can be attributed to the fact that clang does not fallback to the source header when it can't use the PCH.
I think we should change the cannot-use-PCH errors to warnings and fallback to the source header, and maybe add a command-line option to enable the current behavior (hard errors when PCH can't be used).
Anyone have an objection about this ?

- in some C++ sources I still need to set #defines at first lines (e.g.
BOOST_AUTO_TEST_MAIN for Boost.Test library); now it is impossible.

If clang fallbacks to the source header, you can define them at command-line (-D).

That said, I need to create additional projects (without PCH) just for
such exceptional sources to make things to work.

2. I have got some benchmarks (compiler' speed in debug mode) gcc vs
clang for my sources (http://gitorious.org/bombono-dvd). I use Boost *a
little*, and it makes a compiler be somewhat slow. Here are compilation
times (-g -O0) for src/mgui/editor/select.cpp (39kb, not so big!):

a) without PCH:
clang++ 2.9: 5.35 seconds
g++ 4.2: 5.05 seconds

b) with PCH:
clang++ 2.9: 3.47 seconds
g++ 4.2: 2.88 seconds

Interesting, is this gcc 4.4 ? Can you file a bug and include the preprocessed header (move headers of select.cpp to the preprocessed header before including it as well).

-Argiris

#10

Argyrios Kyrtzidis пишет:

Automatically using PCH seems like an architectural change with little benefit.
Most of your woes can be attributed to the fact that clang does not fallback to the source header when it can't use the PCH.
I think we should change the cannot-use-PCH errors to warnings and fallback to the source header, and maybe add a command-line option to enable the current behavior (hard errors when PCH can't be used).
Anyone have an objection about this ?

- in some C++ sources I still need to set #defines at first lines (e.g.
BOOST_AUTO_TEST_MAIN for Boost.Test library); now it is impossible.

If clang fallbacks to the source header, you can define them at command-line (-D).

The main benefit is that one can setup settings (#define's, #include's)
per source file, not per project (= at command-line).

2. I have got some benchmarks (compiler' speed in debug mode) gcc vs
clang for my sources (http://gitorious.org/bombono-dvd). I use Boost *a
little*, and it makes a compiler be somewhat slow. Here are compilation
times (-g -O0) for src/mgui/editor/select.cpp (39kb, not so big!):

a) without PCH:
clang++ 2.9: 5.35 seconds
g++ 4.2: 5.05 seconds

b) with PCH:
clang++ 2.9: 3.47 seconds
g++ 4.2: 2.88 seconds

Interesting, is this gcc 4.4 ? Can you file a bug and include the preprocessed header (move headers of select.cpp to the preprocessed header before including it as well).

No, it is gcc 4.4; I found it the quickest among 4.0-4.5 versions. gcc
4.4 is 30% slower, gcc 4.5.1 is 15% slower on my code.
I try to file a bug if I have time.

Regards,
Ilya

#11

We explicitly didn't do this because the build system should never be telling Clang to use a PCH file if it isn't correct. However, so long as it's a remappable warning, it would be okay to make this change.

  - Doug

#12

Note that I'm not quite sure that our current PCH code can handle this. I've been modifying it under the assumption that failed PCH always means failed compilation, and I might have unknowingly done something that relies on it.

Sebastian

#13

I think we should change the cannot-use-PCH errors to warnings and fallback to the source header, and maybe add a command-line option to enable the current behavior (hard errors when PCH can't be used).
Anyone have an objection about this ?

We explicitly didn't do this because the build system should never be telling Clang to use a PCH file if it isn't correct. However, so long as it's a remappable warning, it would be okay to make this change.

Note that I'm not quite sure that our current PCH code can handle this. I've been modifying it under the assumption that failed PCH always means failed compilation, and I might have unknowingly done something that relies on it.

We don't have to recover from every PCH error, just at the point where it verifies that language options/predefines do not conflict.

-Argiris