Late Parsed Template support for PCH

Hi All,

I’ve discovered a rather nasty issue with -fdelayed-template-parsing enabled. If this is used to build a PCH all of the stored template tokens are lost leading an incomplete AST when the PCH is used to create the complete TU.

Is this by design or a missing feature for the serialization code?

Unfortunately, due to a huge body of MSVC targeted code and a need to use PCH files to maintain acceptable performance I need to make this work seamlessly. The data that would need preserving is currently stored in the Parser and is a map of const FunctionDecl* to LateParsedTemplatedFunction*. The LateParsedTemplatedFunction is essentially a SmallVector of Tokens.

Before I dive in and attempt to add support for serializing this, is there anything I should know?

Cheers,
Will.

Hi, I implemented the original -fdelayed-template-parsing mode and I simply did not consider PCH during the design/implementation. As it is currently this should output an error instead of creating an incomplete AST.

Feel free to add support for fdelayed-template-parsing + PCH.
I never looked at the PCH part of clang so I got no advice really.

I would guess a missing feature. Problems can be demonstrated quite easily:

$ cat t.cpp
template<typename T>
T f(const T& t) {
     return t;
}

int main() {
     f(5);
}

$ clang --version
clang version 3.3 (tags/RELEASE_33/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix

# No problems without -fdelayed-template-parsing...
$ clang -emit-ast t.cpp
$ clang -o t t.ast

# But with -fdelayed-template-parsing...
$ clang -emit-ast -fdelayed-template-parsing t.cpp
$ clang -o t t.ast
/tmp/t-YAfQmm.o: In function `main':
t.ast:(.text+0x14): undefined reference to `int f<int>(int const&)'
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Tom.

It’s a missing feature. One will need to serialize the actual tokens for each of the late-parsed FunctionDecl’s, and be able to lazily deserialize and then parse the tokens when an actual definition is needed.

  • Doug

Hi All,

I've discovered a rather nasty issue with -fdelayed-template-parsing
enabled. If this is used to build a PCH all of the stored template
tokens are lost leading an incomplete AST when the PCH is used to create
the complete TU.

Is this by design or a missing feature for the serialization code?

I would guess a missing feature. Problems can be demonstrated quite
easily:

$ cat t.cpp
template<typename T>
T f(const T& t) {
    return t;
}

int main() {
    f(5);
}

$ clang --version
clang version 3.3 (tags/RELEASE_33/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix

# No problems without -fdelayed-template-parsing...
$ clang -emit-ast t.cpp
$ clang -o t t.ast

# But with -fdelayed-template-parsing...
$ clang -emit-ast -fdelayed-template-parsing t.cpp
$ clang -o t t.ast
/tmp/t-YAfQmm.o: In function `main':
t.ast:(.text+0x14): undefined reference to `int f<int>(int const&)'
clang: error: linker command failed with exit code 1 (use -v to see
invocation)

It's probably a good idea to paste this into a bug report (
http://llvm.org/bugs) to make sure it's not lost.

Thanks for the feedback.

I’ve started work on supporting serialization and I’ve made reasonable progress so far. The need for lazy deserialization hadn’t escaped me but I was concentrating on just getting it up and running initially - as slightly slower would be better than completely broken. I can look into lazy deserialization but I may need a few pointers on the best approach and if it’s going to be a substantial job it might have to wait until a later patch.

  • Will.

It’s already filed as

http://llvm.org/bugs/show_bug.cgi?id=9992

  • Doug