Hi,
BTW: is there any other trick to force instantiation, without generating
any text?
I guess the main C++11 trick for this is to use "extern templates".
FYI, the name for this feature is "explicit instantiation
declarations". "extern templates" is the name of the pre-standard GCC
extension.
I've
used extern templates quite a bit to cut down on compilation times (and
memory usage) of template-heavy code, but its impact is limited by the fact
that building the AST (within the instantiation) is still a pretty big deal.
For example, I had some code that needed about 6GB and lots of time to
compile (with GCC), and by putting some of the main template instantiations
into a separate cpp file, I would end up with about 4GB to compile each of
the two separate cpp files, in other words, 6GB turned into 4GB + 4GB,
roughly, and about the same for compilation time, meaning that this was a
net loss in overall compilation costs, but with benefits like actually being
able to compile it (without overwhelming the system) and reducing the need
for recompilations (e.g., recompiling only one of the cpp files, instead of
everything). BTW, with Clang, this is significantly faster to compile
compared to GCC (1/2 of the time, and 1/3 of the memory, approximately).
But the point is that it seems to me (I might be wrong) that even with an
existing instantiation (extern or not), the compilation time/memory needed
to deal with it is still significant.
I have one related question to the clang devs:
When you have an extern template declaration in a header file that is part
of a module, is the complete AST (or whatever else) of that template
instantiation included in the module?
Whenever a module triggers the instantiation of a template for any
reason, that instantiation is stored in the module and will be reused
if a user of that module needs it.
In principle, we could extend the modules system with a template
instantiation repository to cache the results of instantiating
templates from modules, but I don't think anyone is working on, or
planning, such a system for Clang at the moment.
That sounds to me like automatically making all template instantiations
"extern", right?
No, that wouldn't make any difference -- explicit instantiations in a
user of the module still currently get performed each time that module
user is compiled. The idea here is: if a user of a module triggers a
template instantiation, we'd first check to see if that instantiation
is cached in some external template instantiation cache stored
alongside the module cache, and if not, we'd add it to that cache.
This would likely only be possible if the instantiation only depends
on entities imported from modules (as is the case here), and not if a
template argument refers to a local type.
(This is pretty similar to what we'd do to support exported templates,
as it happens...)
This could actually be pretty awesome. But I'm worried
about the implications for the standard rules (it looks to me like it could
be allowed). That could be part of the rules for modules, that any
instantiation encountered for a template declared/defined inside a module
would implicitly be considered "extern" and cached somewhere alongside the
module's cache.
There is a conformance impact here: we would say that the
instantiation of a template would be performed in a context where only
the "associated modules" are visible (that is, those modules with
which the template-name and template-arguments are associated). That
has the potential to break some legitimate (but probably questionable)
code.
from my experience, it was rather the template instantiation that was
causing long build times and not the preprocessing.
That's right. Last time I ran my code through GCC with compilation profiling
enabled, it showed that template instantiations accounted for about 98% of
the compilation time. For template-heavy code, pre-processing, parsing, AST
building and all the "normal" compilation stuff is negligible. And for code
that isn't template-heavy, compilation times are rarely a big problem.
OK, that might be true in some situations, but "in main source file"
vs "in a header file", and "in template instantiation" vs "in code
written by user" are orthogonal axes, so this doesn't really prove
anything. If you're doing template-heavy things in your header files,
modules should help. If you're doing template-heavy things in your
.cpp file, modules won't help so much; that's not a problem they aim
to solve.
