I ported a big third-party header library to C++20 modules. I expected a decrease in compilation time, as it must only compile once per project and not per translation unit. Instead, the build times increased. -ftime-report reports that half of the time is spent in reading modules. I tried to profile it and most of the time is spent in clang::ASTReader::ReadDeclRecord(unsigned int).
Note that <iostream> is only used, to inflate the AST.
Maybe it would be possible to cache parts of clang::ASTWriter::GenerateNameLookupTable and speed up the AST reading. I don’t know anything about clang internals, so I can’t say how to best fix this issue.
The number of calls to ASTWriter::WriteDecl, ASTDeclWriter::Visit, and DeclContext::lookup stayed nearly the same. The most significant difference is that ASTReader::FindExternalVisibleDeclsByName gets called often when using import. I noticed that the call chains while reading the AST are deep. I tried to print the Declaration it visits by modifying the AST reader, however, I only got crashes.
I created a repo that highlights this issue, note this is neither minimal nor beautiful and only a bad port to C++ Modules, but it could help to find the root cause of this issue. Please notify me if there are problems building this.
In recent weeks I tried using C++ modules on a project. It was so cool at first to be getting rid of header files. Alas, my compilation times eventually went insane. Some files were taking over a minute. It became too hard to get work done. I just spent the time to go back to headers, and compilation time is back to normal.
I’ve reported some bugs connected to modules, but on this one I don’t think I can get a “minimal” reproducer. I suspect it’s something that happens as the includes and imports start to pile up.
But I’ve glimpsed the promised land, so I eagerly await the bug fixes and further implementation…
BTW, is there a lot “constexpr”/“consteval” in your code? If yes, could you test again by removing these “constexpr”/“consteval”? There is a known problem that clang doesn’t cache the evaluated value and the problem become much more significant in modules.
Not that I know of. It’s possible one of my 3rd party libs has some. I’ll keep that in mind.
Maybe related: the size of the .pcm files in my build directory made me wonder. I have only about 50K lines of C++, and the pcms were 1.5GB total. Way more space than the object files.
Some input that could be useful; consider the following files:
C++
// foo.cppm
module;
#include <compare>
export module foo;
export struct Foo
{
int value{};
auto operator <=>(const Foo &) const = default;
};
// bar.cppm
export module bar;
import foo;
export struct Bar
{
Foo foo{};
};
This compiles successfully under clang, but if you compile it with msvc you have to include <compare> in bar or export it in foo. According to an msvc dev this is by design. As msvc is not impacted by these high compile times, this could be why it happens in clang.
Maybe related: the size of the .pcm files in my build directory made me wonder. I have only about 50K lines of C++, and the pcms were 1.5GB total. Way more space than the object files.
You can look at the discussion here (C++ modules support? (#18355) · Issues · CMake / CMake · GitLab) to get a feeling about the one phase compilation model, the two phase compilation model, the fat BMI and thin BMI. Though I don’t think we can get a real “thin” enough BMI without refactoring the current design of BMI in clang…
And in my local experiments, the size of pcms is similar with yours and I have 18 module units. Then the module units are completely independent with each other. And I write cmake scripts manually to support the two phase compilation for my project. Finally, I get 50% compilation time reduced.
For your project, if the number of module units is not a lot, maybe you can try if you can improve it by using two phase compilation manually.
I feel this is an issue in MSVC and it doesn’t matter with the compilation speed. Since in bar we didn’t access anything about comparison, it looks not good for me to emit an error for this.