[Clang] Modules build daemon: build system agnostic support for explicitly built modules

Community GSoC
,
1

Description of the project: Clang currently handles modules independently in each clang instance using the filesystem for synchronization of which instance builds a given module. This has many issues with soundness and performance due to tradeoffs made for module reuse and filesystem contention.

Clang has another way of building modules, explicitly built modules, that currently requires build system changes to adopt. Here the build system determines which modules are needed, for example by using clang-scan-deps, and ensures those modules are built before running the clang compile task that needs them.

In order to allow adoption of this new way of building modules without major build system work we need a module build daemon. With a small change to the command line, clang will connect to this daemon and ask for the modules it needs. The module build daemon then either returns an existing valid module, or builds and then returns it.

There is an existing open source dependency scanning daemon that is in a llvm-project fork. This only handles file dependencies, but has an IPC mechanism. This IPC system could be used as a base for the modules build daemon, but does need to be extended to work on Windows.

More resources:

Expected result: A normal project using Clang modules with an existing build system (like Make or CMake) can be built using only explicitly built modules via a modules build daemon.
Desirable skills: Intermediate C++ programming skills; familiarity with compilers; familiarity with Clang is an asset, but not required.
Project size: 175h or 350h depending on reuse of IPC
Difficulty: medium
Confirmed Mentors: Michael Spencer, Jan Svoboda

4 Likes
2

Hi @Bigcheese! I’m a junior year computer science student and am familiar with compilers (I actually have compiler construction as a subject this semester!), and have good C++ skills. I’m trying to gain familiarity with Clang. Would love to get your guidance on what issues I can explore and try and contribute to in order to be better prepared to take on this task and write a good proposal

3

Hi @Bigcheese and @jansvoboda11, one quick question.

Is the ultimate goal to remove support for implicit modules and only support implicitly found explicit modules via a daemon and pure explicit modules via the libclang API? In “Implicitly discovered, explicitly built Clang modules” Jan mentioned the performance penalty being paid due to leftovers from implicit modules, so I was curious if that meant you all aim to eventually remove support for implicit modules. Thanks!

4

Yes, the goal is to eventually completely remove Clang’s existing implicit modules build system.

5

Hello, I’m a sophomore Computer science student with lots of C++ experience. Do you have any tips for getting started with this project?
Thanks!