Module resolution during preprocessing

Clang Frontend
1

Hi,

I found this surprising behavior of Clang's Modules-TS implementation:

$ clang-5.0 --version
clang version 5.0.0-svn304373-1~exp1 (trunk)

$ cat <<EOF >test.cxx
import foo
EOF

$ clang++-5.0 -std=c++1z -fmodules-ts -E test.cxx
test.cxx:1:8: fatal error: module 'foo' not found

Seeing that modules is a language-level construct, it's not clear why
they must be resolvable during preprocessing. Or am I missing something
subtle here?

Thanks,
Boris

2

Clang supports mixing Modules TS modules with header modules. Header
modules support exporting macros. Therefore, because "import foo;" might
import a module that exports a macro, it must be available during
preprocessing.

3

Richard Smith <richard@metafoo.co.uk> writes:

Clang supports mixing Modules TS modules with header modules. Header
modules support exporting macros. Therefore, because "import foo;" might
import a module that exports a macro, it must be available during
preprocessing.

This causes a bit of a chicken and egg problem for us: we analyze the
preprocessed output to discover which modules are required but need to
know the required modules before preprocessing.

I could remove the -fmodules-ts but that's not ideal since this option
could conceivably alter predefined macros (like __cpp_modules).

Is there a way to disable this "header modules" support and get a pure
Modules TS semantics? If not, do you think such a mode could be added?
We could have only enabled "header modules" if both -fmodules-ts and
-fmodules are specified, but that ship might have already sailed.

Thanks,
Boris

4

Richard Smith <richard@metafoo.co.uk> writes:

> Clang supports mixing Modules TS modules with header modules. Header
> modules support exporting macros. Therefore, because "import foo;" might
> import a module that exports a macro, it must be available during
> preprocessing.

This causes a bit of a chicken and egg problem for us: we analyze the
preprocessed output to discover which modules are required but need to
know the required modules before preprocessing.

I could remove the -fmodules-ts but that's not ideal since this option
could conceivably alter predefined macros (like __cpp_modules).

Is there a way to disable this "header modules" support and get a pure
Modules TS semantics? If not, do you think such a mode could be added?

That wouldn't help if we want to give Modules TS modules the ability to
export macros as an extension (which it seems like we probably would want
to do). But... at that point, the problem is fundamental: an
earlier-imported module could export a macro that is expanded later and
generates a module import. Given that your analysis strategy is
incompatible with such an extension anyway, I think your request for a mode
that sets up the modules-specific build configuration but doesn't handle
'import' in the preprocessor is reasonable.

Would you be interested in writing a patch for this?

We could have only enabled "header modules" if both -fmodules-ts and
-fmodules are specified, but that ship might have already sailed.

It wouldn't be too late for that. The command-line interface for header
modules has been around for a long while, have been used in numerous build
systems, and would be painful to change at this point, but the same is not
true for the modules TS command-line interface. But we would still have the
issue that we explicitly permit a Modules TS import to import a (header)
module that exports macros, so I don't think this solves your problem.

5

Richard Smith <richard@metafoo.co.uk> writes:

That wouldn't help if we want to give Modules TS modules the ability to
export macros as an extension (which it seems like we probably would want
to do). But... at that point, the problem is fundamental: an
earlier-imported module could export a macro that is expanded later and
generates a module import.

In this setup how do you see a build system discovering BMIs it needs
to build? To me it seems the only way is for some kind of a notification
or callback by the compiler back into the build system as it processes
each import. And all the BMIs will have to be built serially.

At the same time, if one wants to have a module that also exports
macros, they can provide a header that imports the module proper and
then defines the macros.

Would you be interested in writing a patch for this?

I can give it a try. In light of the above it seems a more appropriate
approach would be to provide a switch that simply disables import
handling during preprocessing (in a sense similar to -frewrite-includes)
rather than some kind of "pure TS" mode. Maybe -fignore-imports or
-fmodules-ignore?

BTW, a couple of observations:

1. -E -frewrite-includes already appears to ignore imports which is not
   quite correct since a macro defined by a preceding import could be
   used to #ifdef-out an #include.

2. -E does not treat 'module M;' (in module interface unit) as an import.
   This will have to change if TS modules are extended to support macros.
   So in this sense -fmodules-ignore is better than -fignore-imports.

If this sounds like a reasonable approach, I can give it a go.

Thanks,
Boris