modules: cyclic dependency, performance questions

Hi,

I’m trying to set up modules for some platform headers that are found in 3 separate subdirectories:

(platform)/target/include

(platform)/target/include_common

(platform)/host_tools/lib/clang/include

Unfortunately various headers include headers from other directories and visa/versa, such that having a module.map in each of these directories fails to build, giving me an errors such as:

(file): fatal error: cyclic dependency in module ‘platform_include’: platform _include → platform _host_tools → platform _include

Is there a way to address this without fixing the circular dependencies?

One way that seems to work is to put it all in one module map, at the (platform) level, with relative paths in the header directives. It appears the compiler will walk up the directories until it finds the module.map file, right?

Regarding the #include to module mechanism, can I assume the compiler matches the module even if the path given in the #include doesn’t match the path in the “header” directives in the module.map file, but does refer to the same absolute file? I.e. the #include references “stdio.h”, and a “-I(platform)/target/include” option is given to the compiler, but the (platform)/module.map “header” directive references “target/include/stdio.h”.

I have a test bed that includes over 400 stub files that just #include one header, and a makefile to build them. As a rough performance test, with or without the –fmodules option, they all compile okay in a clean make, but the –fmodules version is about 15% slower (in a run after the modules are compiled). (Note that I have the compiler output using –emit-llvm because I’m coercing it to think it’s a ps4 compiler, since our version doesn’t have modules support yet, and the .o output seems to be broken for x86_64.) I heard others got better performance, but I don’t know how it’s measured. Any thoughts?

Thanks.

-John

Hi,****

** **

I’m trying to set up modules for some platform headers that are found in 3
separate subdirectories:****

** **

(platform)/target/include****

(platform)/target/include_common****

(platform)/host_tools/lib/clang/include****

** **

Unfortunately various headers include headers from other directories and
visa/versa, such that having a module.map in each of these directories
fails to build, giving me an errors such as:****

** **

(file): fatal error: cyclic dependency in module 'platform_include':
platform _include -> platform _host_tools -> platform _include****

** **

Is there a way to address this without fixing the circular dependencies?**
**

** **

One way that seems to work is to put it all in one module map, at the
(platform) level, with relative paths in the header directives. It appears
the compiler will walk up the directories until it finds the module.map
file, right?****

** **

Regarding the #include to module mechanism, can I assume the compiler
matches the module even if the path given in the #include doesn’t match the
path in the “header” directives in the module.map file, but does refer to
the same absolute file? I.e. the #include references “stdio.h”, and a
“-I(platform)/target/include” option is given to the compiler, but the
(platform)/module.map “header” directive references
“target/include/stdio.h”.****

** **

I have a test bed that includes over 400 stub files that just #include one
header, and a makefile to build them. As a rough performance test, with or
without the –fmodules option, they all compile okay in a clean make, but
the –fmodules version is about 15% slower (in a run after the modules are
compiled). (Note that I have the compiler output using –emit-llvm because
I’m coercing it to think it’s a ps4 compiler, since our version doesn’t
have modules support yet, and the .o output seems to be broken for
x86_64.) I heard others got better performance, but I don’t know how it’s
measured. Any thoughts?****

**

How much is actually in the header? If it's not much, then I wouldn't
expect to see a speedup, and a slowdown doesn't surprise me. Isn't the
speed advantage for modules meant to be in avoiding redoing parsing/sema
for the header on each inclusion? (i.e., it's basically caching the
results). If there isn't much to cache, then I wouldn't expect a speed
improvement.

-- Sean Silva

Hi,

I’m trying to set up modules for some platform headers that are found in 3 separate subdirectories:

(platform)/target/include
(platform)/target/include_common
(platform)/host_tools/lib/clang/include

Unfortunately various headers include headers from other directories and visa/versa, such that having a module.map in each of these directories fails to build, giving me an errors such as:

(file): fatal error: cyclic dependency in module 'platform_include': platform _include -> platform _host_tools -> platform _include

Is there a way to address this without fixing the circular dependencies?

You've found it: collect the cyclic includes into a single module.

One way that seems to work is to put it all in one module map, at the (platform) level, with relative paths in the header directives. It appears the compiler will walk up the directories until it finds the module.map file, right?

Yes, that's correct.

Regarding the #include to module mechanism, can I assume the compiler matches the module even if the path given in the #include doesn’t match the path in the “header” directives in the module.map file, but does refer to the same absolute file? I.e. the #include references “stdio.h”, and a “-I(platform)/target/include” option is given to the compiler, but the (platform)/module.map “header” directive references “target/include/stdio.h”.

The module map infrastructure matches files via their file system identity (e.g., the inode on *nix systems); it doesn't matter how you spell the path to find that file.

I have a test bed that includes over 400 stub files that just #include one header, and a makefile to build them. As a rough performance test, with or without the –fmodules option, they all compile okay in a clean make, but the –fmodules version is about 15% slower (in a run after the modules are compiled). (Note that I have the compiler output using –emit-llvm because I’m coercing it to think it’s a ps4 compiler, since our version doesn’t have modules support yet, and the .o output seems to be broken for x86_64.) I heard others got better performance, but I don’t know how it’s measured. Any thoughts?

I don't really know what you're measuring. Modules tends to be a win when there is a lot of code in your headers (as tends to happen in C++ and Objective-C, for example), because the cost of parsing said headers is significantly greater than loading the corresponding top-level module. If you're just benchmarking module import vs. including fairly small headers, it won't be a win.

  - Doug