clang windows: clang-cl support for /MP

are there any plans to support the /MP option (parallel compilation within one project) in clang-cl?
at the moment clang for windows prints out a warning that the option is not supported yet and it uses only one core while compiling a single project.

(if msvc is allowed to compile parallel independent projects it uses a multiple cores.)

This is usually handled by the build system (make -j, ninja, etc.). I’d expect this flag to actually control MSBuild, not the driver, but it’d be interesting to find out how it really works.

There are two levels of parallellization in Visual Studio/Visual C++;

1) Machine-wide setting of how many projects to build in parallel --
this informs Visual Studio/MSBuild
2) Compiler setting (/MP) of how many source files to compile in
parallel -- this actually informs CL.EXE and causes it to launch child
processes

See Bruce Dawson's treatise for the gory details:
https://randomascii.wordpress.com/2014/03/22/make-vc-compiles-fast-through-parallel-compilation/

Not sure how easy/desirable this behavior would be in clang-cl, but
from a principal standpoint I'd prefer this kind of thing in the build
system, not the compiler.

- Kim

That was my initial feeling, but if we’d like to plug clang-cl into VS then it should be able to handle this?

i think so, if clang for windows offers MSVC integration (which is brilliant) it should support this /MP option as well.

But from a 'practical standpoint', wouldn't a multithreaded compiler be the faster solution? e.g.:
  - you'd parse precompiled and/or shared headers only once for all the sources being compiled simultaneously/in-parallel
  - multithreaded LTO/LTCG 'for free'
  - threads are cheaper than processes

But from a 'practical standpoint', wouldn't a multithreaded compiler be
the faster solution? e.g.:
- you'd parse precompiled and/or shared headers only once for all the
sources being compiled simultaneously/in-parallel

If all your files have same compiler flags, macros, etc. I just read the
linked blog post and that's essentially what cl.exe does, splits files into
batches. Ninja still beats it.

- threads are cheaper than processes

Not on linux :stuck_out_tongue:

There are two levels of parallellization in Visual Studio/Visual C++;

1) Machine-wide setting of how many projects to build in parallel --
this informs Visual Studio/MSBuild
2) Compiler setting (/MP) of how many source files to compile in
parallel -- this actually informs CL.EXE and causes it to launch child
processes

See Bruce Dawson's treatise for the gory details:

https://randomascii.wordpress.com/2014/03/22/make-vc-compiles-fast-through-parallel-compilation/

Not sure how easy/desirable this behavior would be in clang-cl, but
from a principal standpoint I'd prefer this kind of thing in the build
system, not the compiler.

Given that cl implements /MP by simply launching more cl.exe subprocesses
that can oversaturate your system if MSBuild is building multiple projects
in parallel, I'm not inclined to complicate the driver with parallel
execution. If someone put together a particularly elegant and
low-maintenance burden patch, though, I think we could take it.

If Microsoft really wants to squeeze more parallelism out of VS builds,
they should expose the in-project compilation parallelism to the build
system and let MSBuild handle it.

If individual users want more parallel builds, I have to recommend that
they avoid MSBuild and try to hook VS up to ninja instead.

But from a 'practical standpoint', wouldn't a multithreaded compiler be the

faster solution? e.g.:
- you'd parse precompiled and/or shared headers only once for all the
sources being compiled simultaneously/in-parallel
- multithreaded LTO/LTCG 'for free'
- threads are cheaper than processes

When I was an intern, we thought this would be a good idea. We learned,
though, that at least for the frontend, nobody actually wants a
multi-threaded compiler. This would inevitably lead to threading bugs and
non-deterministic diagnostic output and compiler output. You really don't
want to use the AST for your last parse of a.h in your next compile or
you'll end up not emitting diagnostics that you would've otherwise, and
compilations stop being reproducible.

For basic compilation actions, you really want the safety that process
isolation gives you. Windows processes aren't as expensive as people think
they are, and on better optimized OSs they are pretty cheap.

Something like deterministic, multi-threaded LTO would be pretty cool,
though.