CMake build of LLVM/clang with -DCMAKE_BUILD_TYPE=Release does not create release versions?

my build environment:

Win7 x64
VStudio 2017 Community Edition 15.8.4 (latest)
CMake 3.12.1 (x86)
git 2.19.0 (latest, x64)
Python 2.7.2 (x86)

directory structure

test
 llvm <-- git clone https://github.com/llvm-mirror/llvm
   tools
     clang <-- git clone GitHub - llvm-mirror/clang: Mirror kept for legacy. Moved to https://github.com/llvm/llvm-project
 llvm_build

Debug build: clean build, llvm_build is deleted before

llvm_build> cmake -Thost=x64 -G "Visual Studio 15 2017 Win64" -DCMAKE_BUILD_TYPE=Debug -DLLVM_TARGETS_TO_BUILD=host -DLLVM_BUILD_EXAMPLES=1 -DCLANG_BUILD_EXAMPLES=1 -DLLVM_INCLUDE_TESTS=OFF -DLLVM_INCLUDE_DOCS=OFF ..\llvm

builds for hours, a few warning, no errors -> llvm_build is ~44GB

i can find many working libs/exes(also examples) in llvm_build\Debug\(lib|bin)

then i tried to build release versions

Release build: clean build, llvm_build is deleted before

llvm_build> cmake -Thost=x64 -G "Visual Studio 15 2017 Win64" -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=host -DLLVM_BUILD_EXAMPLES=1 -DCLANG_BUILD_EXAMPLES=1 -DLLVM_INCLUDE_TESTS=OFF -DLLVM_INCLUDE_DOCS=OFF ..\llvm

builds for hours, a few warning, no errors -> llvm_build is ~47GB (i though Release would be smaller?)

i can find many working libs/exes(also examples) in llvm_build\Debug\(lib|bin) - the executables seems to be larger as in "Debug"-Build?

llvm-build\Release\bin just contains llvm-lit.py

why is the debug folder populated and where i can find the Release build libs/exes?

If I understand correctly, you need to set Release mode within the VS IDE because with that generator the CMAKE_BUILD_TYPE variable is ignored because it is a “multi-configuration target”.

see https://stackoverflow.com/questions/24460486/cmake-build-type-not-being-used-in-cmakelists-txt

because with that generator the CMAKE_BUILD_TYPE variable is ignored

>because it is a "multi-configuration target".

thanks for the link, is that a bug in the CMake configuration (or better not getting any warning) or is there just documentation missing?

so i can use --config Debug or --config Release and get the correct results - i hope that works the build takes hours

strange is that -DCMAKE_BUILD_TYPE=Debug results in a different build-directory size compared to -DCMAKE_BUILD_TYPE=Release - so its not fully ignored?

Regarding the Debug build taking hours, are you setting the optimized
tablegen option? If you aren't the Debug build will take a very long
time. In addition, make sure you have plenty of memory, because the
link stages take up a lot on Debug builds.

are you setting the optimized tablegen option?

LLVM_OPTIMIZED_TABLEGEN is not active - i will try that

>In addition, make sure you have plenty of memory

i've got only Intel Core 2 Quad CPU Q8300/2.50Ghz and 8GB RAM for testing
not much cpu power AND memory

beware of small trap:

cmake --build . --config Release

works

cmake --config Release --build .

breaks after does not build - leaves build without errors

thx for the help - debug and release version is built

The relevant documentation regarding the behavior of the VS generator is here:

https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html

The relevant quote is:

“This variable is only meaningful to single-configuration generators (such as Makefile Generators and Ninja) i.e. those which choose a single configuration when CMake runs to generate a build tree as opposed to multi-configuration generators which offer selection of the build configuration within the generated build environment.”

-Chris

With debug builds and gnu ld some of the links take up to 9 or 10 GB of RAM, so with only 8 GB you’d want -DLLVM_PARALLEL_LINK_JOBS=1 and even then still be swapping.

The OS X linker uses far less RAM, and I’m told gold is better too, so maybe there’s also no problem on Windows. Or not as much of one. I think I’d use -DLLVM_PARALLEL_LINK_JOBS=1 anyway with only 8 GB system RAM.

Unfortunately, from personal experience, LLVM_PARALLEL_LINK_JOBS
appears to have no effect for Visual Studio builds, AND some of the
tools linked can take up several GB of RAM, so you'll end up with
paging issues. I ended up getting a RAM upgrade in order to sensibly
build it (note, I think builds with debug information take more memory
than those that don't, and also the clang projects were the worst
culprits.

as i understand it correct cl.exe is self parallelizing, starting serveral sub-cl processes on itself - or am i wrong?

LLVM_PARALLEL_LINK_JOBS is ninja-only option.

just to make it clear: i not running out of memory - the 8GB are enough for Debug/Release build only my 2 cores are dying for at lease 2-3h(i think) at 100% workload :slight_smile:

Hi all,

I’ve seen it pop up a few times now: linking LLVM uses too much memory. For me, using a VM with 10GB RAM was not enough, it still started swapping and the job took way too long (I had to kill it actually).

Using the following CMake options, I reduced peak memory usage to < 4 GB for a RelWithDebInfo build (LLVM only, including tools but not cfe/clang).
Do NOT set LLVM_USE_LINKER or set it to gnu.ld
LLVM_PARALLEL_LINK_JOBS=1
CMAKE_EXE_LINKER_FLAGS=“-Wl,–reduce-memory-overheads -Wl,–hash-size=1021”

The --reduce-memory-overheads and the --hash-size flags are only supported by the GNU linker as far as I know.

The first option actually implies the 1021 value set in the second one (and thus -Wl,--hash-size=1021 could be omitted), but I suspect that you can play around with the value. It is supposed to set the size of some internal hash-table sizes of the linker and not effect the output. (Look up your ld manpage for more info: https://linux.die.net/man/1/ld) Increasing the number might increase your memory usage and speed up compilation a bit, until you start swapping. If I had the time to recompile my LLVM repo a few times, I would try to tune this number to use up most, but not all of my RAM for optimal speed.

I hope this helps someone. If you have any additional tips, comments or gotchas with this approach, please add them.

All the best,
Boldizsár Palotás

This message is intended only for the recipient(s) named above. It may contain proprietary information and/or
protected content. Any unauthorised disclosure, use, retention or dissemination is prohibited. If you have received
this e-mail in error, please notify the sender immediately. ESA applies appropriate organisational measures to protect
personal data, in case of data privacy queries, please contact the ESA Data Protection Officer (dpo@esa.int).

LLVM_PARALLEL_LINK_JOBS is ninja-only option.

I just noticed we didn’t properly document that option in LLVM’s CMake. It is a wrapper around the JOB_POOL_LINK target property, which CMake documents to be Ninja only:

https://cmake.org/cmake/help/latest/prop_tgt/JOB_POOL_LINK.html

-Chris

The GNU linker’s handling of debug information is really terrible, and results in frequently running out of memory when linking LLVM with debug information generated.

You can reduce memory usage by either not building with debug information, or using a different linker.

On linux the Gold linker or LLD are both viable alternatives to gnu.ld, and both have dramatically lower memory usage.

-Chris

The GNU linker’s handling of debug information is really terrible, and results in frequently running out of memory when linking LLVM with debug information generated.

You can reduce memory usage by either not building with debug information, or using a different linker.

On linux the Gold linker or LLD are both viable alternatives to gnu.ld, and both have dramatically lower memory usage.

gold and lld are in the same ballpark if you compare them with GNU bfd linker, but usually lld uses less memory than gold (and is usually faster than gold as well).

LLVM_USE_SPLIT_DWARF=ON also greatly improves the situation by (I
think) skipping linking the debug info entirely. The disadvantage is
that lldb still doesn't quite work on the result (gdb does) and
loading things into gdb takes longer (I think).

Cheers.

Tim.

I hope this helps someone. If you have any additional tips, comments or gotchas with this approach, please add them.

LLVM_USE_SPLIT_DWARF=ON also greatly improves the situation by (I
think) skipping linking the debug info entirely.

Pretty much

The disadvantage is
that lldb still doesn’t quite work on the result (gdb does) and
loading things into gdb takes longer (I think).

The increased loading time in GDB can be avoided by enabling gdb-index (pass -ggnu-pubnames to the compiles, and -Wl,-gdb-index to the link step) so the debugger doesn’t have to parse all the DWARF to start.

  • Dave