Out of Memory Building Clang and LLVM

Greetings,

Seems I'm running out of memory on a host system trying to build llvm, tooling and clang. It was configured
as:
cmake -DLLVM_ENABLE_PROJECTS=clang -G "Unix Makefiles" ../llvm

and build with:

make -j4 using gcc as the host compiler, version 8.

The dmesg I'm getting is related to the linker and not sure if the linker has a bug or it's normal
to expect 3GB or more used for linking clang. Doesn't seem to be but not sure.

dmesg output:
[186517.023397] Out of memory: Kill process 2143 (ld) score 286 or sacrifice child
[186517.023403] Killed process 2143 (ld) total-vm:4622188kB, anon-rss:3202188kB, file-rss:4kB, shmem-rss:0kB

Nick

I can't comment on this particular configuration, but last I
heard it still builds in 3GB. That said, you may want to include
-DLLVM_PARALLEL_LINK_JOBS=1 in your linker arguments. At least with
ninja (not sure about make), there tend to be many programs being linked
at the end and on a low-memory system this can lead to out of memory
conditions.

-- Brooks

HI,

I have a 16 Core/32 Thread system with 32GB RAM, and it can have difficulty building CLANG/LLVM.
The link stage definitely needs to be with make -j1.

That being said, if you force a “Release” version, and not the “Debug” version, much less RAM is needed.

Kind Regards

James

Clang definitely hits a pathological case in LD that makes it do terrible things if you use use more than a process or two in DEBUG.

However, if you use “gold” or “LLD” instead, it works fine. Assuming you have LLD, you can do:

-DLLVM_USE_LINKER=lld or –DLLVM_ENABLE_LLD=ON

Otherwise you should have gold available, which you can do with:

-DLLVM_USE_LINKER=gold

I can only run about 3-4 links at a time with LD and can do –j30 with no problems with Gold.

From

Just finished a clang build on Xubuntu 19.04 using cpp.

cmake -G Ninja -DLLVM_ENABLE_PROJECTS=“clang” …/llvm

Memory usage hit a maximum at the link for bin/clang-9 at 13 gigabytes with ninja -j 1.

Disk usage for the build directory was 61 gigabytes.

My sense is that this kinds of usage is more than what a typical new user would expect. Perhaps the page above might be changed to reflect the expected result and with an easier route noted by Erich below.

What OS are you guys compiling on?

When I compile LLVM in MinSizeRel mode it doesn’t take nearly that much ram, but I’m on OSX with ld64.

13GB to link clang and 60+ GB build directory is consistent with my experience building LLVM+Clang on Linux using the GNU linker.

Using lld will dramatically reduce memory usage during linking and will also radically speed up linking. As will reducing the number of LLVM targets (LLVM_TARGETS_TO_BUILD). The documentation on the GettingStarted page is pretty far out of date, but it is tough to give a reasonable estimate given that there are so many configuration values that could impact the size and system resources required for a build.

Since it was brought up in another email, generally speaking users on OS X with ld64 don’t encounter these kinds of issues. That is because ld64’s algorithm for linking is radically different from the GNU linker which results in shorter link times and lower memory usage.

-Chris

13GB to link clang and 60+ GB build directory is consistent with my experience building LLVM+Clang on Linux using the GNU linker.

Using lld will dramatically reduce memory usage during linking and will also radically speed up linking. As will reducing the number of LLVM targets (LLVM_TARGETS_TO_BUILD). The documentation on the GettingStarted page is pretty far out of date, but it is tough to give a reasonable estimate given that there are so many configuration values that could impact the size and system resources required for a build.

Since it was brought up in another email, generally speaking users on OS X with ld64 don’t encounter these kinds of issues. That is because ld64’s algorithm for linking is radically different from the GNU linker which results in shorter link times and lower memory usage.

I imagine the other reason is the debug info model (those sort of numbers of memory usage and build directory size sound like a Debug build to me) - leaving the debug info in the object files - less work for the linker and smaller linked binaries. You can get /some/ of that on Linux if you enable Split DWARF (LLVM_USE_SPLIT_DWARF).

That sounds like a Debug build. FreeBSD still selfhosts including llvm
8 on i386 and armv7 but that's a Release build. Debug builds haven't
been viable for ages on 32-bit systems.

-- Brooks