LLD issue on a massively parallel build machine

Hi,
On a 1296-core Intel machine with 376 GB, setting

-DLLVM_PARALLEL_LINK_JOB=1

does not help (switching back to ld scales) see:

[5085/5201] Linking CXX executable bin/clang-11
FAILED: bin/clang-11
: && /home/usr4/c74014i/opt/clang/current/bin/clang++ -stdlib=libc++ -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3 -DNDEBUG -stdlib=libc++ -fuse-ld=lld -Wl,–color-diagnostics -Wl,-allow-shlib-undefined -Wl,–export-dynamic -Wl,-O3 tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o tools/clang/tools/driver/CMakeFiles/clang.dir/cc1gen_reproducer_main.cpp.o -o bin/clang-11 -Wl,-rpath,"$ORIGIN/…/lib" lib/libLLVMAArch64CodeGen.a lib/libLLVMAArch64AsmParser.a lib/libLLVMAArch64Desc.a lib/libLLVMAArch64Disassembler.a lib/libLLVMAArch64Info.a lib/libLLVMAArch64Utils.a lib/libLLVMAMDGPUCodeGen.a lib/libLLVMAMDGPUAsmParser.a lib/libLLVMAMDGPUDesc.a lib/libLLVMAMDGPUDisassembler.a lib/libLLVMAMDGPUInfo.a lib/libLLVMAMDGPUUtils.a lib/libLLVMARMCodeGen.a lib/libLLVMARMAsmParser.a lib/libLLVMARMDesc.a lib/libLLVMARMDisassembler.a lib/libLLVMARMInfo.a lib/libLLVMARMUtils.a lib/libLLVMAVRCodeGen.a lib/libLLVMAVRAsmParser.a lib/libLLVMAVRDesc.a lib/libLLVMAVRDisassembler.a lib/libLLVMAVRInfo.a lib/libLLVMBPFCodeGen.a lib/libLLVMBPFAsmParser.a lib/libLLVMBPFDesc.a lib/libLLVMBPFDisassembler.a lib/libLLVMBPFInfo.a lib/libLLVMHexagonCodeGen.a lib/libLLVMHexagonAsmParser.a lib/libLLVMHexagonDesc.a lib/libLLVMHexagonDisassembler.a lib/libLLVMHexagonInfo.a lib/libLLVMLanaiCodeGen.a lib/libLLVMLanaiAsmParser.a lib/libLLVMLanaiDesc.a lib/libLLVMLanaiDisassembler.a lib/libLLVMLanaiInfo.a lib/libLLVMMipsCodeGen.a lib/libLLVMMipsAsmParser.a lib/libLLVMMipsDesc.a lib/libLLVMMipsDisassembler.a lib/libLLVMMipsInfo.a lib/libLLVMMSP430CodeGen.a lib/libLLVMMSP430AsmParser.a lib/libLLVMMSP430Desc.a lib/libLLVMMSP430Disassembler.a lib/libLLVMMSP430Info.a lib/libLLVMNVPTXCodeGen.a lib/libLLVMNVPTXDesc.a lib/libLLVMNVPTXInfo.a lib/libLLVMPowerPCCodeGen.a lib/libLLVMPowerPCAsmParser.a lib/libLLVMPowerPCDesc.a lib/libLLVMPowerPCDisassembler.a lib/libLLVMPowerPCInfo.a lib/libLLVMRISCVCodeGen.a lib/libLLVMRISCVAsmParser.a lib/libLLVMRISCVDesc.a lib/libLLVMRISCVDisassembler.a lib/libLLVMRISCVInfo.a lib/libLLVMRISCVUtils.a lib/libLLVMSparcCodeGen.a lib/libLLVMSparcAsmParser.a lib/libLLVMSparcDesc.a lib/libLLVMSparcDisassembler.a lib/libLLVMSparcInfo.a lib/libLLVMSystemZCodeGen.a lib/libLLVMSystemZAsmParser.a lib/libLLVMSystemZDesc.a lib/libLLVMSystemZDisassembler.a lib/libLLVMSystemZInfo.a lib/libLLVMWebAssemblyCodeGen.a lib/libLLVMWebAssemblyAsmParser.a lib/libLLVMWebAssemblyDesc.a lib/libLLVMWebAssemblyDisassembler.a lib/libLLVMWebAssemblyInfo.a lib/libLLVMX86CodeGen.a lib/libLLVMX86AsmParser.a lib/libLLVMX86Desc.a lib/libLLVMX86Disassembler.a lib/libLLVMX86Info.a lib/libLLVMX86Utils.a lib/libLLVMXCoreCodeGen.a lib/libLLVMXCoreDesc.a lib/libLLVMXCoreDisassembler.a lib/libLLVMXCoreInfo.a lib/libLLVMAnalysis.a lib/libLLVMCodeGen.a lib/libLLVMCore.a lib/libLLVMipo.a lib/libLLVMAggressiveInstCombine.a lib/libLLVMInstCombine.a lib/libLLVMInstrumentation.a lib/libLLVMMC.a lib/libLLVMMCParser.a lib/libLLVMObjCARCOpts.a lib/libLLVMOption.a lib/libLLVMScalarOpts.a lib/libLLVMSupport.a lib/libLLVMTransformUtils.a lib/libLLVMVectorize.a -lpthread lib/libclangBasic.a lib/libclangCodeGen.a lib/libclangDriver.a lib/libclangFrontend.a lib/libclangFrontendTool.a lib/libclangSerialization.a lib/libLLVMAArch64Desc.a lib/libLLVMAArch64Info.a lib/libLLVMAArch64Utils.a lib/libLLVMMIRParser.a lib/libLLVMAMDGPUDesc.a lib/libLLVMAMDGPUInfo.a lib/libLLVMAMDGPUUtils.a lib/libLLVMARMDesc.a lib/libLLVMARMInfo.a lib/libLLVMARMUtils.a lib/libLLVMHexagonDesc.a lib/libLLVMHexagonInfo.a lib/libLLVMLanaiDesc.a lib/libLLVMLanaiInfo.a lib/libLLVMSystemZDesc.a lib/libLLVMSystemZInfo.a lib/libLLVMWebAssemblyDesc.a lib/libLLVMWebAssemblyInfo.a lib/libLLVMCFGuard.a lib/libLLVMGlobalISel.a lib/libLLVMAsmPrinter.a lib/libLLVMDebugInfoDWARF.a lib/libLLVMSelectionDAG.a lib/libLLVMMCDisassembler.a lib/libclangCodeGen.a lib/libLLVMCoverage.a lib/libLLVMLTO.a lib/libLLVMObjCARCOpts.a lib/libLLVMPasses.a lib/libLLVMCodeGen.a lib/libLLVMTarget.a lib/libLLVMCoroutines.a lib/libLLVMipo.a lib/libLLVMInstrumentation.a lib/libLLVMVectorize.a lib/libLLVMBitWriter.a lib/libLLVMIRReader.a lib/libLLVMAsmParser.a lib/libLLVMLinker.a lib/libLLVMScalarOpts.a lib/libLLVMAggressiveInstCombine.a lib/libLLVMInstCombine.a lib/libclangRewriteFrontend.a lib/libclangARCMigrate.a lib/libclangStaticAnalyzerFrontend.a lib/libclangStaticAnalyzerCheckers.a lib/libclangStaticAnalyzerCore.a lib/libclangCrossTU.a lib/libclangIndex.a lib/libclangFrontend.a lib/libclangDriver.a lib/libLLVMOption.a lib/libclangParse.a lib/libclangSerialization.a lib/libclangSema.a lib/libclangAnalysis.a lib/libclangASTMatchers.a lib/libclangEdit.a lib/libclangFormat.a lib/libclangToolingInclusions.a lib/libclangToolingCore.a lib/libclangAST.a lib/libLLVMFrontendOpenMP.a lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a lib/libLLVMProfileData.a lib/libLLVMObject.a lib/libLLVMMCParser.a lib/libLLVMBitReader.a lib/libLLVMTextAPI.a lib/libclangRewrite.a lib/libclangLex.a lib/libclangBasic.a lib/libLLVMCore.a lib/libLLVMRemarks.a lib/libLLVMBitstreamReader.a lib/libLLVMMC.a lib/libLLVMBinaryFormat.a lib/libLLVMDebugInfoCodeView.a lib/libLLVMDebugInfoMSF.a lib/libLLVMSupport.a -lz -lrt -ldl -ltinfo -lpthread -lm lib/libLLVMDemangle.a && :
ld.lld: error: failed to open bin/clang-11: Cannot allocate memory

I also run into problems with lld on machines with lots of cores (208), but I run into rlimits for number of processes I can create, something outside of my control.

Unfortunately things like numactl or taskset do not limit the concurrency of lld currently.

Alex :

Can you please try numactl or taskset after https://github.com/llvm/llvm-project/commit/09158252f777c2e2f06a86b154c44abcbcf9bb74 ?

There was a tiny bug in how sched_getaffinity() was used, see: https://reviews.llvm.org/D75153#1942336

Envoyé : March 28, 2020 12:11 PM

The discussion in https://reviews.llvm.org/D76240
may be of interest to the participants of this thread. Specifically, Fangrui’s planned work on a command line option to limit threads for LLD. Until that becomes a reality, you might want to use the CMake variable -DLLVM_ENABLE_THREADS=OFF when building the LLD that you are using to prevent parallelism.

Alex :

Can you please try `numactl` or `taskset` after https://github.com/llvm/llvm-project/commit/09158252f777c2e2f06a86b154c44abcbcf9bb74 ?

It works! I can finally use lld on that machine :slight_smile:

Hi,
My configuration is below:
cmake -GNinja -DLLVM_ENABLE_LLD=ON -DLLVM_ENABLE_THREADS=OFF -DLLVM_ENABLE_LIBCXX=ON -DCMAKE_BUILD_TYPE=Release -DGCC_INSTALL_PREFIX=/home/usr4/c74014i/opt/gcc-7.5.0/ -DLIBOMPTARGET_ENABLE_DEBUG=ON -DCMAKE_INSTALL_PREFIX=$HOME/opt/clang/${today} -DCMAKE_C_COMPILER=clang -DCMAKE_C_FLAGS="" -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS="" -DLLVM_ENABLE_PROJECTS=“clang;openmp;lld;libcxx;libcxxabi;compiler-rt;libunwind” -DCLANG_OPENMP_NVPTX_DEFAULT_ARCH=sm_60 -DLIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=60 -DLLVM_TARGETS_TO_BUILD=“all” /tmp/llvm-project/llvm

but did not help my case:

[…]
[4802/5201] Linking CXX executable bin/lld
FAILED: bin/lld
: && /home/usr4/c74014i/opt/clang/current/bin/clang++ -stdlib=libc++ -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -DNDEBUG -stdlib=libc++ -fuse-ld=lld -Wl,–color-diagnostics -Wl,-allow-shlib-undefined -Wl,–export-dynamic -Wl,-O3 tools/lld/tools/lld/CMakeFiles/lld.dir/lld.cpp.o -o bin/lld -Wl,-rpath,"$ORIGIN/…/lib" lib/libLLVMSupport.a -lpthread lib/liblldCommon.a lib/liblldCOFF.a lib/liblldDriver.a lib/liblldELF.a lib/liblldMinGW.a lib/liblldWasm.a lib/liblldMachO.a lib/liblldReaderWriter.a lib/liblldYAML.a lib/liblldCore.a lib/liblldCOFF.a lib/libLLVMDebugInfoPDB.a lib/libLLVMLibDriver.a lib/libLLVMWindowsManifest.a /usr/lib64/libxml2.so lib/liblldCommon.a lib/libLLVMOption.a lib/libLLVMAArch64CodeGen.a lib/libLLVMAArch64AsmParser.a lib/libLLVMAArch64Disassembler.a lib/libLLVMAArch64Desc.a lib/libLLVMAArch64Info.a lib/libLLVMAArch64Utils.a lib/libLLVMAMDGPUCodeGen.a lib/libLLVMMIRParser.a lib/libLLVMAMDGPUAsmParser.a lib/libLLVMAMDGPUDisassembler.a lib/libLLVMAMDGPUDesc.a lib/libLLVMAMDGPUInfo.a lib/libLLVMAMDGPUUtils.a lib/libLLVMARMCodeGen.a lib/libLLVMARMAsmParser.a lib/libLLVMARMDisassembler.a lib/libLLVMARMDesc.a lib/libLLVMARMInfo.a lib/libLLVMARMUtils.a lib/libLLVMAVRCodeGen.a lib/libLLVMAVRAsmParser.a lib/libLLVMAVRDesc.a lib/libLLVMAVRDisassembler.a lib/libLLVMAVRInfo.a lib/libLLVMBPFCodeGen.a lib/libLLVMBPFAsmParser.a lib/libLLVMBPFDesc.a lib/libLLVMBPFDisassembler.a lib/libLLVMBPFInfo.a lib/libLLVMHexagonCodeGen.a lib/libLLVMHexagonAsmParser.a lib/libLLVMHexagonDisassembler.a lib/libLLVMHexagonDesc.a lib/libLLVMHexagonInfo.a lib/libLLVMLanaiCodeGen.a lib/libLLVMLanaiAsmParser.a lib/libLLVMLanaiDisassembler.a lib/libLLVMLanaiDesc.a lib/libLLVMLanaiInfo.a lib/libLLVMMipsCodeGen.a lib/libLLVMMipsAsmParser.a lib/libLLVMMipsDesc.a lib/libLLVMMipsDisassembler.a lib/libLLVMMipsInfo.a lib/libLLVMMSP430CodeGen.a lib/libLLVMMSP430AsmParser.a lib/libLLVMMSP430Desc.a lib/libLLVMMSP430Disassembler.a lib/libLLVMMSP430Info.a lib/libLLVMNVPTXCodeGen.a lib/libLLVMNVPTXDesc.a lib/libLLVMNVPTXInfo.a lib/libLLVMPowerPCCodeGen.a lib/libLLVMPowerPCAsmParser.a lib/libLLVMPowerPCDesc.a lib/libLLVMPowerPCDisassembler.a lib/libLLVMPowerPCInfo.a lib/libLLVMRISCVCodeGen.a lib/libLLVMRISCVAsmParser.a lib/libLLVMRISCVDesc.a lib/libLLVMRISCVDisassembler.a lib/libLLVMRISCVInfo.a lib/libLLVMRISCVUtils.a lib/libLLVMSparcCodeGen.a lib/libLLVMSparcAsmParser.a lib/libLLVMSparcDesc.a lib/libLLVMSparcDisassembler.a lib/libLLVMSparcInfo.a lib/libLLVMSystemZCodeGen.a lib/libLLVMSystemZAsmParser.a lib/libLLVMSystemZDisassembler.a lib/libLLVMSystemZDesc.a lib/libLLVMSystemZInfo.a lib/libLLVMWebAssemblyCodeGen.a lib/libLLVMWebAssemblyAsmParser.a lib/libLLVMWebAssemblyDisassembler.a lib/libLLVMWebAssemblyDesc.a lib/libLLVMWebAssemblyInfo.a lib/libLLVMX86CodeGen.a lib/libLLVMCFGuard.a lib/libLLVMGlobalISel.a lib/libLLVMX86AsmParser.a lib/libLLVMX86Desc.a lib/libLLVMX86Disassembler.a lib/libLLVMX86Info.a lib/libLLVMX86Utils.a lib/libLLVMXCoreCodeGen.a lib/libLLVMAsmPrinter.a lib/libLLVMDebugInfoDWARF.a lib/libLLVMSelectionDAG.a lib/libLLVMXCoreDesc.a lib/libLLVMXCoreDisassembler.a lib/libLLVMMCDisassembler.a lib/libLLVMXCoreInfo.a lib/libLLVMLTO.a lib/libLLVMObjCARCOpts.a lib/libLLVMPasses.a lib/libLLVMCodeGen.a lib/libLLVMTarget.a lib/libLLVMCoroutines.a lib/libLLVMipo.a lib/libLLVMBitWriter.a lib/libLLVMScalarOpts.a lib/libLLVMVectorize.a lib/libLLVMAggressiveInstCombine.a lib/libLLVMInstCombine.a lib/libLLVMLinker.a lib/libLLVMFrontendOpenMP.a lib/libLLVMIRReader.a lib/libLLVMAsmParser.a lib/libLLVMInstrumentation.a lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a lib/libLLVMObject.a lib/libLLVMBitReader.a lib/libLLVMMCParser.a lib/libLLVMMC.a lib/libLLVMDebugInfoCodeView.a lib/libLLVMDebugInfoMSF.a lib/libLLVMTextAPI.a lib/libLLVMProfileData.a lib/libLLVMCore.a lib/libLLVMBinaryFormat.a lib/libLLVMRemarks.a lib/libLLVMBitstreamReader.a lib/libLLVMSupport.a -lz -lrt -ldl -ltinfo -lpthread -lm lib/libLLVMDemangle.a && cd /tmp/build/202003290405/tools/lld/tools/lld && /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E create_symlink lld /tmp/build/202003290405/./bin/lld-link && cd /tmp/build/202003290405/tools/lld/tools/lld && /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E create_symlink lld /tmp/build/202003290405/./bin/ld.lld && cd /tmp/build/202003290405/tools/lld/tools/lld && /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E create_symlink lld /tmp/build/202003290405/./bin/ld64.lld && cd /tmp/build/202003290405/tools/lld/tools/lld && /home/usr4/c74014i/opt/cmake-3.16.3-Linux-x86_64/bin/cmake -E create_symlink lld /tmp/build/202003290405/./bin/wasm-ld
terminating with uncaught exception of type std::__1::system_error: thread constructor failed: Resource temporarily unavailable
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
#0 0x00000000007924f4 PrintStackTraceSignalHandler(void*) (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7924f4)
#1 0x00000000007902ce llvm::sys::RunSignalHandlers() (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x7902ce)
#2 0x0000000000792bd5 SignalHandler(int) (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x792bd5)
#3 0x00007fddd54bb630 __restore_rt (/lib64/libpthread.so.0+0xf630)
#4 0x00007fddd3b40377 raise /usr/src/debug/glibc-2.17-c758a686/signal/…/nptl/sysdeps/unix/sysv/linux/raise.c:55:0
#5 0x00007fddd3b41a68 abort /usr/src/debug/glibc-2.17-c758a686/stdlib/abort.c:92:0
#6 0x00007fddd411ae8b (/home/usr4/c74014i/opt/clang/202003290200/bin/…/lib/libc++abi.so.1+0x2be8b)
#7 0x00007fddd40fdb9f demangling_terminate_handler() (/home/usr4/c74014i/opt/clang/202003290200/bin/…/lib/libc++abi.so.1+0xeb9f)
#8 0x00007fddd411a0a3 std::__terminate(void ()()) (/home/usr4/c74014i/opt/clang/202003290200/bin/…/lib/libc++abi.so.1+0x2b0a3)
#9 0x00007fddd411ce06 (/home/usr4/c74014i/opt/clang/202003290200/bin/…/lib/libc++abi.so.1+0x2de06)
#10 0x00007fddd411cd9f (/home/usr4/c74014i/opt/clang/202003290200/bin/…/lib/libc++abi.so.1+0x2dd9f)
#11 0x00007fddd43bded4 std::__1::__throw_system_error(int, char const
) (/home/usr4/c74014i/opt/clang/202003290200/bin/…/lib/libc++.so.1+0x90ed4)
#12 0x0000000002a7a316 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_deletestd::__1::__thread_struct >, llvm::parallel::detail::(anonymous namespace)::ThreadPoolExecutor::ThreadPoolExecutor(llvm::ThreadPoolStrategy)::‘lambda’()> >(void*) (/home/usr4/c74014i/opt/clang/current/bin/ld.lld+0x2a7a316)
#13 0x00007fddd54b3ea5 start_thread /usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:307:0
#14 0x00007fddd3c088cd clone /usr/src/debug////////glibc-2.17-c758a686/misc/…/sysdeps/unix/sysv/linux/x86_64/clone.S:113:0
clang-11: error: unable to execute command: Aborted
clang-11: error: linker command failed due to signal (use -v to see invocation)

Enable threads is for building llvm with or without thread support. To my knowledge it has no effect on the build system.

Like was discussed above you could try limiting the number of processes created (assuming your lld was after that commit) with numactl or taskset.

In your case though it looked like you were running out of memory. You could try increasing your swap space?

Alex

$ free -g
total used free shared buff/cache available
Mem: 376 149 20 1 207 225
Swap: 3 0 3

Too small swap size for a 72-core login machine?

As I am not the admin, can’t change the swap size.

Would taskset -c 0-3 ninja check-all -j4 work?

Envoyé : March 28, 2020 3:37 PM

That is slowing down the build visibly, for the speed I should stick with ld at the moment.

Yes unfortunately that would limit you to 4 cores.

There’s no easy way to use lld with —no-threads with our build system at the moment unfortunately.

I also had just been using ld.gold, but will switch now that numactl works for me.

Good news, I was able to use up to 37 cores for LLVM build with LLD.
The build speed, did not measure precisely though, is comparable to the build with GNU ld case.

Thank you all for your help!

Glad you got it working.
My suggestion about LLVM_ENABLE_THREADS didn’t work because you didn’t apply it when building the build linker.

When you don’t have the ability to rebuild the build compiler, this doesn’t apply. In those cases, I end up doing a dirty hack where I use a wrapper script with something like:

“$@” -Wl,–no-threads

To invoke the build compiler. That way you still get full parallelism while compiling and linking, but you prevent LLD from using threads.

Glad you got it working.
My suggestion about LLVM_ENABLE_THREADS didn't work because you didn't
apply it when building the build linker.

When you don't have the ability to rebuild the build compiler, this doesn't
apply. In those cases, I end up doing a dirty hack where I use a wrapper
script with something like:

<path-to-build-compiler> "$@" -Wl,--no-threads

To invoke the build compiler. That way you still get full parallelism while
compiling and linking, but you prevent LLD from using threads.

⚙ D76885 [lld][COFF][ELF][WebAssembly] Replace --[no-]threads /threads[:no] with --threads={1,2,...} /threads:{1,2,...} (committed yesterday) changed
--no-threads to --threads={1,2,...} (--no-threads is used rarely (GNU ld
does not support it) so I'd rather not keep the option for
compatibility).

You may try -Wl,--threads=1

Thanks for the heads up the supercomputer

Is down for maintenance this week.
I’ll give it a try when it gets back.

Thanks for the heads up the supercomputer
Is down for maintenance this week.
I’ll give it a try when it gets back.

This doesn't look to me like it's necessarily an lld issue. Trying
to build all the sub-projects without limiting the number of ninja jobs
is almost guaranteed to run out of memory on a machine like this with such
a low memory:core ratio.

-Tom

Tom,

Then what ratio do you think it’s minimal?

On another login node which is 256 (GB)/48 (nodes) JURECA at JSC, I never had an LLD issue without setting -j when executing ninja
in the past few weeks.

On another login node which is 256 (GB)/48 (nodes) JURECA at JSC, I never had an LLD issue without setting -j when executing ninja
in the past few weeks.

    Tom,
    Then what ratio do you think it’s minimal?

It really depends on your configuration, but I usually try to have at least 2 GB
of memory per core. However, I usually do Release builds, so Debug builds might
need more. If you aren't using LLVM_PARALLEL_LINK_JOBS it's pretty easy to
run out of memory once ninja starts linking the tools and unittests.

-Tom