we have clang 3.6.2 on ubuntu 16.04 to build our project and it works well on physical machines. but when running in VM (HyperV on Windows Server 2012R2), it crashes quite often in linking. Is this a known issue? It seems possibly related to memory, as the crash happens more frequently when the memory size of the VM is relatively small. Thanks.
clang-3.6: error: unable to execute command: Killed
clang-3.6: error: clang frontend command failed due to signal (use -v to see invocation)
clang version 3.6.2 (tags/RELEASE_362/final 252038)
Thread model: posix
clang-3.6: note: diagnostic msg: PLEASE submit a bug report to http://llvm.org/bugs/ and include the crash backtrace, preprocessed source, and associated run script.
"error: unable to execute command: Killed" sounds a lot like what I
get when clang gets killed for eating all of my system's memory. Are
you sure that your VM has enough RAM to handle linking whatever it is
that you're linking?
Thanks for replying, George.
I allocated 16GB to the VM, as well as 4GB swap. Our executable/shared-library are pretty big, usually 500MB to 2GB in size with debug symbols, and we use “make -j 12” for the 6-core (12 hyperthread) CPU to be fully used. I guess it might be due to parallel linking, the memory usage would skyrocket? Is there a graceful way to avoid this?
I guess it might be due to parallel linking, the memory usage would skyrocket?
Yup, that would do it.
Is there a graceful way to avoid this?
If your project uses CMake, you might be able to have it generate
files for `ninja` instead of Makefiles. Ninja has job pools just for
cases like this: The Ninja build system (and
I've heard many LLVM devs talk about how using `ninja` instead of
`make` made incremental rebuilds of LLVM/clang faster for them, so...)
If you're stuck with Makefiles, though, I don't know of a great way to
limit to N parallel link jobs. If you don't mind linking being 100%
serial, you may find the (nonstandard, apparently) .NOTPARALLEL
There’s a CMake option LLVM_PARALLEL_LINK_JOBS that you could set to something smaller than 12, try experimenting with that.
For N=1, you can limit the number of parallel linker processes by
making the linker command
flock /tmp/linker.lock gcc
instead of gcc (or whatever you use to link)
Then the rest of the build can go in parallel.