I found that after moving to Clang pre-3.6 (git
65d8b4c4998b3a0c20934ea72ede72ef4838a004) and trunk (git
718825a8666acd9ceaab70fc7868332f20e2758f) our internal build machines started
going offline in Jenkins. Clang after 3.5 release is consuming extreme amounts
of memory in some cases.
I could reproduce this locally with clang 3.5.1 from the ArchLinux packages. I
ran it through my heaptrack [1] tool until ~630MB peak memory where consumed,
which should give enough of an insight already. The full report can be found
at [2]. Here is an excerpt:
7137759 calls with 475.81MB peak consumption from:
0x7f83d81ff4ef
in /usr/bin/../lib/libLLVM-3.5.so
llvm::LazyValueInfo::getConstant(llvm::Value*, llvm::BasicBlock*)
in /usr/bin/../lib/libLLVM-3.5.so
0x7f83d80ba72f
in /usr/bin/../lib/libLLVM-3.5.so
llvm::FPPassManager::runOnFunction(llvm::Function&)
in /usr/bin/../lib/libLLVM-3.5.so
0x7f83d78df9c3
in /usr/bin/../lib/libLLVM-3.5.so
llvm::legacy::PassManagerImpl::run(llvm::Module&)
in /usr/bin/../lib/libLLVM-3.5.so
clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::CodeGenOptions
const&, clang::TargetOptions const&, clang::LangOptions const&,
llvm::StringRef, llvm::Module*, clang::BackendAction, llvm::raw_ostream*)
in /usr/bin/clang
0x8735a1
in /usr/bin/clang
clang::ParseAST(clang::Sema&, bool, bool)
in /usr/bin/clang
clang::CodeGenAction::ExecuteAction()
in /usr/bin/clang
clang::FrontendAction::Execute()
in /usr/bin/clang
clang::CompilerInstance::ExecuteAction(clang::FrontendAction&)
in /usr/bin/clang
clang::ExecuteCompilerInvocation(clang::CompilerInstance*)
in /usr/bin/clang
cc1_main(char const**, char const**, char const*, void*)
in /usr/bin/clang
main
in /usr/bin/clang
It seems that we always disabled optimizer on some files for Clang and
I dropped this particular patch from our code base.
I tried to bisect Clang and went as far as 3.3 LLVM/Clang release with
the same memory consumption.
I moved to Fedora 21 based VM. GCC 4.9.2 used <500MB, and Clang 3.3 till
trunk above 10GB until VM hanged.
We didn't use --disable-assertions. Clang still has a number of issue
with our code base (incl. crashing/asserting). pre-3.6 Clang improves
(no more aborting), but some issues remain. For those I filled bug reports.
Yet to test the current 3.6 RC1 with our code base to see if it regressed
or not.
I have seen a number of cases in pkgsrc, where the Value Propagation
pass results in a very high CPU and memory use. Attached is a hack that
allows disabling the pass via "-mllvm -disable-value-propagation". It
seems you are hitting the same issue.