Can't build LLVM 14.0.6 on Linux

Beginners
1

I upgraded LLVM for my targets from 3.4.3 to 13.0.0 some time ago and this worked fine on Linux and macOS. Now I want to upgrade to the latest version LLVM 14.0.6. I had no problems building LLVM on macOS during upgrading. But I can’t build LLVM 14.0.6 on Linux due to linking errors within LLVM. Even if LLVM comes straight from the archive without my changes I get these errors.

> cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug -DLLVM_TARGETS_TO_BUILD="ARM" -DLLVM_ENABLE_DUMP=ON -DLLVM_ENABLE_CRASH_DUMPS=ON -DLLVM_BUILD_TOOLS=OFF -DLLVM_BUILD_UTILS=OFF -DLLVM_BUILD_RUNTIMES=OFF -DLLVM_INCLUDE_TESTS=OFF -DLLVM_INCLUDE_GO_TESTS=OFF -DLLVM_INCLUDE_BENCHMARKS=OFF -DCMAKE_INSTALL_PREFIX=$PWD/install $MY_BASE_DIR/llvm-14.0.6.src/
> ninja -j 2
[1706/1811] Linking CXX shared library lib/libLTO.so.14
FAILED: : && /usr/bin/c++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -g  -Wl,-z,defs -Wl,-z,nodelete   -Wl,-rpath-link,/home/boesler/projekte/vid_vmp_llvm-extension/build-server-arm/./lib   -Wl,--version-script,"$MY_BASE_DIR/build-server-arm/tools/lto/LTO.exports" -shared -Wl,-soname,libLTO.so.14 -o lib/libLTO.so.14 tools/lto/CMakeFiles/LTO.dir/LTODisassembler.cpp.o tools/lto/CMakeFiles/LTO.dir/lto.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVMARMAsmParser.a  lib/libLLVMARMCodeGen.a  lib/libLLVMARMDesc.a  lib/libLLVMARMDisassembler.a  lib/libLLVMARMInfo.a  lib/libLLVMBitReader.a  lib/libLLVMCore.a  lib/libLLVMCodeGen.a  lib/libLLVMLTO.a  lib/libLLVMMC.a  lib/libLLVMMCDisassembler.a  lib/libLLVMSupport.a  lib/libLLVMTarget.a  lib/libLLVMAsmPrinter.a  lib/libLLVMDebugInfoMSF.a  lib/libLLVMGlobalISel.a  lib/libLLVMSelectionDAG.a  lib/libLLVMCFGuard.a  lib/libLLVMARMDesc.a  lib/libLLVMARMInfo.a  lib/libLLVMMCDisassembler.a  lib/libLLVMARMUtils.a  lib/libLLVMCodeGen.a  lib/libLLVMExtensions.a  lib/libLLVMPasses.a  lib/libLLVMTarget.a  lib/libLLVMObjCARCOpts.a  lib/libLLVMCoroutines.a  lib/libLLVMipo.a  lib/libLLVMBitWriter.a  lib/libLLVMInstrumentation.a  lib/libLLVMLinker.a  lib/libLLVMFrontendOpenMP.a  lib/libLLVMIRReader.a  lib/libLLVMAsmParser.a  lib/libLLVMVectorize.a  lib/libLLVMScalarOpts.a  lib/libLLVMAggressiveInstCombine.a  lib/libLLVMInstCombine.a  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a  lib/libLLVMProfileData.a  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMObject.a  lib/libLLVMBitReader.a  lib/libLLVMCore.a  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a  lib/libLLVMMCParser.a  lib/libLLVMMC.a  lib/libLLVMDebugInfoCodeView.a  lib/libLLVMTextAPI.a  lib/libLLVMBinaryFormat.a  lib/libLLVMSupport.a  -lrt  -ldl  -lpthread  -lm  /usr/lib/x86_64-linux-gnu/libz.so  /usr/lib/x86_64-linux-gnu/libtinfo.so  lib/libLLVMDemangle.a && :
lib/libLLVMCodeGen.a(MachineDominators.cpp.o): In function `llvm::MachineDominatorTree::calculate(llvm::MachineFunction&)':
$MY_BASE_DIR/llvm-14.0.6.src/lib/CodeGen/MachineDominators.cpp:60: undefined reference to `llvm::DominatorTreeBase<llvm::MachineBasicBlock, false>::DominatorTreeBase()'
...

I’m out of ideas. What do I not see?
Any hints?
Thanks,
Boris

2

Well, I can build release builds (-DCMAKE_BUILD_TYPE=Release).

3

It seems caused by memory exhaustion for the build failure, the release build need less memory than debug build. You can try “ninja -j1”.

4

Could also set CMake option -DLLVM_PARALLEL_LINK_JOBS=1 so that you still get parallel compiles but only one link job at a time.

1 Like
5

Thanks BaoshanPang, I tried, but that didn’t work. I’m working on a 128GB machine.
Linking debug builds worked up to LLVM 13.0.0. I don’t know about LLVM 13.0.1. I’m getting linking errors since LLVM 14.0.0.

Do you build debug builds on Linux?

6

Thanks pogo59. I haven’t seen this option. I tried, but that didn’t work.
Linking debug builds worked up to LLVM 13.0.0. I don’t know about LLVM 13.0.1. I’m getting linking errors since LLVM 14.0.0.

Do you build debug builds on Linux?

7

On Linux I typically do a Release build, with -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_C_FLAGS=-g1 -DCMAKE_CXX_FLAGS=-g1 so that crash tracebacks have symbolized names and source locations; this gives me faster builds and smaller memory consumption during links (I don’t usually use a debugger). On Ubuntu I used System Monitor’s Resources tab to watch memory consumption during a build, and picked a value for LLVM_PARALLEL_LINK_JOBS that would make sure my builds didn’t fill up memory. With my setup, links peak at around 6GB. For a Debug build it would surely be more, but I don’t have a number to suggest.

8

You could also try either using a different linker (gold or LLD, or even mold?) as opposed to the stock GNU LD, and also there is an option called -DBUILD_SHARED_LIBS=ON which massively reduces the memory use in the link step.

Are you sure you only want to build the LLVM backend, and no frontends like Clang or Flang?

I have done a test. On

commit f28c006a5895fc0e329fe15fead81e37457cb1d1 (HEAD → release/14.x, tag: llvmorg-14.0.6, upstream/release/14.x)
Author: Tom Stellard
Date: Wed Jun 22 09:46:24 2022 -0700

Bump version to 14.0.6

I executed your command (with minor following changes, highlighted): cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug -DLLVM_TARGETS_TO_BUILD="ARM" -DLLVM_ENABLE_DUMP=ON -DLLVM_ENABLE_CRASH_DUMPS=ON -DLLVM_BUILD_TOOLS=OFF -DLLVM_BUILD_UTILS=OFF -DLLVM_BUILD_RUNTIM ES=OFF -DLLVM_INCLUDE_TESTS=OFF -DLLVM_INCLUDE_GO_TESTS=OFF -DLLVM_INCLUDE_BENCHMARKS=OFF -DCMAKE_INSTALL_PREFIX=$PWD/install -DLLVM_PARALLEL_LINK_JOBS=2 ../llvm and then ninja -j 12

Toolchain that I have configured (which is the stock available on the system):

  • x86_64
  • 16 logical threads, 64 GiB memory
  • ninja 1.10.2
  • g++ (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
  • GNU ld (GNU Binutils for Ubuntu) 2.30

There were 1811 actions executed by ninja. It took roundabout 10 minutes to execute, but it went through successfully. Memory use was about 600 MiB per compiler invocation. I was able to succeed, and the linked file contains the referred symbol:

[1811/1811] Linking CXX executable bin/llvm-yaml-parser-fuzzer

readelf --wide --syms lib/libLLVMCodeGen.a | grep "FUNC" | grep "WEAK" | awk '{ print $8; }' | grep "DominatorTreeBase" | c++filt | grep "MachineBasicBlock, false" | grep "::DominatorTree
Base()" | sort | uniq -c
     16 llvm::DominatorTreeBase<llvm::MachineBasicBlock, false>::DominatorTreeBase()
1 Like
9

@borisboesler I didn’t see the error message last time, and with this error message:
undefined reference to `llvm::DominatorTreeBase

I would suggest you check why this function is not found in the library.

10

Usually I use clang as frontend. But I wanted to keep the test as small as possible.

I use the same command and it fails again:

[1759/1811] Linking CXX shared library lib/libLTO.so.14
FAILED: : && /usr/bin/c++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -g  -Wl,-z,defs -Wl,-z,nodelete   -Wl,-rpath-link,/data/tmp/boesler/vid_vmp_llvm-extension/build-server-debug-arm/./lib   -Wl,--version-script,"/data/tmp/boesler/vid_vmp_llvm-extension/build-server-debug-arm/tools/lto/LTO.exports" -shared -Wl,-soname,libLTO.so.14 -o lib/libLTO.so.14 tools/lto/CMakeFiles/LTO.dir/LTODisassembler.cpp.o tools/lto/CMakeFiles/LTO.dir/lto.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVMARMAsmParser.a  lib/libLLVMARMCodeGen.a  lib/libLLVMARMDesc.a  lib/libLLVMARMDisassembler.a  lib/libLLVMARMInfo.a  lib/libLLVMBitReader.a  lib/libLLVMCore.a  lib/libLLVMCodeGen.a  lib/libLLVMLTO.a  lib/libLLVMMC.a  lib/libLLVMMCDisassembler.a  lib/libLLVMSupport.a  lib/libLLVMTarget.a  lib/libLLVMAsmPrinter.a  lib/libLLVMDebugInfoMSF.a  lib/libLLVMGlobalISel.a  lib/libLLVMSelectionDAG.a  lib/libLLVMCFGuard.a  lib/libLLVMARMDesc.a  lib/libLLVMARMInfo.a  lib/libLLVMMCDisassembler.a  lib/libLLVMARMUtils.a  lib/libLLVMCodeGen.a  lib/libLLVMExtensions.a  lib/libLLVMPasses.a  lib/libLLVMTarget.a  lib/libLLVMObjCARCOpts.a  lib/libLLVMCoroutines.a  lib/libLLVMipo.a  lib/libLLVMBitWriter.a  lib/libLLVMInstrumentation.a  lib/libLLVMLinker.a  lib/libLLVMFrontendOpenMP.a  lib/libLLVMIRReader.a  lib/libLLVMAsmParser.a  lib/libLLVMVectorize.a  lib/libLLVMScalarOpts.a  lib/libLLVMAggressiveInstCombine.a  lib/libLLVMInstCombine.a  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a  lib/libLLVMProfileData.a  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMObject.a  lib/libLLVMBitReader.a  lib/libLLVMCore.a  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a  lib/libLLVMMCParser.a  lib/libLLVMMC.a  lib/libLLVMDebugInfoCodeView.a  lib/libLLVMTextAPI.a  lib/libLLVMBinaryFormat.a  lib/libLLVMSupport.a  -lrt  -ldl  -lpthread  -lm  /usr/lib/x86_64-linux-gnu/libz.so  /usr/lib/x86_64-linux-gnu/libtinfo.so  lib/libLLVMDemangle.a && :
lib/libLLVMCodeGen.a(MachineDominators.cpp.o): In function `llvm::MachineDominatorTree::calculate(llvm::MachineFunction&)':
/home/boesler/projekte/vid_vmp_llvm-extension/llvm-14.0.6.src/lib/CodeGen/MachineDominators.cpp:60: undefined reference to `llvm::DominatorTreeBase<llvm::MachineBasicBlock, false>::DominatorTreeBase()'

My toolchain:

  • x84_64
  • 12x Intel(R) Xeon(R) CPU E5-1650 v4 @ 3.60GHz
  • MemTotal: 131916292 kB
  • ninja 1.5.1
  • g++ (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609
  • GNU ld (GNU Binutils for Ubuntu) 2.26.1

This command doesn’t produce any result on the unfinished DEBUG build (obviously).
This command doesn’t produce any result on the finished RELEASE build. Does this make sense?

I tried:

[1694/1887] Linking CXX shared library lib/libLLVMAnalysis.so.14
FAILED: : && /usr/bin/c++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -g  -Wl,-z,defs -Wl,-z,nodelete 
...
lib/Analysis/CMakeFiles/LLVMAnalysis.dir/BranchProbabilityInfo.cpp.o: In function `llvm::PostDominatorTree::PostDominatorTree(llvm::Function&)':
/home/boesler/projekte/vid_vmp_llvm-extension/llvm-14.0.6.src/include/llvm/Analysis/PostDominators.h:33: undefined reference to `llvm::DominatorTreeBase<llvm::BasicBlock, true>::DominatorTreeBase()'
lib/Analysis/CMakeFiles/LLVMAnalysis.dir/PostDominators.cpp.o: In function `llvm::PostDominatorTree::PostDominatorTree()':
/home/boesler/projekte/vid_vmp_llvm-extension/llvm-14.0.6.src/include/llvm/Analysis/PostDominators.h:32: undefined reference to `llvm::DominatorTreeBase<llvm::BasicBlock, true>::DominatorTreeBase()'

I guess the toolchain is to old/buggy.
But I can live with Release builds on this machine.

Thank to all of you.

11

GCC 5 is old and won’t be supported at all in LLVM 15.x but since you can get a release build of clang done you can use that as your bootstrap compiler.

12

@borisboesler Hi, I have the same issue as yours.
With ubuntu 16.04, I can also build Debug/Release verison for llvm 13.0.1, but could not sucess with llvm 14.0.6.
have you solved the problem?

13

Update:20220907
After changing Ubuntu to 20.04, I can build LLVM 14.0.6 debug/Release.
gcc version: 9.4.0