Help with LLVM standalone cross-compiler build

Hello everyone,

I am trying to build LLVM project for x86 (host), Sparc and RISCV. I have, apparently done that successfully with:
cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/home/march/opt/llvm -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld;openmp" -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libunwind" -DLLVM_TARGETS_TO_BUILD="X86;Sparc;RISCV" -DLLVM_INSTALL_UTILS=ON ../llvm

I say apparently because after building I can successfully use clang hello.c, but when I try to do the same for a different target i.e. clang hello.c --target=riscv32 I encounter two errors:

  • stdio.h file not found: I can solve this by running clang hello.c --target=riscv32 -I/home/march/opt/gcc-riscv-gaisler/riscv-gaisler-elf/include
  • ld.lld command failed: This can be solved by doing the compilation manually for each step, and then perform the linking using /home/march/opt/gcc-riscv-gaisler/bin/riscv-gaisler-elf-gcc

In both cases I depend on a GCC implementation, which, although allows me to generate code is not a permanent solution.

What do I need to do in order for my LLVM installation to be not dependent on a GCC implementation for each architecture?

Thank you all for your help!

Marc

1 Like

The short answer is build a sysroot using your bare-metal libc of choice (newlib is a common one) and point Clang at it with --sysroot. For your stdio.h workaround you’re not using GCC for anything other than the sysroot it provides, and for the ld.lld workaround, which is presumably because you have no standard library (and CRT) files to link against, you’re using GCC both to invoke BFD (which should not be necessary) and to get the files you need. I imagine clang --sysroot=/home/march/opt/gcc-riscv-gaisler/riscv-gaisler-elf would resolve both your issues and demonstrate that it’s just a sysroot you need (and if you do have errors with that, please post them here so we can actually help without guessing; “ld.lld command failed” is not very helpful).

Nit: openmp belongs into runtimes. It needs a fresh clang.

Hi, thanks for your answer. I did as you said and the ld.lld error persists.

march@MARC-PC:/tmp$ clang hello.c --sysroot=/home/march/opt/gcc-riscv-gaisler/riscv-gaisler-elf --target=riscv32 -v
clang version 16.0.0 (https://gitlab.bsc.es/msolebon/llvm-sparrow.git 5f759beaae171d39c04040f6503244e2b7cd1138)
Target: riscv32
Thread model: posix
InstalledDir: /home/march/opt/llvm/bin
“/mnt/wsl/PHYSICALDRIVE2p3/march/opt/llvm/bin/clang-16” -cc1 -triple riscv32 -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name hello.c -mrelocation-model static -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -fno-verbose-asm -mconstructor-aliases -nostdsysteminc -target-cpu generic-rv32 -target-feature +m -target-feature +a -target-feature +c -target-feature +relax -target-feature -save-restore -target-abi ilp32 -msmall-data-limit 8 -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -v -fcoverage-compilation-dir=/tmp -resource-dir /mnt/wsl/PHYSICALDRIVE2p3/march/opt/llvm/lib/clang/16 -isysroot /home/march/opt/gcc-riscv-gaisler/riscv-gaisler-elf -I/opt/cuda/targets/x86_64-linux/include/ -I/usr/include/pccts/ -I. -internal-isystem /mnt/wsl/PHYSICALDRIVE2p3/march/opt/llvm/lib/clang/16/include -internal-isystem /home/march/opt/gcc-riscv-gaisler/riscv-gaisler-elf/include -fdebug-compilation-dir=/tmp -ferror-limit 19 -fno-signed-char -fgnuc-version=4.2.1 -fcolor-diagnostics -faddrsig -o /tmp/hello-49b2b5.o -x c hello.c
clang -cc1 version 16.0.0 based upon LLVM 16.0.0git default target x86_64-unknown-linux-gnu
ignoring nonexistent directory “/opt/cuda/targets/x86_64-linux/include/”
ignoring nonexistent directory “/usr/include/pccts/”
ignoring duplicate directory “/mnt/wsl/PHYSICALDRIVE2p3/march/opt/llvm/lib/clang/16/include”
#include “…” search starts here:
#include <…> search starts here:
.
/mnt/wsl/PHYSICALDRIVE2p3/march/opt/llvm/lib/clang/16/include
/home/march/opt/gcc-riscv-gaisler/riscv-gaisler-elf/include
End of search list.
“/home/march/opt/llvm/bin/ld.lld” /tmp/hello-49b2b5.o -Bstatic -L/home/march/opt/gcc-riscv-gaisler/riscv-gaisler-elf/lib -L/mnt/wsl/PHYSICALDRIVE2p3/march/opt/llvm/lib/clang/16/lib/baremetal -lc -lm -lclang_rt.builtins-riscv32 -o a.out
ld.lld: error: unable to find library -lclang_rt.builtins-riscv32
clang-16: error: ld.lld command failed with exit code 1 (use -v to see invocation)

I also tested for Sparc and in this case not even the stdio.h error was solved.
Here I see that is searching in /home/march/opt/gcc-sparc-gaisler/sparc-gaisler-elf/usr/include so I guess this can be easily solved.

march@MARC-PC:/tmp$ clang hello.c --sysroot=/home/march/opt/gcc-sparc-gaisler/sparc-gaisler-elf --target=sparc -v
clang version 16.0.0 (https://gitlab.bsc.es/msolebon/llvm-sparrow.git 5f759beaae171d39c04040f6503244e2b7cd1138)
Target: sparc
Thread model: posix
InstalledDir: /home/march/opt/llvm/bin
“/mnt/wsl/PHYSICALDRIVE2p3/march/opt/llvm/bin/clang-16” -cc1 -triple sparc -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name hello.c -mrelocation-model static -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -mfloat-abi hard -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -v -fcoverage-compilation-dir=/tmp -resource-dir /mnt/wsl/PHYSICALDRIVE2p3/march/opt/llvm/lib/clang/16 -isysroot /home/march/opt/gcc-sparc-gaisler/sparc-gaisler-elf -I/opt/cuda/targets/x86_64-linux/include/ -I/usr/include/pccts/ -I. -fdebug-compilation-dir=/tmp -ferror-limit 19 -fgnuc-version=4.2.1 -fcolor-diagnostics -faddrsig -o /tmp/hello-3460f0.o -x c hello.c
clang -cc1 version 16.0.0 based upon LLVM 16.0.0git default target x86_64-unknown-linux-gnu
ignoring nonexistent directory “/opt/cuda/targets/x86_64-linux/include/”
ignoring nonexistent directory “/usr/include/pccts/”
ignoring nonexistent directory “/home/march/opt/gcc-sparc-gaisler/sparc-gaisler-elf/usr/local/include”
ignoring nonexistent directory “/home/march/opt/gcc-sparc-gaisler/sparc-gaisler-elf/usr/include”
#include “…” search starts here:
#include <…> search starts here:
.
/mnt/wsl/PHYSICALDRIVE2p3/march/opt/llvm/lib/clang/16/include
End of search list.
hello.c:1:9: fatal error: ‘stdio.h’ file not found
#include<stdio.h>
^~~~~~~~~
1 error generated.

On the other hand when using -I/home/march/opt/gcc-sparc-gaisler/sparc-gaisler-elf/include it looks like it’s just calling the host gcc.

march@MARC-PC:/tmp$ clang hello.c -I/home/march/opt/gcc-sparc-gaisler/sparc-gaisler-elf/include --target=sparc -v
clang version 16.0.0 (https://gitlab.bsc.es/msolebon/llvm-sparrow.git 5f759beaae171d39c04040f6503244e2b7cd1138)
Target: sparc
Thread model: posix
InstalledDir: /home/march/opt/llvm/bin
“/mnt/wsl/PHYSICALDRIVE2p3/march/opt/llvm/bin/clang-16” -cc1 -triple sparc -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name hello.c -mrelocation-model static -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -mfloat-abi hard -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -v -fcoverage-compilation-dir=/tmp -resource-dir /mnt/wsl/PHYSICALDRIVE2p3/march/opt/llvm/lib/clang/16 -I /home/march/opt/gcc-sparc-gaisler/sparc-gaisler-elf/include -I/opt/cuda/targets/x86_64-linux/include/ -I/usr/include/pccts/ -I. -fdebug-compilation-dir=/tmp -ferror-limit 19 -fgnuc-version=4.2.1 -fcolor-diagnostics -faddrsig -o /tmp/hello-27c8b1.o -x c hello.c
clang -cc1 version 16.0.0 based upon LLVM 16.0.0git default target x86_64-unknown-linux-gnu
ignoring nonexistent directory “/opt/cuda/targets/x86_64-linux/include/”
ignoring nonexistent directory “/usr/include/pccts/”
#include “…” search starts here:
#include <…> search starts here:
/home/march/opt/gcc-sparc-gaisler/sparc-gaisler-elf/include
.
/usr/local/include
/mnt/wsl/PHYSICALDRIVE2p3/march/opt/llvm/lib/clang/16/include
/usr/include
End of search list.
“/usr/bin/gcc” -o a.out /tmp/hello-27c8b1.o
/usr/bin/ld: /tmp/hello-27c8b1.o: relocations in generic ELF (EM: 2)
/usr/bin/ld: /tmp/hello-27c8b1.o: error adding symbols: file in wrong format
collect2: error: ld returned 1 exit status
clang-16: error: linker (via gcc) command failed with exit code 1 (use -v to see invocation)

Thanks again for your help
Marc

SPARC and RISC-V likely have different defaults here. LLD doesn’t even support pre-v9 SPARC so it’s always going to have to use GNU tools for that. For RISC-V you’re getting a Clang that wants to use compiler-rt by default; either build a compiler-rt for it or use --rtlib=libgcc (I think that’s the option). For SPARC some of the behaviour comes from the fact that bare-metal toolchains aren’t really supported - BareMetal::handlesTarget only accepts Arm (32-bit and 64-bit) and RISC-V - so it’s falling back on generic ELF which behaves a bit differently and is more of a “generic but unknown ELF-using OS” driver, I believe.

1 Like

What’s the intended target? Baremetal, Linux, or some other OS? And which libc would you like to use? This is important to give a useful answer.

If Linux, you can build your own sysroot using compiler-rt and musl to build static executables (this is slightly involved but possible). For baremetal there are other libc variants, my preferred one is picolibc. If you target a particular Linux distribution you could also obtain a sysroot by downloading a rootfs (via LXC, Docker, or some other way) and using it in --sysroot.
In any case, you probably need to use -fuse-ld=lld to avoid linking with GNU ld.

The important thing to know is that the libc (and therefore stdio.h) is not part of the LLVM build. You have to supply one yourself if you’re cross compiling.

1 Like

Hi,

The intended target is baremetal.

I have been able to link using lld for RISC-V. Regarding linking with the appropriate libraries I see there is no current way to do that with just LLVM.
I have found this video on how to use llvm libc, although the support is not completed it may be of help for someone. However there is no support for RISC-V (link)

Thank you all for your time!