Error encountered while compiling clang and OpenMP for RISC-V target

Hello,

I’m sorry if this isn’t the correct place to ask such questions, please direct me to a more suitable place. Apologies and thanks in advance.

I’m trying to compile LLVM and OpenMP runtime for RISC-V, however right at the end of the entire process I’m encountering build errors. It seems that since I’m using the -DLLVM_ENABLES_RUNTIMES=openmp flag to build it for RISC-V, at the end of the build process the just built clang is missing the --sysroot and --gcc-toolchain. I say this because the error I encounter is just a missing header file error. So, I copied the compile command which fails, and removed unnecessary flags to build a test file I got the same error, which I fixed by adding the --sysroot and --gcc-toolchain flags.

LLVM CMAKE Command

cmake -G Ninja -DCMAKE_BUILD_TYPE="Release" \
  -DBUILD_SHARED_LIBS=True -DLLVM_USE_SPLIT_DWARF=True \
  -DCMAKE_INSTALL_PREFIX="/home/rahat/mphil-project/local/installs/riscv-llvm-lgnu" \
  -DLLVM_OPTIMIZED_TABLEGEN=True -DLLVM_BUILD_TESTS=False \
  -DDEFAULT_SYSROOT="/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot" \
  -DLLVM_DEFAULT_TARGET_TRIPLE="riscv64-unknown-linux-gnu" \
  -DLLVM_TARGETS_TO_BUILD="RISCV" \
  -DGCC_INSTALL_PREFIX="/home/rahat/mphil-project/local/install/rv-gnu-tc-linux" \
  -DLLVM_ENABLE_PROJECTS="clang" -DLLVM_ENABLE_RUNTIMES=openmp \
  -DCMAKE_CXX_COMPILER=g++ \
  -DCMAKE_C_COMPILER=gcc \
  -DLLVM_PARALLEL_COMPILE_JOBS=16 -DLLVM_PARALLEL_LINK_JOBS=8 ../llvm

LLVM build error

FAILED: openmp/runtime/src/CMakeFiles/omp.dir/kmp_debug.cpp.o 
/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/clang++ --target=riscv64-unknown-linux-gnu -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Domp_EXPORTS -I/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -I/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/openmp/runtime/src -I/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/openmp/runtime/src/i18n -I/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/openmp/runtime/src/include -I/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/openmp/runtime/src/thirdparty/ittnotify -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -O3 -DNDEBUG -fPIC   -D _GNU_SOURCE -D _REENTRANT -MD -MT openmp/runtime/src/CMakeFiles/omp.dir/kmp_debug.cpp.o -MF openmp/runtime/src/CMakeFiles/omp.dir/kmp_debug.cpp.o.d -o openmp/runtime/src/CMakeFiles/omp.dir/kmp_debug.cpp.o -c /home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/openmp/runtime/src/kmp_debug.cpp
In file included from /home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/openmp/runtime/src/kmp_debug.cpp:13:
/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/openmp/runtime/src/kmp.h:73:10: fatal error: 'signal.h' file not found
   73 | #include <signal.h>
      |          ^~~~~~~~~~
1 error generated.
[2/33] Building CXX object openmp/runtime/src/CMakeFiles/omp.dir/kmp_csupport.cpp.o
FAILED: openmp/runtime/src/CMakeFiles/omp.dir/kmp_csupport.cpp.o 
/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/clang++ --target=riscv64-unknown-linux-gnu -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Domp_EXPORTS -I/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src -I/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/openmp/runtime/src -I/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/openmp/runtime/src/i18n -I/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/openmp/runtime/src/include -I/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/openmp/runtime/src/thirdparty/ittnotify -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -O3 -DNDEBUG -fPIC   -D _GNU_SOURCE -D _REENTRANT -MD -MT openmp/runtime/src/CMakeFiles/omp.dir/kmp_csupport.cpp.o -MF openmp/runtime/src/CMakeFiles/omp.dir/kmp_csupport.cpp.o.d -o openmp/runtime/src/CMakeFiles/omp.dir/kmp_csupport.cpp.o -c /home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/openmp/runtime/src/kmp_csupport.cpp
In file included from /home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/openmp/runtime/src/kmp_csupport.cpp:14:
/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src/omp.h:19:13: fatal error: 'stdlib.h' file not found
   19 | #   include <stdlib.h>
      |             ^~~~~~~~~~
1 error generated.
[3/33] Building CXX object openmp/runtime/src/CMakeFiles/omp.dir/kmp_alloc.cpp.o

Test File

#include <stdlib.h>
#include <signal.h>
int main(void) {return 0;}

Test file compile command (Successful)

/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/bin/clang++ 
--target=riscv64-unknown-linux-gnu -D__STDC_CONSTANT_MACROS 
-D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS 
--sysroot=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot 
--gcc-toolchain=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux 
-Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual 
-Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -DNDEBUG 
-fPIC main.cc -o main

The runtime arguments are separate from the ones you pass just for the LLVM build. We can probably try to improve this, but for now does it work if you do this? (Might have the CMake lists slightly wrong)

-DRUNTIMES_CMAKE_ARGS="-DCMAKE_C_FLAGS=--sysroot=${ROOT};--gcc-install-dir=${GCC_ROOT};-DCMAKE_CXX_FLAGS=--sysroot=${ROOT};--gcc-install-dir=${GCC_ROOT}"

Hello, thanks for your reply. I appreciate it.

I added the extra flag as you suggested and rebuilt the entire compiler.

However, it seems that now the build process doesn’t like the --gcc-toolchain flag.

Thanks in advance for all your help, I appreciate it. And sorry for dumping this long error log here.

Here is the error I get:

cmake --build . --target install             
[3904/3909] Performing configure step for 'runtimes'
Not searching for unused variables given on the command line.
CMake Error: Unknown argument --gcc-toolchain=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux
CMake Error: Run 'cmake --help' for all supported options.
FAILED: runtimes/runtimes-stamps/runtimes-configure /home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/runtimes/runtimes-stamps/runtimes-configure 
cd /home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/runtimes/runtimes-bins && /usr/bin/cmake 
-DCMAKE_C_FLAGS=--sysroot=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot --gcc-toolchain=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux 
-DCMAKE_CXX_FLAGS=--sysroot=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot --gcc-toolchain=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux --no-warn-unused-cli 
-DCMAKE_C_COMPILER=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/clang 
-DCMAKE_CXX_COMPILER=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/clang++ 
-DCMAKE_ASM_COMPILER=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/clang 
-DCMAKE_AR=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/llvm-ar 
-DCMAKE_RANLIB=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/llvm-ranlib 
-DCMAKE_NM=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/llvm-nm 
-DCMAKE_OBJDUMP=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/llvm-objdump 
-DCMAKE_OBJCOPY=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/llvm-objcopy 
-DCMAKE_STRIP=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/llvm-strip 
-DCMAKE_READELF=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/llvm-readelf 
-DCMAKE_C_COMPILER_TARGET=riscv64-unknown-linux-gnu 
-DCMAKE_CXX_COMPILER_TARGET=riscv64-unknown-linux-gnu 
-DCMAKE_ASM_COMPILER_TARGET=riscv64-unknown-linux-gnu 
-DCMAKE_INSTALL_PREFIX=/home/rahat/mphil-project/local/installs/riscv-llvm-lgnu 
-DLLVM_BINARY_DIR=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build 
-DLLVM_CONFIG_PATH=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/bin/llvm-config 
-DLLVM_ENABLE_WERROR=OFF 
-DLLVM_HOST_TRIPLE=x86_64-unknown-linux-gnu 
-DLLVM_HAVE_LINK_VERSION_SCRIPT=1 
-DLLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO=OFF 
-DLLVM_USE_RELATIVE_PATHS_IN_FILES=OFF -DLLVM_LIT_ARGS=-sv -DLLVM_SOURCE_PREFIX= -DPACKAGE_VERSION=19.0.0git -DCMAKE_BUILD_TYPE=Release 
-DCMAKE_MAKE_PROGRAM=/usr/bin/ninja -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DCOMPILER_RT_BUILD_BUILTINS=Off 
-DLLVM_INCLUDE_TESTS=ON 
-DLLVM_DEFAULT_TARGET_TRIPLE=riscv64-unknown-linux-gnu -DLLVM_ENABLE_PROJECTS_USED=ON 
-DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON -DLLVM_BUILD_TOOLS=ON 
-DCMAKE_C_COMPILER_WORKS=ON 
-DCMAKE_CXX_COMPILER_WORKS=ON -DCMAKE_ASM_COMPILER_WORKS=ON 
-DHAVE_LLVM_LIT=ON -DCLANG_RESOURCE_DIR= 
-DCMAKE_C_FLAGS=--sysroot=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot --gcc-toolchain=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux 
-DCMAKE_CXX_FLAGS=--sysroot=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot --gcc-toolchain=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux 
-DLLVM_ENABLE_RUNTIMES=openmp 
"-DRUNTIMES_CMAKE_ARGS=-DCMAKE_C_FLAGS=--sysroot=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot;--gcc-toolchain=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux;-DCMAKE_CXX_FLAGS=--sysroot=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot;--gcc-toolchain=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux" 
-GNinja /home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/llvm/runtimes/../../runtimes && /usr/bin/cmake -E touch /home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/runtimes/runtimes-stamps//runtimes-configure
ninja: build stopped: subcommand failed.
➜  build git:(main) ✗ 


Yeah, that’s what I meant about the separator not being right in the first post. What you’re seeing is a CMake error because it’s passing --gcc-toolchain to CMake instead of inside of the variable. I didn’t know how to actually pass two different arguments with that syntax. I wonder if it works with this? I’ve never had to pass such a weird amalgam of CMake arguments before.

-DRUNTIMES_CMAKE_ARGS="-DCMAKE_C_FLAGS=--sysroot=${ROOT}|--gcc-install-dir=${GCC_ROOT};-DCMAKE_CXX_FLAGS=--sysroot=${ROOT}|--gcc-install-dir=${GCC_ROOT}"

Sorry for the late replies, I will get back to you as soon as build is done. I changed compilers in between, and the build now needs to run again.

Okay, the | symbol is automatically being replaced by ;.

Also now the build seems to error out much earlier (I think), giving the below error.

Again thanks for your help, and sorry for the long error.

CMake Error at /home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/openmp/runtime/cmake/LibompUtils.cmake:26 (message):
  LIBOMP: libomp_check_variable(): LIBOMP_ARCH = is unknown
Call Stack (most recent call first):
  /home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/openmp/runtime/cmake/LibompUtils.cmake:140 (libomp_error_say)
  /home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/openmp/runtime/CMakeLists.txt:94 (libomp_check_variable)


-- Configuring incomplete, errors occurred!
See also "/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/runtimes/runtimes-bins/CMakeFiles/CMakeOutput.log".
See also "/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/runtimes/runtimes-bins/CMakeFiles/CMakeError.log".
[3861/3909] Building CXX object unittests/Passes/Plugins/TestPlugin/CMakeFiles/TestPlugin.dir/TestPlugin.cpp.o
FAILED: runtimes/runtimes-stamps/runtimes-configure /home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/runtimes/runtimes-stamps/runtimes-configure 
cd /home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/runtimes/runtimes-bins && /usr/bin/cmake 
"-DCMAKE_C_FLAGS=--sysroot=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot;--gcc-toolchain=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux" 
"-DCMAKE_CXX_FLAGS=--sysroot=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot;--gcc-toolchain=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux" 
--no-warn-unused-cli 
-DCMAKE_C_COMPILER=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/clang 
-DCMAKE_CXX_COMPILER=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/clang++ 
-DCMAKE_ASM_COMPILER=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/clang 
DCMAKE_AR=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/llvm-ar 
-DCMAKE_RANLIB=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/llvm-ranlib 
-DCMAKE_NM=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/llvm-nm 
-DCMAKE_OBJDUMP=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/llvm-objdump 
-DCMAKE_OBJCOPY=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/llvm-objcopy 
-DCMAKE_STRIP=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/llvm-strip 
-DCMAKE_READELF=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/./bin/llvm-readelf 
-DCMAKE_C_COMPILER_TARGET=riscv64-unknown-linux-gnu 
-DCMAKE_CXX_COMPILER_TARGET=riscv64-unknown-linux-gnu 
-DCMAKE_ASM_COMPILER_TARGET=riscv64-unknown-linux-gnu 
-DCMAKE_INSTALL_PREFIX=/home/rahat/mphil-project/local/installs/riscv-llvm-lgnu 
-DLLVM_BINARY_DIR=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build 
-DLLVM_CONFIG_PATH=/home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/bin/llvm-config 
-DLLVM_ENABLE_WERROR=OFF 
-DLLVM_HOST_TRIPLE=x86_64-unknown-linux-gnu -DLLVM_HAVE_LINK_VERSION_SCRIPT=1 
-DLLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO=OFF 
-DLLVM_USE_RELATIVE_PATHS_IN_FILES=OFF
-DLLVM_LIT_ARGS=-sv 
-DLLVM_SOURCE_PREFIX= 
-DPACKAGE_VERSION=19.0.0git 
-DCMAKE_BUILD_TYPE=Release 
-DCMAKE_MAKE_PROGRAM=/usr/bin/ninja 
-DCMAKE_EXPORT_COMPILE_COMMANDS=1 
-DCOMPILER_RT_BUILD_BUILTINS=Off 
-DLLVM_INCLUDE_TESTS=ON 
-DLLVM_DEFAULT_TARGET_TRIPLE=riscv64-unknown-linux-gnu -DLLVM_ENABLE_PROJECTS_USED=ON 
-DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON 
-DLLVM_BUILD_TOOLS=ON 
-DCMAKE_C_COMPILER_WORKS=ON 
-DCMAKE_CXX_COMPILER_WORKS=ON -DCMAKE_ASM_COMPILER_WORKS=ON 
-DHAVE_LLVM_LIT=ON 
-DCLANG_RESOURCE_DIR= "-DCMAKE_C_FLAGS=--sysroot=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot;--gcc-toolchain=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux" "-DCMAKE_CXX_FLAGS=--sysroot=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot;--gcc-toolchain=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux" 
-DLLVM_ENABLE_RUNTIMES=openmp "-DRUNTIMES_CMAKE_ARGS=-DCMAKE_C_FLAGS=--sysroot=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot;--gcc-toolchain=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux;-DCMAKE_CXX_FLAGS=--sysroot=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot;--gcc-toolchain=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux" 
-GNinja /home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/llvm/runtimes/../../runtimes && /usr/bin/cmake -E touch /home/rahat/mphil-project/local/sources/riscv-llvm/llvm-project/build/runtimes/runtimes-stamps//runtimes-configure
[3867/3909] Building CXX object unittests/Passes/Plugins/TestPlugin/CMakeFiles/TestPlugin.dir/TestPlugin.cpp.o
ninja: build stopped: subcommand failed.

Here is the cmake command:

cmake -G Ninja -DCMAKE_BUILD_TYPE="Release" \
  -DBUILD_SHARED_LIBS=True -DLLVM_USE_SPLIT_DWARF=True \
  -DCMAKE_INSTALL_PREFIX="/home/rahat/mphil-project/local/installs/riscv-llvm-lgnu" \
  -DLLVM_OPTIMIZED_TABLEGEN=True -DLLVM_BUILD_TESTS=False \
  -DDEFAULT_SYSROOT="/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot" \
  -DLLVM_DEFAULT_TARGET_TRIPLE="riscv64-unknown-linux-gnu" \
  -DLLVM_TARGETS_TO_BUILD="RISCV" \
  -DGCC_INSTALL_PREFIX="/home/rahat/mphil-project/local/install/rv-gnu-tc-linux" \
  -DLLVM_ENABLE_PROJECTS="clang" -DLLVM_ENABLE_RUNTIMES="openmp" \
  -DCMAKE_CXX_COMPILER=g++ \
  -DCMAKE_C_COMPILER=gcc \
  -DLLVM_PARALLEL_COMPILE_JOBS=16 -DLLVM_PARALLEL_LINK_JOBS=8 \
  -DRUNTIMES_CMAKE_ARGS="-DCMAKE_C_FLAGS=--sysroot=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot|--gcc-toolchain=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux;-DCMAKE_CXX_FLAGS=--sysroot=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux/sysroot|--gcc-toolchain=/home/rahat/mphil-project/local/installs/rv-gnu-tc-linux" ../llvm

Seems like it isn’t detecting the OpenMP architecture. There should be some code for RISCV-64, so I’m unsure why it’s not detected. I’d look around llvm-project/openmp/runtime/CMakeLists.txt at main · llvm/llvm-project · GitHub to try and see what’s not detecting it or what’s missing from the arguments.

I see, thanks alot for your help. I’ll update this post when I find something.