Problem building llvm and clang 7.0.0

I am building LLVM and Clang using gcc 7.1 with Cmake and Ninja on a system based on Ubuntu 12.04.

The following commands worked for clang 6.0.0 with all the source files installed:

/opt/cmake/bin/cmake -GNinja
-DCMAKE_C_COMPILER=/opt/gcc-7.1/bin/gcc
-DCMAKE_CXX_COMPILER=/opt/gcc-7.1/bin/g++
-DGCC_INSTALL_PREFIX=/opt/gcc-7.1
-DCMAKE_INSTALL_PREFIX=/home/fletcher/LLVM/llvm-6.0.distrib/local -DCMAKE_BUILD_TYPE=Release -DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_TARGETS_TO_BUILD=“host” -DLLVM_PARALLEL_LINK_JOBS=1 -DCMAKE_EXE_LINKER_FLAGS="-Xlinker -R/opt/gcc-7.1/lib64 -L/opt/gcc-7.1/lib64" …/llvm

Some things seem to have changed with version 7.0.0

(1) There has been a change associated with libomptarget and now it fails as it is picking up an old version of cuda on my computer and complaining that the gcc is too new. The only way I could find to get rid of this is to remove openmp.

(2) When I did this it still does not build as it cannot link a library with the following messages:

FAILED: lib/clang/7.0.0/lib/linux/libclang_rt.fuzzer_no_main-x86_64.a
–localize-hidden: not found

–localize-hidden seems to be an option for ld. I have not traced any documentation.

What makes things difficult is that these failures occur after several hours of processing, having not been picked up by Cmake.

It may be that the system version of ld is being picked up and I need to take some action on that.

Any help will be appreciated.

Thanks

John Fletcher

libomptarget uses the default FindCUDA.cmake. Can you try setting CUDA_TOOLKIT_ROOT_DIR and / or CUDA_BIN_PATH to pont to the "right" version of CUDA?
Alternatively you can point LIBOMPTARGET_NVPTX_ALTERNATE_HOST_COMPILER to an older version of GCC that is compatible with the detected version of CUDA.

Cheers,
Jonas

Hi John,

_______________________________________
From: Jonas Hahnfeld [hahnjo@hahnjo.de]
Sent: 10 August 2018 18:08
To: Fletcher, John P
Cc: cfe-dev ‎[cfe-dev@lists.llvm.org]‎
Subject: Re: [cfe-dev] Problem building llvm and clang 7.0.0

I am building LLVM and Clang using gcc 7.1 with Cmake and Ninja on a
system based on Ubuntu 12.04.

The following commands worked for clang 6.0.0 with all the source
files installed:

/opt/cmake/bin/cmake -GNinja \
-DCMAKE_C_COMPILER=/opt/gcc-7.1/bin/gcc \
-DCMAKE_CXX_COMPILER=/opt/gcc-7.1/bin/g++ \
-DGCC_INSTALL_PREFIX=/opt/gcc-7.1 \
-DCMAKE_INSTALL_PREFIX=/home/fletcher/LLVM/llvm-6.0.distrib/local
-DCMAKE_BUILD_TYPE=Release -DLLVM_BUILD_LLVM_DYLIB=ON
-DLLVM_TARGETS_TO_BUILD="host" -DLLVM_PARALLEL_LINK_JOBS=1
-DCMAKE_EXE_LINKER_FLAGS="-Xlinker -R/opt/gcc-7.1/lib64
-L/opt/gcc-7.1/lib64" ../llvm

Some things seem to have changed with version 7.0.0

(1) There has been a change associated with libomptarget and now it
fails as it is picking up an old version of cuda on my computer and
complaining that the gcc is too new. The only way I could find to get
rid of this is to remove openmp.

libomptarget uses the default FindCUDA.cmake. Can you try setting
CUDA_TOOLKIT_ROOT_DIR and / or CUDA_BIN_PATH to pont to the "right"
version of CUDA?
Alternatively you can point LIBOMPTARGET_NVPTX_ALTERNATE_HOST_COMPILER
to an older version of GCC that is compatible with the detected version
of CUDA.

Cheers,
Jonas

------------------------

Jonas

Thank you for your suggestion. I have tried various things and in the
end I have renamed my out of date link to cuda as cudaold so that it
is no longer picked up. That is not enough as it still fails in
openmp so I have taken that out.

To the CLANG community.

(1) I am concerned that something has changed and I cannot find any
documentation, in particular something which worked does not and is
NOT picked up by Cmake so that it crashes after 2.5 hours of running.
What is it that has changed?

Of course something has changed, we are not standing still after all. If there were no changes what would be the point of new releases?
In this particular case the OpenMP runtime for Nvidia GPUs is now included in the release.

(2) There has been no response on this problem.
-------
(2) When I did this it still does not build as it cannot link a
library with the following messages:

FAILED: lib/clang/7.0.0/lib/linux/libclang_rt.fuzzer_no_main-x86_64.a
--localize-hidden: not found

--localize-hidden seems to be an option for ld. I have not traced any
documentation.
--------
I now think that the problem about --localize-hidden: not found is
because it should be preceded by objcopy, for which it is an argument,
so that something in the command processing is broken.

Or CMake just didn't find your objcopy - which would be bad. You have that installed, haven't you?

I am spending my time building on rc1 to help you get things sorted.
Where are you all?

Please keep in mind that you are not guaranteed to get support from the community.
To maximize chances you should post as much information as possible to narrow down the problem:
  - Are there errors during configuration?
  - What are the exact error messages you get when building?
  - What versions of software are you using? You already mentioned GCC 7.1, but which version of CUDA?
  - Does CMake find the versions you are intending to use? A log file might help here...

Regards,
Jonas

Jonas - see later down the message
________________________________________
From: Jonas Hahnfeld [hahnjo@hahnjo.de]
Sent: 15 August 2018 08:03
To: Fletcher, John P
Cc: cfe-dev ‎[cfe-dev@lists.llvm.org]‎
Subject: Re: [cfe-dev] Problem building llvm and clang 7.0.0

Hi John,

_______________________________________
From: Jonas Hahnfeld [hahnjo@hahnjo.de]
Sent: 10 August 2018 18:08
To: Fletcher, John P
Cc: cfe-dev ‎[cfe-dev@lists.llvm.org]‎
Subject: Re: [cfe-dev] Problem building llvm and clang 7.0.0

I am building LLVM and Clang using gcc 7.1 with Cmake and Ninja on a
system based on Ubuntu 12.04.

The following commands worked for clang 6.0.0 with all the source
files installed:

/opt/cmake/bin/cmake -GNinja \
-DCMAKE_C_COMPILER=/opt/gcc-7.1/bin/gcc \
-DCMAKE_CXX_COMPILER=/opt/gcc-7.1/bin/g++ \
-DGCC_INSTALL_PREFIX=/opt/gcc-7.1 \
-DCMAKE_INSTALL_PREFIX=/home/fletcher/LLVM/llvm-6.0.distrib/local
-DCMAKE_BUILD_TYPE=Release -DLLVM_BUILD_LLVM_DYLIB=ON
-DLLVM_TARGETS_TO_BUILD="host" -DLLVM_PARALLEL_LINK_JOBS=1
-DCMAKE_EXE_LINKER_FLAGS="-Xlinker -R/opt/gcc-7.1/lib64
-L/opt/gcc-7.1/lib64" ../llvm

Some things seem to have changed with version 7.0.0

(1) There has been a change associated with libomptarget and now it
fails as it is picking up an old version of cuda on my computer and
complaining that the gcc is too new. The only way I could find to get
rid of this is to remove openmp.

libomptarget uses the default FindCUDA.cmake. Can you try setting
CUDA_TOOLKIT_ROOT_DIR and / or CUDA_BIN_PATH to pont to the "right"
version of CUDA?
Alternatively you can point LIBOMPTARGET_NVPTX_ALTERNATE_HOST_COMPILER
to an older version of GCC that is compatible with the detected version
of CUDA.

Cheers,
Jonas

------------------------

Jonas

Thank you for your suggestion. I have tried various things and in the
end I have renamed my out of date link to cuda as cudaold so that it
is no longer picked up. That is not enough as it still fails in
openmp so I have taken that out.

To the CLANG community.

(1) I am concerned that something has changed and I cannot find any
documentation, in particular something which worked does not and is
NOT picked up by Cmake so that it crashes after 2.5 hours of running.
What is it that has changed?

Of course something has changed, we are not standing still after all. If
there were no changes what would be the point of new releases?
In this particular case the OpenMP runtime for Nvidia GPUs is now
included in the release.

(2) There has been no response on this problem.
-------
(2) When I did this it still does not build as it cannot link a
library with the following messages:

FAILED: lib/clang/7.0.0/lib/linux/libclang_rt.fuzzer_no_main-x86_64.a
--localize-hidden: not found

--localize-hidden seems to be an option for ld. I have not traced any
documentation.
--------
I now think that the problem about --localize-hidden: not found is
because it should be preceded by objcopy, for which it is an argument,
so that something in the command processing is broken.

Or CMake just didn't find your objcopy - which would be bad. You have
that installed, haven't you?

I am spending my time building on rc1 to help you get things sorted.
Where are you all?

Please keep in mind that you are not guaranteed to get support from the
community.
To maximize chances you should post as much information as possible to
narrow down the problem:
  - Are there errors during configuration?
  - What are the exact error messages you get when building?
  - What versions of software are you using? You already mentioned GCC
7.1, but which version of CUDA?
  - Does CMake find the versions you are intending to use? A log file
might help here...

Regards,
Jonas

-------------------------

Jonas

Thank you again.

The comments I made about the lack of response were because it was the
middle of the night and it had failed again. I am building rc1 and
reporting failure, which is what we are encouraged to do!

I have now made progress with things.

On (1) The version of Cuda is an old one. 6.5. The point is that I
don't want it included in the build at all, and I cannot find an
option to exclude it. I have some hardware for CUDA on my computer
which I am not currently using. The solution to this problem at the
moment is to remove openmp completely.

Ok, that explains what you are seeing: Cuda 6.5 is older (from 2014) than GCC-7 (from last year). As I mentioned in my first email you may try to set -DLIBOMPTARGET_NVPTX_ALTERNATE_HOST_COMPILER=/usr/bin/gcc which is probably older on Ubuntu 12.04. For disabling, see below.

When things have changed I look at the files reporting changes. It is
helpful if there is a note there if it is a breaking change or how to
turn off a change.

It's added functionality so I wouldn't consider it a "breaking change". I think the build is breaking because of the particular setup which is just bad luck.
AFAIK there is no option to turn off building libomptarget-nvptx. But if you are not interested in OpenMP offloading you can pass -DOPENMP_ENABLE_LIBOMPTARGET=Off (which is undocumented as I just became aware) to disable all of libomptarget. That way you will at least get libomp, the OpenMP runtime library for host features.

On (2) I have found that the CmakeCache.txt file produced for 6.0.0
and 7.0.0 are different.

The following lines which help locate binaries are missing in the file
for 7.0.0.

//Path to a program.
CMAKE_NM:FILEPATH=/usr/bin/nm

//Path to a program.
CMAKE_OBJCOPY:FILEPATH=/usr/bin/objcopy

//Path to a program.
CMAKE_OBJDUMP:FILEPATH=/usr/bin/objdump

//Path to a program.
CMAKE_RANLIB:FILEPATH=/usr/bin/ranlib

I am using cmake 3.6.2 in both cases.

I do not know why the action has changed.

That's weird, assuming that you didn't remove these binaries in the meantime. I'll ask a colleague of mine who was facing the same problem but I can't remember the solution...

Cheers,
Jonas

Hi Jonas,
I have a related question: is it possible to build the nvidia OpenMP support with clang itself, instead of nvcc ?

I am trying to build clang 7 (r339664) with support for CUDA 9.2.148, using gcc 8.1, on a CentOS 7.5 system.

It builds fine if I leave out OpenMP support:

$ cmake $BASE/llvm-project/llvm -G Ninja
-DLLVM_TARGETS_TO_BUILD:STRING=“X86;NVPTX”
-DLLVM_ENABLE_PROJECTS=“clang;clang-tools-extra;compiler-rt;lld”
-DCMAKE_BUILD_TYPE:STRING=Release
-DCMAKE_INSTALL_PREFIX:PATH="$BASE/install-r339664"
-DLLVM_LIBDIR_SUFFIX:STRING=64
-DBUILD_SHARED_LIBS:BOOL=ON
-DLLVM_ENABLE_EH:BOOL=ON
-DLLVM_ENABLE_PIC:BOOL=ON
-DLLVM_ENABLE_RTTI:BOOL=ON

– Targeting X86
– Targeting NVPTX

– Clang version: 7.0.0
– Performing Test CXX_SUPPORTS_NO_NESTED_ANON_TYPES_FLAG
– Performing Test CXX_SUPPORTS_NO_NESTED_ANON_TYPES_FLAG - Failed
– LLD version: 7.0.0
– Configuring done
– Generating done
– Build files have been written to: /data/user/fwyzard/llvm/build-r339664

$ ninja

[3874/3875] Linking CXX executable bin/llvm-readobj
[3875/3875] Generating …/…/bin/llvm-readelf

If I try to include OpenMP support, it correctly finds the CUDA installation (and libelf / libffi with some help):

$ cmake $BASE/llvm-project/llvm -G Ninja
-DLLVM_TARGETS_TO_BUILD:STRING=“X86;NVPTX”
-DLLVM_ENABLE_PROJECTS=“clang;clang-tools-extra;compiler-rt;lld;openmp”
-DLIBOMPTARGET_DEP_LIBELF_INCLUDE_DIR:PATH="$GCC_ROOT/include"
-DLIBOMPTARGET_DEP_LIBELF_LIBRARIES:PATH="$GCC_ROOT/lib/libelf.so"
-DLIBOMPTARGET_DEP_LIBFFI_INCLUDE_DIR:PATH="$LIBFFI_ROOT/include"
-DLIBOMPTARGET_DEP_LIBFFI_LIBRARIES:PATH="$LIBFFI_ROOT/lib64/libffi.so"
-DCMAKE_BUILD_TYPE:STRING=Release
-DCMAKE_INSTALL_PREFIX:PATH="$BASE/install-r339664"
-DLLVM_LIBDIR_SUFFIX:STRING=64
-DBUILD_SHARED_LIBS:BOOL=ON
-DLLVM_ENABLE_EH:BOOL=ON
-DLLVM_ENABLE_PIC:BOOL=ON
-DLLVM_ENABLE_RTTI:BOOL=ON

– Targeting X86
– Targeting NVPTX

– Found LIBOMPTARGET_DEP_LIBELF: /data/user/fwyzard/llvm/build/slc7_amd64_gcc810.llvm/slc7_amd64_gcc810/external/gcc/8.1.0/lib/libelf.so
– Found LIBOMPTARGET_DEP_LIBFFI: /data/user/fwyzard/llvm/build/slc7_amd64_gcc810.llvm/slc7_amd64_gcc810/external/libffi/3.2.1/lib64/libffi.so
– LIBOMPTARGET: Building offloading runtime library libomptarget.
– LIBOMPTARGET: Not building aarch64 offloading plugin: machine not found in the system.
– LIBOMPTARGET: Building CUDA offloading plugin.
– LIBOMPTARGET: Not building PPC64 offloading plugin: machine not found in the system.
– LIBOMPTARGET: Not building PPC64le offloading plugin: machine not found in the system.
– LIBOMPTARGET: Building x86_64 offloading plugin.
– LIBOMPTARGET: Building CUDA offloading device RTL.
– Clang version: 7.0.0
– LLD version: 7.0.0
– Configuring done
– Generating done
– Build files have been written to: /data/user/fwyzard/llvm/build-r339664

but then it fails because CUDA 9.2 does not support gcc 8:

ninja

[780/3744] Building NVCC (Device) object projects/openmp/libomptarget/deviceRTLs/nvptx/CMakeFiles/omptarget-nvptx.dir/src/omptarget-nvptx_generated_cancel.cu.o
FAILED: projects/openmp/libomptarget/deviceRTLs/nvptx/CMakeFiles/omptarget-nvptx.dir/src/omptarget-nvptx_generated_cancel.cu.o
cd /data/user/fwyzard/llvm/build-r339664/projects/openmp/libomptarget/deviceRTLs/nvptx/CMakeFiles/omptarget-nvptx.dir/src && /data/user/fwyzard/llvm/build/slc7_amd64_gcc810.llvm/slc7_amd64_gcc810/external/cmake/3.11.1/bin/cmake -E make_directory /data/user/fwyzard/llvm/b
In file included from /data/user/fwyzard/llvm/build/slc7_amd64_gcc810.llvm/slc7_amd64_gcc810/external/cuda/9.2.148/include/host_config.h:50,
from /data/user/fwyzard/llvm/build/slc7_amd64_gcc810.llvm/slc7_amd64_gcc810/external/cuda/9.2.148/include/cuda_runtime.h:78,
from :
/data/user/fwyzard/llvm/build/slc7_amd64_gcc810.llvm/slc7_amd64_gcc810/external/cuda/9.2.148/include/crt/host_config.h:119:2: error: #error – unsupported GNU version! gcc versions later than 7 are not supported!
#error – unsupported GNU version! gcc versions later than 7 are not supported!

and so on.

Is it possible to build openmp with offload capabilities, using the just-built clang 7 itself as the compiler ?

Ciao,
.Andrea

Hi Andrea,

I think right now CMake is only prepared to build the OpenMP runtime using nvcc. That said, it could now be possible to add support for building with Clang. This probably needs to use Relocatable Device Code which wasn't available before Clang 7. I'm not sure if somebody is already working on this, but that sounds like a good improvement to make!

For now you can workaround the too recent version of GCC by using the mentioned LIBOMPTARGET_NVPTX_ALTERNATE_HOST_COMPILER. This makes the check pass even though the compiler is never used (libomptarget-nvptx doesn't have any host code).

Regards,
Jonas

Hi Jonas,