OpenMP offloading to the sm_35 device from x86 compute nodes

Hi,
I’m wondering if LLVM/Clang supports OpenMP offloading to NVIDA
targets from x86 nodes.

Thanks,
Itaru.

Yes, it should support it.

Best regards,
Alexey Bataev

Thanks. Does setting

LLVM_ENABLE_PROJECTS:STRING=clang;openmp;llvm

enough to create the corresponding .bc lib?

I’m not sure about it, there are some others who can help with this.

Best regards,
Alexey Bataev

Hi,

I’ve been building the offloading toolchain for Nvidia GPUs on x86-systems before and found Jonas Hahnfeld’s blog-post extremely useful to setup the compiler:

Best regards,

Lukas Sommer

Thanks for reming me about that excellent docs! I’ll try building the tool chain again, but using the official multi-project single repo on GitHub.

At the HEAD of Trunk of llvm-project, CMake seems to disregards these variables:
[…]
CMake Warning:
Manually-specified variables were not used by the project:

CLANG_OPENMP_NVPTX_DEFAULT_ARCH
LIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES

Have you had a chance to build the latest one?

This typically happens when cmake cannot find the CUDA installation.

Thanks,

–Doru

Doru et al.,
After checking the CUDA_ROOT environment variable gets set properly, I configured the
Makefile for my application with CMake, however:

– Could NOT find LIBOMPTARGET_DEP_LIBFFI (missing: LIBOMPTARGET_DEP_LIBFFI_LIBRARIES LIBOMPTARGET_DEP_LIBFFI_INCLUDE_DIRS)
CMake Warning (dev) at /p/project/cjzam11/kitayama1/projects/llvm-project/openmp/libomptarget/cmake/Modules/LibomptargetGetDependencies.cmake:117 (find_package):
Policy CMP0074 is not set: find_package uses _ROOT variables.
Run “cmake --help-policy CMP0074” for policy details. Use the cmake_policy
command to set the policy and suppress this warning.

Environment variable CUDA_ROOT is set to:

/usr/local/software/jureca/Stages/2018b/software/CUDA/9.2.88

For compatibility, CMake is ignoring the variable.
Call Stack (most recent call first):
/p/project/cjzam11/kitayama1/projects/llvm-project/openmp/libomptarget/CMakeLists.txt:30 (include)
This warning is for project developers. Use -Wno-dev to suppress it.

– LIBOMPTARGET: Building offloading runtime library libomptarget.
– LIBOMPTARGET: Not building aarch64 offloading plugin: machine not found in the system.
– LIBOMPTARGET: Building CUDA offloading plugin.
– LIBOMPTARGET: Not building PPC64 offloading plugin: machine not found in the system.
– LIBOMPTARGET: Not building PPC64le offloading plugin: machine not found in the system.
– LIBOMPTARGET: Not building x86_64 offloading plugin: libffi dependency not found.
– LIBOMPTARGET: Building CUDA offloading device RTL.
CMake Error at /p/project/cjzam11/kitayama1/projects/llvm-project/openmp/libomptarget/cmake/Modules/LibomptargetUtils.cmake:26 (message):
LIBOMPTARGET: Cannot build CUDA LLVM bitcode offloading device RTL!
Call Stack (most recent call first):
/p/project/cjzam11/kitayama1/projects/llvm-project/openmp/libomptarget/deviceRTLs/nvptx/CMakeLists.txt:115 (libomptarget_error_say)

I’m at today’s HEAD of Trunk of llvm-project.

Hi,

Doru et al.,
After checking the CUDA_ROOT environment variable gets set properly, I
configured the
Makefile for my application with CMake, however:

-- Could NOT find LIBOMPTARGET_DEP_LIBFFI (missing:
LIBOMPTARGET_DEP_LIBFFI_LIBRARIES
LIBOMPTARGET_DEP_LIBFFI_INCLUDE_DIRS)
CMake Warning (dev) at
/p/project/cjzam11/kitayama1/projects/llvm-project/openmp/libomptarget/cmake/Modules/LibomptargetGetDependencies.cmake:117
(find_package):
  Policy CMP0074 is not set: find_package uses <PackageName>_ROOT
variables.
  Run "cmake --help-policy CMP0074" for policy details. Use the
cmake_policy
  command to set the policy and suppress this warning.

  Environment variable CUDA_ROOT is set to:

    /usr/local/software/jureca/Stages/2018b/software/CUDA/9.2.88

  For compatibility, CMake is ignoring the variable.
Call Stack (most recent call first):

/p/project/cjzam11/kitayama1/projects/llvm-project/openmp/libomptarget/CMakeLists.txt:30
(include)
This warning is for project developers. Use -Wno-dev to suppress it.

-- LIBOMPTARGET: Building offloading runtime library libomptarget.
-- LIBOMPTARGET: Not building aarch64 offloading plugin: machine not
found in the system.
-- LIBOMPTARGET: Building CUDA offloading plugin.
-- LIBOMPTARGET: Not building PPC64 offloading plugin: machine not
found in the system.
-- LIBOMPTARGET: Not building PPC64le offloading plugin: machine not
found in the system.
-- LIBOMPTARGET: Not building x86_64 offloading plugin: libffi
dependency not found.
-- LIBOMPTARGET: Building CUDA offloading device RTL.

This looks good so far: You'll get the CUDA offloading plugin and the pure CUDA device RTL.
(libffi is needed for host-offloading which you don't need for Nvidia GPUs...)

CMake Error at
/p/project/cjzam11/kitayama1/projects/llvm-project/openmp/libomptarget/cmake/Modules/LibomptargetUtils.cmake:26
(message):
  LIBOMPTARGET: Cannot build CUDA LLVM bitcode offloading device RTL!
Call Stack (most recent call first):

/p/project/cjzam11/kitayama1/projects/llvm-project/openmp/libomptarget/deviceRTLs/nvptx/CMakeLists.txt:115
(libomptarget_error_say)

This is the code path when you manually specify LIBOMPTARGET_NVPTX_ENABLE_BCLIB, but have no capable compiler. This needs a (more or less) recent Clang compiler, what's your CMAKE_CXX_COMPILER?

Regarding the earlier errors about manually specified variables: Yes, the warning about LIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES could be related to CMake not finding CUDA, but the one about CLANG_OPENMP_NVPTX_DEFAULT_ARCH is not: Clang does not have a build-time dependency to CUDA and this variable is used unconditionally. Are you sure your set -DLLVM_ENABLE_PROJECTS=clang;openmp without typos in the initial cmake invocation? (you don't need llvm because that's the root project that you're pointing CMake to)

Cheers,
Jonas

Jonas, all,

Although Alexey warned me to use the recent Clan 7 or higher I recall, I was building the trunk llvm-project with GCC.

Can the future update to LLVM/Clang make GCC also usable in terms of building a toolchain which has OpenMP offload
capabilities?

Thanks,
Itaru.