nvlink fatal : Could not open input file '/tmp/get_mem-decc5d.cubin'

Hi,

During my app build I get an error (Clang is the latest from master
with CUDA Toolkit 10.1):

...
[ 3%] Linking CXX shared library libsli.so
cd /home/usr4/c74014k/builds/nest-clang-offloading/sli &&
/home/usr4/c74014k/cmake-3.17.1-Linux-x86_64/bin/cmake -E
cmake_link_script CMakeFiles/sli_lib.dir/link.txt --verbose=1
/home/usr4/c74014k/opt/clang/current/bin/clang++ -fPIC -fopenmp
-fopenmp-targets=nvptx64 -std=c++11 -O2 -Wall -fopenmp=libomp
-shared -Wl,-soname,libsli.so -o libsli.so
CMakeFiles/sli_lib.dir/allocator.cpp.o
CMakeFiles/sli_lib.dir/arraydatum.cc.o
CMakeFiles/sli_lib.dir/booldatum.cc.o
CMakeFiles/sli_lib.dir/charcode.cc.o CMakeFiles/sli_lib.dir/datum.cc.o
CMakeFiles/sli_lib.dir/dict.cc.o CMakeFiles/sli_lib.dir/dictdatum.cc.o
CMakeFiles/sli_lib.dir/dictstack.cc.o
CMakeFiles/sli_lib.dir/dictutils.cc.o
CMakeFiles/sli_lib.dir/doubledatum.cc.o
CMakeFiles/sli_lib.dir/fdstream.cc.o
CMakeFiles/sli_lib.dir/filesystem.cc.o
CMakeFiles/sli_lib.dir/functiondatum.cc.o
CMakeFiles/sli_lib.dir/get_mem.c.o
CMakeFiles/sli_lib.dir/integerdatum.cc.o
CMakeFiles/sli_lib.dir/interpret.cc.o
CMakeFiles/sli_lib.dir/iostreamdatum.cc.o
CMakeFiles/sli_lib.dir/literaldatum.cc.o
CMakeFiles/sli_lib.dir/name.cc.o CMakeFiles/sli_lib.dir/slinames.cpp.o
CMakeFiles/sli_lib.dir/namedatum.cc.o
CMakeFiles/sli_lib.dir/oosupport.cc.o
CMakeFiles/sli_lib.dir/parser.cc.o
CMakeFiles/sli_lib.dir/processes.cc.o
CMakeFiles/sli_lib.dir/scanner.cc.o CMakeFiles/sli_lib.dir/sli_io.cc.o
CMakeFiles/sli_lib.dir/sliactions.cc.o
CMakeFiles/sli_lib.dir/sliarray.cc.o
CMakeFiles/sli_lib.dir/slibuiltins.cc.o
CMakeFiles/sli_lib.dir/slicontrol.cc.o
CMakeFiles/sli_lib.dir/slidata.cc.o
CMakeFiles/sli_lib.dir/slidict.cc.o
CMakeFiles/sli_lib.dir/sliexceptions.cc.o
CMakeFiles/sli_lib.dir/sligraphics.cc.o
CMakeFiles/sli_lib.dir/slimath.cc.o
CMakeFiles/sli_lib.dir/slimodule.cc.o
CMakeFiles/sli_lib.dir/sliregexp.cc.o
CMakeFiles/sli_lib.dir/slistack.cc.o
CMakeFiles/sli_lib.dir/slistartup.cc.o
CMakeFiles/sli_lib.dir/slitype.cc.o
CMakeFiles/sli_lib.dir/slitypecheck.cc.o
CMakeFiles/sli_lib.dir/specialfunctionsmodule.cc.o
CMakeFiles/sli_lib.dir/stringdatum.cc.o
CMakeFiles/sli_lib.dir/symboldatum.cc.o
CMakeFiles/sli_lib.dir/tarrayobj.cc.o
CMakeFiles/sli_lib.dir/token.cc.o
CMakeFiles/sli_lib.dir/tokenarray.cc.o
CMakeFiles/sli_lib.dir/tokenstack.cc.o
CMakeFiles/sli_lib.dir/tokenutils.cc.o
CMakeFiles/sli_lib.dir/triedatum.cc.o
CMakeFiles/sli_lib.dir/typechk.cc.o CMakeFiles/sli_lib.dir/utils.cc.o
-Wl,-rpath,/home/usr4/c74014k/builds/nest-clang-offloading/libnestutil::::::::::::::::::
../libnestutil/libnestutil.so /usr/lib64/libgsl.so
/usr/lib64/libgslcblas.so
nvlink fatal : Could not open input file '/tmp/get_mem-decc5d.cubin'
clang-11: error: nvlink command failed with exit code 1 (use -v to see
invocation)
make[2]: *** [sli/libsli.so] Error 1
make[2]: Leaving directory `/home/usr4/c74014k/builds/nest-clang-offloading'
make[1]: *** [sli/CMakeFiles/sli_lib.dir/all] Error 2
make[1]: Leaving directory `/home/usr4/c74014k/builds/nest-clang-offloading'
make: *** [all] Error 2

Hi Alexey,

Here it is:

cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_C_COMPILER=clang
-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS="-fopenmp
-fopenmp-targets=nvptx64"
-DCMAKE_INSTALL_PREFIX=$HOME/opt/nest-clang-offloading
-Dwith-python=OFF ~/projects/nest-simulator

Clang is still on yesterday's master built with CUDA 10.1. May I have
yours, I will give it a try on the x86 login node.

Okay, I am rebuilding LLVM master with CUDA 8.0 locally, may I have
your NEST CMake config parameters
so I can test it?

I could not get CUDA 8 working with the trunk Clang (older GCC
requirement etc.), so I will
wait for you to re-try the build with a newer CUDA Toolkit.

Do you use CUDA Toolkit 8 or 10 release?

I’ll try your config in my dev environment.

Thank you, Alexey for building the app. In my environment, I stick to
CUDA Toolkt 10.1, as 8 seems
require older GCC releases, but higher than CentOS 7's default 4.8.
Anyway, in summary
by configuring NEST with yours, builds went well without an issue.
Both -O0 and -O3 works.

Below is mine
cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_C_COMPILER=clang
-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS="-fopenmp
-fopenmp-targets=nvptx64"
-DCMAKE_INSTALL_PREFIX=$HOME/opt/nest-clang-offloading
-Dwith-python=OFF ~/projects/nest-simulator

Yes NEST has some C code.

Will ld or LLD be able to handle mixed objects

in future? That’s slightly un intuitive adding -fopenmp-targets=nvptx64 to the C_FLAGS while there’s zero C files containing omp target pragmas.

As far as I know, it is not a linker problem but a driver problem.

It will eventually go away but we are still working on other issues in this area.

Ok, thanks.