llvm-6.0.0rc2: fatal error: clang/Basic/Version.h: No such file or directory

Siegmar_Gross · February 15, 2018, 2:45pm

Hi,

today I've tried to build llvm-6.0.0rc2 using Cmake on my "SUSE Linux
Enterprise Server 12.3 (x86_64)" with the following commands (gcc-6.4.0
is necessary for CUDA-9.0).

wget http://prereleases.llvm.org/6.0.0/rc2/llvm-6.0.0rc2.src.tar.xz
wget http://prereleases.llvm.org/6.0.0/rc2/cfe-6.0.0rc2.src.tar.xz
wget http://prereleases.llvm.org/6.0.0/rc2/clang-tools-extra-6.0.0rc2.src.tar.xz
wget http://prereleases.llvm.org/6.0.0/rc2/compiler-rt-6.0.0rc2.src.tar.xz
wget http://prereleases.llvm.org/6.0.0/rc2/lldb-6.0.0rc2.src.tar.xz
wget http://prereleases.llvm.org/6.0.0/rc2/lld-6.0.0rc2.src.tar.xz
wget http://prereleases.llvm.org/6.0.0/rc2/polly-6.0.0rc2.src.tar.xz
wget http://prereleases.llvm.org/6.0.0/rc2/openmp-6.0.0rc2.src.tar.xz

tar xf llvm-6.0.0rc2.src.tar.xz
cd llvm-6.0.0rc2.src/tools
tar xf ../../cfe-6.0.0rc2.src.tar.xz
tar xf ../../polly-6.0.0rc2.src.tar.xz
tar xf ../../lldb-6.0.0rc2.src.tar.xz
tar xf ../../lld-6.0.0rc2.src.tar.xz
cd cfe-6.0.0rc2.src/tools
tar xf ../../../../clang-tools-extra-6.0.0rc2.src.tar.xz
cd ../../../projects
tar xf ../../compiler-rt-6.0.0rc2.src.tar.xz
tar xf ../../openmp-6.0.0rc2.src.tar.xz
cd ../..

rm -r build
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr/local/llvm-6.0.0\
   -GNinja \
   -DLLVM_TARGETS_TO_BUILD:STRING="NVPTX;X86" \
   -DCMAKE_BUILD_TYPE:STRING="Release" \
   -DLLVM_PARALLEL_COMPILE_JOBS:STRING="4" \
   -DLLVM_PARALLEL_LINK_JOBS:STRING="4" \
   -DCMAKE_C_COMPILER:STRING="/usr/local/gcc-6.4.0/bin/gcc" \
   -DCMAKE_C_FLAGS:STRING="-m64 -I/usr/local/valgrind/include" \
   -DCMAKE_CXX_COMPILER:STRING="/usr/local/gcc-6.4.0/bin/g++" \
   -DCMAKE_CXX_FLAGS:STRING="-m64 -I/usr/local/valgrind/include" \
   -DCMAKE_EXE_LINKER_FLAGS:STRING="-m64" \
   -DLLVM_LIBDIR_SUFFIX:STRING="64" \
   -DLLVM_POLLY_LINK_INTO_TOOLS:BOOL=ON \
-DLIBOMPTARGET_DEP_LIBELF_INCLUDE_DIR:STRING="/usr/local/elfutils-0.169/include" \
-DLIBOMPTARGET_DEP_LIBELF_LIBRARIES:STRING="/usr/local/elfutils-0.169/lib64/libelf.so" \
   -DLIBOMPTARGET_DEP_LIBFFI_INCLUDE_DIR:STRING="/usr/include" \
   -DLIBOMPTARGET_DEP_LIBFFI_LIBRARIES:STRING="/usr/lib64/libffi.so" \
   -DCUDA_INCLUDE_DIRS:STRING="/usr/local/cuda/include" \
   -DCUDA_LIBRARIES:STRING="/usr/local/cuda/lib64/libcudart.so" \
   -DBUILD_SHARED_LIBS:BOOL=ON \
   ../llvm-6.0.0rc2.src \
   >& tee log.cmake

ninja |& tee log.ninja-build

Unfortunately, I get the following error, although the missing file is
available.

loki build 188 tail -12 log.ninja-build
[3186/4317] Building CXX object tools/llvm-objdump/CMakeFiles/llvm-objdump.dir/WasmDump.cpp.o
[3187/4317] Building CXX object tools/lldb-6.0.0rc2.src/source/CMakeFiles/lldbBase.dir/lldb.cpp.o
FAILED: tools/lldb-6.0.0rc2.src/source/CMakeFiles/lldbBase.dir/lldb.cpp.o
/usr/local/gcc-6.4.0/bin/g++ -DGTEST_HAS_RTTI=0 -DHAVE_ROUND -DLIBXML2_DEFINED -DLLDB_CONFIGURATION_RELEASE -DLLDB_USE_BUILTIN_DEMANGLER -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/lldb-6.0.0rc2.src/source -I/export2/src/llvm-6.0.0/llvm-6.0.0rc2.src/tools/lldb-6.0.0rc2.src/source -Itools/lldb-6.0.0rc2.src/include -I/export2/src/llvm-6.0.0/llvm-6.0.0rc2.src/tools/lldb-6.0.0rc2.src/include -I/usr/include/libxml2 -Iinclude -I/export2/src/llvm-6.0.0/llvm-6.0.0rc2.src/include -I/usr/include/python2.7 -I/export2/src/llvm-6.0.0/llvm-6.0.0rc2.src/tools/clang/include -Itools/lldb-6.0.0rc2.src/../clang/include -I/export2/src/llvm-6.0.0/llvm-6.0.0rc2.src/tools/lldb-6.0.0rc2.src/source/. -m64 -I/usr/local/valgrind/include -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -ffunction-sections -fdata-sections -Wno-deprecated-declarations -Wno-unknown-pragmas -Wno-strict-aliasing -Wno-deprecated-register -Wno-vla-extension -O3 -DNDEBUG -fno-exceptions -fno-rtti -MMD -MT tools/lldb-6.0.0rc2.src/source/CMakeFiles/lldbBase.dir/lldb.cpp.o -MF tools/lldb-6.0.0rc2.src/source/CMakeFiles/lldbBase.dir/lldb.cpp.o.d -o tools/lldb-6.0.0rc2.src/source/CMakeFiles/lldbBase.dir/lldb.cpp.o -c /export2/src/llvm-6.0.0/llvm-6.0.0rc2.src/tools/lldb-6.0.0rc2.src/source/lldb.cpp
/export2/src/llvm-6.0.0/llvm-6.0.0rc2.src/tools/lldb-6.0.0rc2.src/source/lldb.cpp:15:33: fatal error: clang/Basic/Version.h: No such file or directory
#include "clang/Basic/Version.h"
^
compilation terminated.
[3188/4317] Building CXX object tools/llvm-cfi-verify/lib/CMakeFiles/LLVMCFIVerify.dir/GraphBuilder.cpp.o
[3189/4317] Building CXX object tools/llvm-objdump/CMakeFiles/llvm-objdump.dir/llvm-objdump.cpp.o
[3190/4317] Building CXX object tools/llvm-objdump/CMakeFiles/llvm-objdump.dir/MachODump.cpp.o
ninja: build stopped: subcommand failed.
loki build 189
loki build 189 find /export2/src/llvm-6.0.0/llvm-6.0.0rc2.src -name Version.h
/export2/src/llvm-6.0.0/llvm-6.0.0rc2.src/tools/cfe-6.0.0rc2.src/include/clang/Basic/Version.h
/export2/src/llvm-6.0.0/llvm-6.0.0rc2.src/tools/lld-6.0.0rc2.src/include/lld/Common/Version.h
loki build 190

Is it necessary to unpack the archives in different directories or to set
some environment variables or symbolic links so that Version.h will be found?
Thank you very much for any help in advance.

Kind regards

Siegmar

DimitryAndric · February 15, 2018, 7:02pm

After this, the directory names will not be correct, unfortunately. You'll need to do this instead:

tar xf llvm-6.0.0rc2.src.tar.xz
tar xf cfe-6.0.0rc2.src.tar.xz
tar xf polly-6.0.0rc2.src.tar.xz
tar xf lldb-6.0.0rc2.src.tar.xz
tar xf lld-6.0.0rc2.src.tar.xz
tar xf clang-tools-extra-6.0.0rc2.src.tar.xz
tar xf compiler-rt-6.0.0rc2.src.tar.xz
tar xf openmp-6.0.0rc2.src.tar.xz

mv cfe-6.0.0rc2.src llvm-6.0.0rc2.src/tools/clang
mv polly-6.0.0rc2.src llvm-6.0.0rc2.src/tools/polly
mv lldb-6.0.0rc2.src llvm-6.0.0rc2.src/tools/lldb
mv lld-6.0.0rc2.src llvm-6.0.0rc2.src/tools/lld
mv clang-tools-extra-6.0.0rc2.src llvm-6.0.0rc2.src/tools/clang/tools/extra
mv compiler-rt-6.0.0rc2.src llvm-6.0.0rc2.src/projects/compiler-rt
mv openmp-6.0.0rc2.src llvm-6.0.0rc2.src/projects/openmp

I think it might be better to distribute the source tarballs all rooted at llvm-6.0.0rc2, with the correct subpaths already embedded. Hans, what are your thoughts about that?

Hopefully this will become moot at some point, if the monorepo ever gets off the ground...

-Dimitry

Andrew_Kelley1 · February 15, 2018, 7:35pm

What is the purpose of supporting this "copy directories around" sort
of build strategy anyway?

At the end of the day, the way these packages get installed is via
system package managers is independently, each project finding its
dependencies with the standard prefix path.

When I compile from source it's always like this:

$ cd llvm-6.0.0rc2.src/
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/local
-DCMAKE_PREFIX_PATH=$HOME/local -DCMAKE_BUILD_TYPE=Release
$ make install

$ cd cfe-6.0.0rc2.src/
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/local
-DCMAKE_PREFIX_PATH=$HOME/local -DCMAKE_BUILD_TYPE=Release
$ make install

$ cd lld-6.0.0rc2.src/
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/local
-DCMAKE_PREFIX_PATH=$HOME/local -DCMAKE_BUILD_TYPE=Release
$ make install

etc

This is how stuff gets installed on Debian, Ubuntu, NixOS, Arch,
Gentoo, etc. And it works for Windows too:

mkdir C:\Users\Andy\llvm-6.0.0rc2.src\build-release
cd C:\Users\Andy\llvm-6.0.0rc2.src\build-release
"c:\Program Files\CMake\bin\cmake.exe" .. -Thost=x64 -G"Visual Studio 14 2015 Win64" -DCMAKE_INSTALL_PREFIX=C:\Users\Andy\llvm+clang-6.0.0rc2-win64-msvc-release -DCMAKE_PREFIX_PATH=C:\Users\Andy\llvm+clang-6.0.0rc2-win64-msvc-release -DCMAKE_BUILD_TYPE=Release
msbuild -p:Configuration=Release INSTALL.vcxproj

What are our use cases for the strategy where we move directories into
llvm to build?

DimitryAndric · February 15, 2018, 7:45pm

What is the purpose of supporting this "copy directories around" sort
of build strategy anyway?

The purpose, at least it seems to me in this particular case, is to isolate the needed headers into a build directory, without pulling in all the other headers from the source directory.

For example, both on Linux and FreeBSD, cxxabi.h is installed into the system C++ header directory. On Debian, this is /usr/include/c++/6/cxxabi.h (as part of libstdc++), on FreeBSD it is /usr/include/c++/v1 (as part of libc++).

If you would simply add -I /usr/include/c++/6/cxxabi.h or -I /usr/include/c++/v1 to your compilation flags, you would also pull in other C++ headers, which you do not want during the build of libc++.

In the past I have also manually created some empty directory, symlinked the needed C++ ABI headers in there, and pointed the libc++ build to that directory for inclusion. But copying is also OK, of course.

The point is, usually the system does not make the C++ ABI headers available in a nicely isolated directory, so this is why you have to work around that.

At the end of the day, the way these packages get installed is via
system package managers is independently, each project finding its
dependencies with the standard prefix path.

When I compile from source it's always like this:

$ cd llvm-6.0.0rc2.src/
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/local
-DCMAKE_PREFIX_PATH=$HOME/local -DCMAKE_BUILD_TYPE=Release
$ make install

$ cd cfe-6.0.0rc2.src/
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/local
-DCMAKE_PREFIX_PATH=$HOME/local -DCMAKE_BUILD_TYPE=Release
$ make install

$ cd lld-6.0.0rc2.src/
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/local
-DCMAKE_PREFIX_PATH=$HOME/local -DCMAKE_BUILD_TYPE=Release
$ make install

I never do that. I git clone or svn checkout all the directories in their right places, and run cmake for all of them in one go, without needing to "make install" in intermediate steps. In fact, make install only occurs at the very end, after everything has been built and tested.

This is also the way the test-release.sh script does things, btw.

etc

This is how stuff gets installed on Debian, Ubuntu, NixOS, Arch,
Gentoo, etc.

It seems that this is not very handy, as you will have to package up each individual component first, install it, and only then can you proceed with the build of the next component. You also need root to do that.

Much easier to build everything in one huge sandbox, then roll packages from there, if you ask me.

And it works for Windows too:

mkdir C:\Users\Andy\llvm-6.0.0rc2.src\build-release
cd C:\Users\Andy\llvm-6.0.0rc2.src\build-release
"c:\Program Files\CMake\bin\cmake.exe" .. -Thost=x64 -G"Visual Studio 14 2015 Win64" -DCMAKE_INSTALL_PREFIX=C:\Users\Andy\llvm+clang-6.0.0rc2-win64-msvc-release -DCMAKE_PREFIX_PATH=C:\Users\Andy\llvm+clang-6.0.0rc2-win64-msvc-release -DCMAKE_BUILD_TYPE=Release
msbuild -p:Configuration=Release INSTALL.vcxproj

What are our use cases for the strategy where we move directories into
llvm to build?

It looks like the source tree has always been organized this way. Maybe it has recently become possible to build all those projects in a separated fashion, as you described earlier, but I was not aware of it. I also do not see the advantage, and significant risk that you might mix and match incompatible versions...

-Dimitry

Siegmar_Gross · February 16, 2018, 10:35am

Hi Dimitry,

thank you very much for your help. I was able to build everything, after
renaming the directories and adding "-I/usr/include/ncurses" to
CMAKE_C_FLAGS and CMAKE_CXX_FLAGS. "panel.h" wasn't found without that
directory as you can see below.

[3627/4660] Building CXX object tools/lldb/source/Core/CMakeFiles/lldbCore.dir/IOHandler.cpp.o
FAILED: tools/lldb/source/Core/CMakeFiles/lldbCore.dir/IOHandler.cpp.o
/usr/local/gcc-6.4.0/bin/g++ -DGTEST_HAS_RTTI=0 -DHAVE_ROUND -DLIBXML2_DEFINED -DLLDB_CONFIGURATION_RELEASE -DLLDB_USE_BUILTIN_DEMANGLER -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/lldb/source/Core -I/export2/src/llvm-6.0.0/llvm/tools/lldb/source/Core -Itools/lldb/include -I/export2/src/llvm-6.0.0/llvm/tools/lldb/include -I/usr/include/libxml2 -Iinclude -I/export2/src/llvm-6.0.0/llvm/include -I/usr/include/python2.7 -I/export2/src/llvm-6.0.0/llvm/tools/clang/include -Itools/lldb/../clang/include -I/export2/src/llvm-6.0.0/llvm/tools/lldb/source/. -m64 -I/usr/local/valgrind/include -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -ffunction-sections -fdata-sections -Wno-deprecated-declarations -Wno-unknown-pragmas -Wno-strict-aliasing -Wno-deprecated-register -Wno-vla-extension -O3 -DNDEBUG -fno-exceptions -fno-rtti -MMD -MT tools/lldb/source/Core/CMakeFiles/lldbCore.dir/IOHandler.cpp.o -MF tools/lldb/source/Core/CMakeFiles/lldbCore.dir/IOHandler.cpp.o.d -o tools/lldb/source/Core/CMakeFiles/lldbCore.dir/IOHandler.cpp.o -c /export2/src/llvm-6.0.0/llvm/tools/lldb/source/Core/IOHandler.cpp
/export2/src/llvm-6.0.0/llvm/tools/lldb/source/Core/IOHandler.cpp:15:19: fatal error: panel.h: No such file or directory
#include <panel.h>
^
compilation terminated.

Kind regards

Siegmar

Hans_Wennborg · February 16, 2018, 10:48am

today I've tried to build llvm-6.0.0rc2 using Cmake on my "SUSE Linux
Enterprise Server 12.3 (x86_64)" with the following commands (gcc-6.4.0
is necessary for CUDA-9.0).

wget http://prereleases.llvm.org/6.0.0/rc2/llvm-6.0.0rc2.src.tar.xz
wget http://prereleases.llvm.org/6.0.0/rc2/cfe-6.0.0rc2.src.tar.xz
wget http://prereleases.llvm.org/6.0.0/rc2/clang-tools-extra-6.0.0rc2.src.tar.xz
wget http://prereleases.llvm.org/6.0.0/rc2/compiler-rt-6.0.0rc2.src.tar.xz
wget http://prereleases.llvm.org/6.0.0/rc2/lldb-6.0.0rc2.src.tar.xz
wget http://prereleases.llvm.org/6.0.0/rc2/lld-6.0.0rc2.src.tar.xz
wget http://prereleases.llvm.org/6.0.0/rc2/polly-6.0.0rc2.src.tar.xz
wget http://prereleases.llvm.org/6.0.0/rc2/openmp-6.0.0rc2.src.tar.xz

tar xf llvm-6.0.0rc2.src.tar.xz
cd llvm-6.0.0rc2.src/tools
tar xf ../../cfe-6.0.0rc2.src.tar.xz
tar xf ../../polly-6.0.0rc2.src.tar.xz
tar xf ../../lldb-6.0.0rc2.src.tar.xz
tar xf ../../lld-6.0.0rc2.src.tar.xz
cd cfe-6.0.0rc2.src/tools
tar xf ../../../../clang-tools-extra-6.0.0rc2.src.tar.xz
cd ../../../projects
tar xf ../../compiler-rt-6.0.0rc2.src.tar.xz
tar xf ../../openmp-6.0.0rc2.src.tar.xz
cd ../..

After this, the directory names will not be correct, unfortunately. You'll need to do this instead:

tar xf llvm-6.0.0rc2.src.tar.xz
tar xf cfe-6.0.0rc2.src.tar.xz
tar xf polly-6.0.0rc2.src.tar.xz
tar xf lldb-6.0.0rc2.src.tar.xz
tar xf lld-6.0.0rc2.src.tar.xz
tar xf clang-tools-extra-6.0.0rc2.src.tar.xz
tar xf compiler-rt-6.0.0rc2.src.tar.xz
tar xf openmp-6.0.0rc2.src.tar.xz

mv cfe-6.0.0rc2.src llvm-6.0.0rc2.src/tools/clang
mv polly-6.0.0rc2.src llvm-6.0.0rc2.src/tools/polly
mv lldb-6.0.0rc2.src llvm-6.0.0rc2.src/tools/lldb
mv lld-6.0.0rc2.src llvm-6.0.0rc2.src/tools/lld
mv clang-tools-extra-6.0.0rc2.src llvm-6.0.0rc2.src/tools/clang/tools/extra
mv compiler-rt-6.0.0rc2.src llvm-6.0.0rc2.src/projects/compiler-rt
mv openmp-6.0.0rc2.src llvm-6.0.0rc2.src/projects/openmp

I think it might be better to distribute the source tarballs all rooted at llvm-6.0.0rc2, with the correct subpaths already embedded. Hans, what are your thoughts about that?

I'm not sure. I agree the current situation is impractical, but some
folks like to build the packages in separate locations rather than
under tools/ or projects/ so I'm not sure embedding those sub-paths is
a good idea.

Hopefully this will become moot at some point, if the monorepo ever gets off the ground...

Fingers crossed