I would like to be able to build clang tools/executables from recent source (I’m using llvm-project-llvmorg-14.0.6), initially using visual studio clang tools (v 13.0.1), but thereafter and for newer versions of clang, using these newly built clang tools themself.
However, where I may differ from others who I’ve seen asking for help with similar issues is that I’m attempting to build clang from source as explicitly as possible (i.e. avoiding the cmake machinery that flails around and attempts to automatically find latest versions among multiple installations of toolchains, winsdk and standard library header and lib locations), with as simple, self-contained, and portable/loose directory structure of tools; limiting as much as possible the need for anything to be ‘installed’ and explicitly pointing to everything (one way or another) that ends up being needed.
“Why do it this way?”, you might well ask: Well, it strikes me as a much more clean, portable, and reproducible way of going about things, to explicitly point to a complier, to system include and lib directories, and even to explicitly point to versions of cmake to use, from just one or two simple config/setup steps. There are plenty of people asking for help with issues for which the answer is frequently, “you haven’t installed X properly. Uninstall and reinstall…”, which just strikes me as a quite error prone solution, relying on a lot of ‘flail and find’ cmake build machinery. If a build needs a compiler, linker, various headers, and libs and someone knows exactly where to find them, they should be able to simply explicitly point to them. Plus, this is also a learning excercise for me and something which is not entirely well documented (that I can find).
This whole endevour has had a few hurdles that I document for the possible benefit of others at the bottom, but I also end up at a problem that I can’t get past and would be greatful for some help.
So far, I have the following general directory structure -
c:/dev/ /cmake-3.24.1-windows-x86_64 /bin/... etc. /llvm-project-llvmorg-14.0.6 /build/... mytoolchain_msvcclang.cmake etc. /meson-0.63.1 ninja.exe (v 1.11.0) [msvc llvm x64 bin] clang.exe (v 13.0.1) clang++.exe etc. [windows sdk run-time headers] [msvc toolchain headers] [windows sdk libs]
I believe the cleanest way to glue the everything together for cmake is to specify a toolchain file that explicitly points to the paths of everything needed.
Where ‘mytoolchain_msvcclang.cmake’ contains the following (for now) -
set(CMAKE_SYSTEM_NAME Windows) set(WIN32 1) set(CMAKE_SYSTEM_PROCESSOR x64) set(CMAKE_C_COMPILER "C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/Llvm/x64/bin/clang.exe") set(CMAKE_C_COMPILER_ID "Clang") set(CMAKE_C_COMPILER_FRONTEND_VARIANT "GNU") set(CMAKE_CXX_COMPILER "C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/Llvm/x64/bin/clang++.exe") set(CMAKE_CXX_COMPILER_ID "Clang") set(CMAKE_CXX_COMPILER_FRONTEND_VARIANT "GNU") set(LLVM_HOST_TRIPLE "x86_64" CACHE STRING "" FORCE) set(LLVM_NATIVE_ARCH "x86_64" CACHE STRING "" FORCE) set(LLVM_TARGET_ARCH "x86_64" CACHE STRING "" FORCE) set(CMAKE_MT "C:/Program Files (x86)/Windows Kits/10/bin/10.0.20348.0/x64/mt.exe") set(CMAKE_RC_COMPILER_INIT "C:/Program Files (x86)/Windows Kits/10/bin/10.0.20348.0/x64/rc.exe") set(CMAKE_RC_FLAGS_INIT "/nologo") set(MIDL_COMPILER "C:/Program Files (x86)/Windows Kits/10/bin/10.0.20348.0/x64/midl.exe") set(MDMERGE_TOOL "C:/Program Files (x86)/Windows Kits/10/bin/10.0.20348.0/x64/mdmerge.exe") include_directories(SYSTEM "C:/Program Files (x86)/Windows Kits/10/Include/10.0.20348.0/ucrt") include_directories(SYSTEM "C:/Program Files (x86)/Windows Kits/10/Include/10.0.20348.0/shared") include_directories(SYSTEM "C:/Program Files (x86)/Windows Kits/10/Include/10.0.20348.0/um") include_directories(SYSTEM "C:/Program Files (x86)/Windows Kits/10/Include/10.0.20348.0/winrt") include_directories(SYSTEM "C:/Program Files (x86)/Windows Kits/10/Include/10.0.20348.0/cppwinrt") include_directories(SYSTEM "C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.32.31326/include") link_directories("C:/Program Files (x86)/Windows Kits/10/Lib/10.0.20348.0/ucrt/x64") link_directories("C:/Program Files (x86)/Windows Kits/10/Lib/10.0.20348.0/um/x64") add_definitions(-D_CRT_SECURE_NO_WARNINGS) add_definitions(-D_CRT_USE_BUILTIN_OFFSETOF)
I’ve arrived at this through quite a bit of flailing so suspect one or two things in here could be unnecessary and trimmed out.
I also add the following to my local/temporary PATH envroment -
> set PATH=[...path to msvc llvm x64 bin...];[...path to cmake-3.24.1-windows-x86_64\bin...];[...path to dir containing ninja.exe...];%PATH%
Then I can do -
> cd /llvm-project-llvmorg-14.0.6 > mkdir build > cd build > cmake -DCMAKE_TOOLCHAIN_FILE=../mytoolchain_msvcclang.cmake -DLLVM_ENABLE_PROJECTS=clang -G Ninja -DCMAKE_BUILD_TYPE=Release ../llvm
and this now seems to happily generate everything I need to then fire off -
which happily gets running compiling and building hundreds of steps … before ultimately failing with -
... ninja: build stopped: subcommand failed. FAILED: NATIVE/bin/llvm-tblgen.exe C:/dev/llvm-project-llvmorg-14.0.6/build/NATIVE/bin/llvm-tblgen.exe cmd.exe /C "cd /D C:\dev\llvm-project-llvmorg-14.0.6\build\NATIVE && C:\dev\cmake-3.24.1-windows-x86_64\bin\cmake.exe --build C:/dev/llvm-project-llvmorg-14.0.6/build/NATIVE --target llvm-tblgen --config Release" ninja: build stopped: subcommand failed.
What’s going on here? It looks like there’s a non-ninja / explicit cmake command line step that it’s trying to run. Firstly, it’s not specifying the toolchain file that I suspect I need to use for all cmake steps. Secondly, it looks like it’s invoking cmake with options that are now obsolete.
In other words, if I manually try to run -
C:\dev\llvm-project-llvmorg-14.0.6\build\NATIVE>cmake -DCMAKE_TOOLCHAIN_FILE=../../mytoolchain_msvcclang.cmake --build C:/dev/llvm-project-llvmorg-14.0.6/build/NATIVE --target llvm-tblgen --config Release
it gives me -
CMake Warning: Ignoring extra path from command line: "Release" CMake Error: Unknown argument --config
and ‘cmake --help’ confirms that --config (and --target and --build for that matter) are not valid arguments.
So am I missing some configuration steps that would avoid this problem or is there a bug somewhere in clang’s cmake build configurations (and a fix/workaround)?
Backing up a little bit, prior to the sticking point above, I also encountered the following issues -
- The initial cmake command above resulted in the error, “CMake Error at cmake/modules/CheckAtomic.cmake:53 (message): Host compiler appears to require libatomic, but cannot find it.”, which has been discussed here (c - How to solve libatomic issue in windows? - Stack Overflow) and worked around by making the suggested hack to ‘CheckAtomic.cmake’.
- After the hack/work-around for #1, I then hit, “CMake Warning at cmake/modules/GetHostTriple.cmake:46 (message): unable to determine host target triple”, which seems to have given others the same problem (cmake/config-ix.cmake:401 (get_host_triple) error when trying to build LLVM with CMake on Windows - Stack Overflow) and appears to be worked-around by the addition of ‘set(LLVM_HOST_TRIPLE “x86_64” CACHE STRING “” FORCE)’ to my toolchain file. I don’t know if this is expected or if this fix/work-around is likely to lead to problems later on. However it did seem to allow me to progress.
- The initial cmake command now completes successfully but I notice it spits out, “Native target architecture is x86” even though my toolchain file configures as much as it can to configure or point to x64. I tried adding “-A x64 -Thost=x64” to the cmake command line but that just errors, saying, “Generator Ninja does not support platform specification, but platform x64 was specified”. I hope that the rest of my toolchain configuration will be sufficient for ultimately building clang for the x86_64 target, but if not, does anyone know what addition/change might be necessary?
- After the eventual successful completion of the initial cmake command, the first ‘> ninja’ build failed with -
C:/dev/llvm-project-llvmorg-14.0.6/third-party/benchmark/src/benchmark.cc:172:17: error: static_assert expression is not an integral constant expression static_assert(offsetof(State, error_occurred_) <= ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ C:/Program Files (x86)/Windows Kits/10/Include/10.0.20348.0/ucrt\stddef.h:47:31: note: expanded from macro 'offsetof' #define offsetof(s,m) ((::size_t)&reinterpret_cast<char const volatile&>((((s*)0)->m))) ^ C:/dev/llvm-project-llvmorg-14.0.6/third-party/benchmark/src/benchmark.cc:172:17: note: cast that performs the conversions of a reinterpret_cast is not allowed in a constant expression
which prompted me to add “add_definitions(-D_CRT_USE_BUILTIN_OFFSETOF)” to my toolchain file (along with “add_definitions(-D_CRT_SECURE_NO_WARNINGS)” to silence the thousands of warnings relating to using the non-msvc-secure flavour of many standard functions)