Question about the build

Sorry to keep posting, but I'm hoping continued evidence will enable someone to understand what is going on.

If I touch a lib/TableGen file and then build bin/llvm-tblgen.exe, Ninja rebuilds build/bin/llvm-tblgen.exe

If I touch a file and then build lib/Target/all, Ninja rebuilds both build/bin/llvm-tblgen.exe and build/NATIVE/bin/llvm-tblgen.exe. In neither case does it rebuild any .inc files.

This seems like a bug in LLVM_OPTIMIZED_TABLEGEN. What you are describing works in the standard configuration.

Thank you, sir! Indeed, -DLLVM_OPTIMIZED_TABLEGEN=ON was the problem. Everything is back to normal with =OFF.

For what it’s worth, I’ve recently been running into problems related to the NATIVE tablegen build for optimized tablegen usage in debug builds. In my case, it was (and may still be) unnecessarily rebuilding tablegen related files every time, even though nothing has changed, which in turn causes the rest of the build to trigger (as Visual Studio sees things as being out-of-date). I forget the exact details, but can dig them out later if someone is interested in investigating further. I haven’t had the time to dig into this properly myself, but it is frequently hurting my productivity.

I was having the opposite problem: Building the native TableGen did not trigger downstream rebuilds.

Could you see if the problem is still happening and then post some details here? I know almost nothing about the build process, but I can investigate whether TableGen is unnecessarily rewriting identical output files.

I’m not sure what sort of details exactly you want. I use Visual Studio 2017 to build my LLVM checkout. The LLVM_OPTIMIZED_TABLEGEN setting is enabled, and when I build the Debug build it always complains that a number of the projects that use tablegen are out of date. Specifically, if I turn up the build log verbosity to “Detailed”, I get a large number of projects which say something like this:

23>------ Up-To-Date check: Project: utils\TableGen\LLVM-tablegen-host.vcxproj, Configuration: Debug x64 ------

23>Project is not up-to-date: build input ‘c:\llvm\build\native\release\bin\llvm-tblgen’ is missing.

Subsequently, tablegen runs and emits “Building Options.inc…” and similar lines for all tablegen inputs, rather than doing nothing, resulting in virtually the whole build being run ultimately.

Note that c:\llvm\build\native\release\bin\llvm-tblgen.exe exists and the project build to build that executable says there is nothing to update. When investigating this last year, I found that applying a small change to CrossCompile.cmake to add “.exe” to the end of the output_path for the optimized tablegen output was sufficient to fix things, but I don’t think that’s a realistic long-term solution, as it’s not cross-platform.

I don’t really know enough about cmake or tablegen to have any further clue what to do to fix things.

James

To just get the extension, one can make use of CMAKE_EXECUTABLE_SUFFIX
(https://cmake.org/cmake/help/latest/variable/CMAKE_EXECUTABLE_SUFFIX.html)
within cmake. The idiomatic solution within the same cmake build would
be to use the `$<TARGET_FILE:llvm-tblgen>` generator expression.

Additionally, tblgen-gen should check whether content of the target
file is identical before overwriting an existing file, such that its
time stamp does not change and dependent files do not need to be
rebuilt. This also seems to not always work, but I don't know whether
this is a limitation of Visual Studio/MSBuild, e.g. does not re-check
whether files in a dependency chain actually have changed. In my
experience, it is often that the mlir-tablegen dependees are rebuilt
regardless, but clang-tblgen and llvm-tblgen seem to work.

Michael