I have a minimum not-working example here (may be downloaded as a .zip with the button on the top-right), that uses CMake + Ninja to compile a module, test contained in test.cppm, and a module consumer, main.cpp.
CMake is set to generate a compilation database in the build artifacts folder. I have also added a clang-tidy check to CMake, and CMake configures fine. However, when I attempt to build the project, I receive the following error:
[3/4] Building CXX object CMakeFiles\test.dir\main.cpp.obj
FAILED: CMakeFiles/test.dir/main.cpp.obj
"C:\Program Files\CMake\bin\cmake.exe" -E __run_co_compile --tidy=clang-tidy;-checks=-*,readability-*;--extra-arg=-Xclang=-fprebuilt-module-path=D:/Libraries/Desktop/test/build/CMakeFiles/test_lib.dir/;-p=D:/Libraries/Desktop/test/build;--extra-arg-before=--driver-mode=cl --source=D:\Libraries\Desktop\test\main.cpp -- C:\PROGRA~1\MIB055~1\2022\ENTERP~1\VC\Tools\MSVC\1436~1.325\bin\Hostx86\x64\cl.exe /nologo /TP /DWIN32 /D_WINDOWS /EHsc /Ob0 /Od /RTC1 -std:c++20 -MDd -Zi /showIncludes @CMakeFiles\test.dir\main.cpp.obj.modmap /FoCMakeFiles\test.dir\main.cpp.obj /FdCMakeFiles\test.dir\ /FS -c D:\Libraries\Desktop\test\main.cpp
D:\Libraries\Desktop\test\main.cpp:1:8: error: module 'test' not found [clang-diagnostic-error]
import test;
~~~~~~~^~~~
1 error generated.
Error while processing D:\Libraries\Desktop\test\main.cpp.
Found compiler error(s).
ninja: build stopped: subcommand failed.
The command: "C:\Program Files\CMake\bin\cmake.EXE" --build D:/Libraries/Desktop/test/build --target test exited with code: 1
Build completed: 00:00:00.250
Build finished with exit code 1
Failed to prepare executable target with name "undefined"
Even if I run clang-tidy standalone, I receive the same error:
> clang-tidy.exe -checks=readability-* --extra-arg=-Xclang=-fprebuilt-module-path=build\CMakeFiles\test_lib.dir -p=build\ .\main.cpp
1 error generated.
Error while processing D:\Libraries\Desktop\test\.\main.cpp.
D:\Libraries\Desktop\test\main.cpp:1:8: error: module 'test' not found [clang-diagnostic-error]
import test;
~~~~~~~^~~~
Found compiler error(s).
My setup is as follows:
Windows 10
Visual Studio 17.6.2 (cl.exe 19.36.32532.0)
CMake 3.26.4
Ninja 1.11.0
LLVM/Clang toolset 16.0.0
Am I using clang-tidy correctly in the first place, or is this a bug (that has possibly since been fixed)?
It is undefined whether or not this is a bug. I am not sure if I can describe this precisely. The key reason for such failures are that the introduction of modules breaks an important assumption of C++: that the compilation of each TU are independent with other TUs. And we have generally 2 solutions: (1) requires the tools to build the modules implicitly so that we don’t need to dependent on the build tools. (2) requires the users to build the BMI files ahead of time so that the tools can find the BMI files. We are trying to do (1) for clangd. But we’re not sure what we should do in clang-tidy. This is the reason I said it is undefined whether or not this is a bug.
For the practical problem, in our downsteam usage, we choose (2) to build clangd ahead of time and pass compilation options to clang-tidy to tell the position of BMIs. Such options are missing naturally from the compilation database since cmake generate such options in build time instead of configuration time. I think it may be helpful for you to edit the compilation database manually now.
@ben.boeckel How do you think such problems due to the compilation database not contain modules dependency informations? I know this is by design. But if it is possible that there are any other method to help the situation. Like to generate another compilation database after every build.
My understanding is that this ought not to affect compilation, at least, because CMake uses P1689R5, which cl.exe has implemented for a while now, and on Clang, is present since 16.0.0. And of course, when I disable Clang-Tidy, everything works fine.
Perhaps Clang-Tidy also now needs to know about P1689R5, which is right now (I think) only present in clang-scan-deps and Clang itself.
Hmm, which files do you think I should add? There are a lot of build artifacts with modules now, and I am not sure which ones are relevant.
For the record, someone else has also encountered this problem, but has apparently fixed it by adding --extra-arg=-fprebuilt-module-path=..., which I have also done, but does not work for me. Maybe it is a platform issue: that user is using exclusively Clang tools, whereas my compiler is MSVC.
Maybe we are not saying the same thing. I mean the compilation database generated by cmake now is incomplete since the compilation database doesn’t contain the modules dependency information.
Hmm, which files do you think I should add? There are a lot of build artifacts with modules now, and I am not sure which ones are relevant.
For the record, someone else has also encountered this problem, but has apparently fixed it by adding --extra-arg=-fprebuilt-module-path=..., which I have also done, but does not work for me. Maybe it is a platform issue: that user is using exclusively Clang tools, whereas my compiler is MSVC.
It should be enough to add the corresponding -fprebuilt-module-path. You can go to the corresponding directory and see the name of the generated BMI. It should work if its name is test.pcm.
I see what you mean now: given that the compilation database is generated at configure time, whereas the module dependency information is generated at build time. It should be straightforward to get CMake to run the scan-dependencies pass at configure time and insert the relevant information into the compilation database, which is what P1689R5 is all about, anyway: establishing module dependencies without needing to build.
Ah, MSVC uses .ifc. I tried switching compilers to clang-cl.exe and clang++.exe, both 16.0.0, but still run into the same issue, although test_lib.pcm is now generated. At this juncture, I’m not sure what to do, and might disable clang-tidy in my project.
I see what you mean now: given that the compilation database is generated at configure time, whereas the module dependency information is generated at build time. It should be straightforward to get CMake to run the scan-dependencies pass at configure time and insert the relevant information into the compilation database, which is what P1689R5 is all about, anyway: establishing module dependencies without needing to build.
Yeah. But this may not be so straight forward. For example, clangd will monitor the compile_commands.json (the compilation database) and once the compilation database changes, clangd will try to re-index files. And if cmake touch the compilation database every time it builds, it will be a lose for clangd. So this is the reason why I say we may need a new form to pass such informations.
Ah, MSVC uses .ifc . I tried switching compilers to clang-cl.exe and clang++.exe , both 16.0.0, but still run into the same issue, although test_lib.pcm is now generated. At this juncture, I’m not sure what to do, and might disable clang-tidy in my project.
So cmake gives it a different name. Now it should work if its name is test.pcm and it is generated by clang with the same version of the clang-tidy. Maybe you can file an issue or discuss this in cmake community. BTW, from the link you give, it looks like cmake generates the BMI file with no suffix in linux. I am not sure if this is a target difference.
There are thoughts about modules_commands.json. CMake would have per-target instances of these which could be made at the target-collation time (so before artifacts, but after all the relevant scanning).
Note that compile_commands.json reference response files which are written out during the build (in the mentioned collation command), so after a full build, clang-tidyshould be able to pick up the module arguments.
Not in general, no. Some sources may not exist until build-time at all (and if they do exist at configure time, may be outdated).
@ChuanqiXu Is it possible to run clang-tidy after only a cmake configure step? Is it the responsibility of cmake to add the modules to the compile_commands.json or is there another standard that can be used? What are the BMI files that clang-tidy needs?
A quick run of the cmake configure step, I see that there are CXXDependInfo.json files that have cxx-modules with appropriate source file information in them. Is that a file used by clang-tidy?
The issue is that tools like cmake-pre-commit-hooks have to be adapted and some tips on how to
I don’t know cmake a lot. Maybe @ben.boeckel can help here. As far as I know, it is impossible to run clang-tidy after the configuration of cmake. Since the dependency information is generated during run time.
How does this one work? Is it on the level of cmakelists or source file? Why isn’t the same issue occurring on header files?
@ChuanqiXu but on the clang side what files are needed? If I want to manually test clang-tidy with modules (even after a build when all dependencies are resolved) how can I get it working in either: compile_commands.json, native cmake module files, or anything else? Hooking it through cmake doesn’t work because it fails to build, so a rundown on the manual approaches would be helpful
We need BMI (.pcm) files and the command line arguments that leads to the paths of the BMI files. It should be easy to reproduce this with a hello world example.
I have created a MWE example so we can be on the same page of what is expected to work. According to @ben.boeckel, running clang-tidy with CMAKE_EXPORT_COMPILE_COMMANDS=ON after a build is supposed to work, but that doesn’t work in my mwe (reference). @ChuanqiXu it is also point clang-tidy to .pcm files. Can it not recursively search for any pcm files in the build directory?
No, we won’t search recursively. Although it may not be hard to implement, it looks bad to do so.
BTW, I suggest to start with a hello world example without cmake. I mean a hello world example with 2 files and a manually written compile_commands.json. Personally, I feel this may be simpler.
Sure, but manual compile_commands.json is not meant for production right? Feel free to add patches and suggestions there to get examples of what works and what not. For me I have waited for the cmake support before I was able to dable in the modules, so it’s preferable if at least there can be a mwe of how to hack in the clang-tidy in a working cmake project.
D:\Libraries\Desktop\test
> clang-tidy.exe -checks="-*,readability-*" --extra-arg=-Xclang=-fprebuilt-module-path=$pwd -p=$pwd .\main.cpp
1 warning generated.
Suppressed 1 warnings (1 in non-user code).
Use -header-filter=.* to display errors from all non-system headers. Use -system-headers to display errors from system headers as well.
D:\Libraries\Desktop\test
>
Is this the file you were referring to @ben.boeckel, or is it supposed to be within compile_commands.json? Is there a missing flag in order to export the .pcm commands there?
I did confirm that --extra-arg=-fprebuilt-module-path=... also works, but I don’t suppose there’s an equivalent of CMAKE_Fortran_MODULE_DIRECTORY for cxx modules. So that’s out of the question.
No, the required information to use just compile_commands.json for clang-tidy is not available yet.
They’re “there”, but in response files that do not exist until the build occurs.
These are internal-to-CMake files. I would recommend that no one try to read them for meaningful information as there is absolutely no compatibility guarantee to what they contain.
I don’t see pre-commit hooks as vital because git commit -n exists for anyone that wants to skirt “rules” about using them and CI should be the canonical “does what we care about work well?” arbiter anyways (as I may be running some whack-a-doodle LFS setup that no one actually cares about working besides me).
It does happen with header files that get generated at build time. The #include lines just don’t resolve in such cases after just a configure.
Today, I would run a full build and then give compile_commands.json to clang-tidy. Note that any leaf EXCLUDE_FROM_ALL targets must be built manually or clang-tidy will be sad about those bits not existing.
The .pcm information is included in a per-TU .modmap files which are fed to clang as response files.
Where is the “response files”? The compile_commands.json is not updated, and only. I can add a print print step on the GH action to show that there is no difference between a configurePreset run or a workflowPreset run (with build step). Could you check mwe cmake file and preset to confirm this?
Well, I’m in the other camp and I want the clang-format, clang-tidy, etc. to run only on CIs in order to keep the cmakelists file simple and with minimum requirements. But that’s neither here nor there, since getting them to work in cmake is also challenging atm.
You mean header files created by add_custom_commands and such? Wouldn’t that affect normal clang-tidy run as well (external clang-tidy not linked to cmake targets). Or is there a way to generate a header file from module files?
So what’s the plan for this one? would cmake create a new standard of modules_commands.json, or are there already standards that have just not yet been implemented in cmake? Is there a blocking issue somewhere?