Running Clang-Tidy on a Large Project

I ran clang-tidy in the root directory of the source code of an application (MPlayer-1.3.0). Precisely, I use run-clang-tidy.py python script, as follows:

run-clang-tidy.py -header-filter='.*' -checks='-*,readability-braces-around-statements' -fix

I ran clang-tidy in the root directory of the source code of an application (MPlayer-1.3.0). Precisely, I use run-clang-tidy.py python script, as follows:

run-clang-tidy.py -header-filter='.*' -checks='-*,readability-braces-around-statements' -fix

The commands database is also stored in the root directory in a file named compile_commands.json. After gathering all fixes, it tries to apply them but no fix is applied on any of the source files compiled from the inner directories. Here is the first part of the error report:

Applying fixes ...
Described file './libavutil/internal.h' doesn't exist. 
Ignoring...
Described file './libavutil/x86/intmath.h' doesn't exist. 
Ignoring...
Described file 'libavformat/internal.h' doesn't exist. 
Ignoring...
Described file './libavcodec/bytestream.h' doesn't exist. 
Ignoring...
Described file './libavcodec/flac.h' doesn't exist. 
Ignoring...
Described file './libavcodec/get_bits.h' doesn't exist. 
Ignoring...
Described file './libavcodec/internal.h' doesn't exist. 
Ignoring...
Described file './libavcodec/mathops.h' doesn't exist. 
Ignoring...
Described file './libavcodec/put_bits.h' doesn't exist. 
Ignoring...
Described file 'libavformat/matroskaenc.c' doesn't exist. 
Ignoring...
Described file 'libavformat/subtitles.h' doesn't exist. 
Ignoring...
Described file 'libavformat/apngdec.c' doesn't exist. 
Ignoring...
...

These files are compiled using the Makefile located in the folder ffmpeg. For example, libavformat/apngdec.c is located at ./ffmpeg/libavformat/apngdec.c where .is the root directory of MPlayer-1.3.0. How can I fix the problem?
It seems that the fixes exported in yaml format stored relative paths. These paths are imported to clang-apply-replacemens which can not recognize the correct prefix for them.

Hi,

Hi Miklos,

The compilation database is correct. It is generated using Bear. It has correct directory (for the build directory) and file (for the file to be compiled) entries.
The tool works correctly with single file. I think because the absolute path is recreated in that case. It can be seen at “clang-extra-tools-source/clang-tidy/ClangTidy.cpp:line131” (clang-extra-tools-6.0.0):

SmallString<128> FixAbsoluteFilePath = Repl.getFilePath();

Files.makeAbsolutePath(FixAbsoluteFilePath);


But the same operation is not performed when the information is exported to yaml, which is done when I run “run-clang-tidy” script with “-fix” option.
It seems that the problem with large projects is not new. There was a bug in clang-extra-tools-3.9.0 “run-clang-tidy” script which is revised in the current version. In that version, even clang-tidy was called with relative paths and therefore did not work for source files built from somewhere other than the root directory of the project. Looking forward to your suggestions!
Thanks!

Hi,

path is recreated in that case. It can be seen at "clang-extra-tools-source/
clang-tidy/ClangTidy.cpp:line131" (clang-extra-tools-6.0.0):
...

SmallString<128> FixAbsoluteFilePath = Repl.getFilePath();

Files.makeAbsolutePath(FixAbsoluteFilePath);
...
But the same operation is not performed when the information is exported to
yaml, which is done when I run "run-clang-tidy" script with "-fix" option.

Sounds like if you submit a patch to fix this, it would makes sense. (I
haven't tried run-clang-tidy with -fix myself; what I usually do is
invoking clang-tidy directly with -export-fixes=, then
clang-apply-replacements, but in theory this is the same.)

Perhaps the reason why nobody hit this before is the usage of relative
paths; I think cmake just writes absolute paths in the compile commands
db, so you don't hit the codepath where making the paths absolute
matters.

Regards,

Miklos

Many Thanks! You helped me pinpoint the problem.
The problem is with the compile commands database. In fact, clang-tidy expects the database format to be a little bit restricted (which is the real cause of the problem). The input paths to build commands in MPlayer are all relative (determined by the Makefile) as can be seen here:


“-I/usr/local/lib/clang/3.9.0/include/”,
“-fpie”,
“-DPIC”,
“-D_REENTRANT”,
“-DZLIB_CONST”,
“-o”,
“libavformat/cafdec.o”,
“libavformat/cafdec.c”
],
“directory”: “/home/ahmad/Programs/MPlayer-1.3.0/ffmpeg”,
“file”: “/home/ahmad/Programs/MPlayer-1.3.0/ffmpeg/libavformat/cafdec.c”


This is part of the commands database generated using Bear with the file entry revised to be absolute (I did that). The problem is still there. Because the last two lines of the compile command arguments are relative paths:
“libavformat/cafdec.o”
“libavformat/cafdec.c”
which should be changed to:
“/home/ahmad/Programs/MPlayer-1.3.0/ffmpeg/libavformat/cafdec.o”

“/home/ahmad/Programs/MPlayer-1.3.0/ffmpeg/libavformat/cafdec.c”
It seems that the fastest way to fix the problem is to postprocess the commands database and make all paths absolute.

I revised the compile commands database. But there is still problem with header files. Their path is relative in yaml file. I will try to change clang-tidy source code in order to fix the problem.

Hi
This is what I did, and it worked:

— ClangTidy.cpp 2017-05-17 19:09:47.000000000 +0430
+++ ClangTidyRevised.cpp 2018-07-24 00:31:31.219938476 +0430
@@ -575,9 +575,44 @@
raw_ostream &OS) {
TranslationUnitDiagnostics TUD;
TUD.MainSourceFile = MainFilePath;

  • for (const auto &Error : Errors) {
  • tooling::Diagnostic Diag = Error;
  • FileManager *Files = new FileManager(FileSystemOptions());
  • vfs::FileSystem &FileSystem = *Files->getVirtualFileSystem();
  • auto InitialWorkingDir = FileSystem.getCurrentWorkingDirectory();
  • if (!InitialWorkingDir)
  • llvm::report_fatal_error(“Cannot get current working path.”);

Hi,

Sure! But, not within two weeks. Because I should clone the entire repository and I am not at the university, right now.