cc1 problems when loading multiple AST files

Hi,

The scene of my usage is I want to analyze multiple source code files with CSA and CTU enabled. Since it will report a header file not found error when using clang -cc1 to analyze source code files (*.cpp), I turn to AST files for simplicity.

Here is the bug. If I only analyze a single AST file, it works fine. However, when I append multiple AST files for analysis, it crashes all the time. It has been reproduced both on my fedora-29 desktop and ubuntu-18.04 server with the code from git commit e03301a3b32.

The code I use to trigger this bug is provided in the attachments. See core dump file (./coredump) for more details. According to the stack trace, I think the bug is not in CSA and CTU.

Besides, could you please tell me how to make cc1 to parse AST with the compile commands provided in the compile_commands.json file? Or make clang-check enable CTU IPA? I did not find any code to enable CTU IPA in clang-check, and if it is useful to others, I would like to share my customized clang-check.

Regards,
Ella

P.S. I do not know whether it is allowed to ask a question together with a bug report here. Sorry for annoying you.

trigger.tar.xz (3.43 MB)

Hi,

I probably won't be able to help with the crash.

Generally, the trivial way to obtain a Static Analyzer invocation from a compilation database (assuming the project is already compiled with clang) is to append `--analyze` to it. Then you can convert it to a cc1 run-line via `-###`. Clang, on its own, doesn't know how to read compilation databases; it's supposed to work the other way round.

You also cannot construct a single cc1 run-line that would conduct cross-translation-unit analysis. CTU is, by definition, a collaborative effort of multiple clang instances; in order to analyze a single file, you'll need the *whole* compilation database. That said, if you already have precompiled ASTs and function map files, you should be able to invoke clang -cc1 on a single file just by passing the correct ctu_dir; it should be enough to reproduce the crash or to debug it.

I think right now the "official" (in-tree, but still mostly unmaintained) way of invoking CTU is via scan-build-py, which is an effort to rewrite scan-build in python. It also happily consumes compilation databases. On the other hand, CTU developers mostly target the out-of-tree Ericsson CodeChecker tool for this purpose.

Hope this slightly helps.

Hi Ella,

../bin/clang -cc1 -analyze -analyzer-checker=core,debug.ExprInspection -o reports/ home/ella/workspace/llvm-project/build/analyzer-test/src/getzero.cpp.ast home/ella/workspace/llvm-project/build/analyzer-test/src/test.cpp.ast

The way you executed clang does not merge the AST files. (Still, this
is probably an error, but rather in ASTUnit. Do you get this error
when you pass only one .ast file as a parameter?)

The golden way to achieve what you want, is to use CodeChecker
(https://github.com/Ericsson/codechecker/).
With it you can log the compile commands from a build process (CodeChecker log).
Once you have the compile_commands.json, you can configure the
analysis to properly create all the PCH files (--ctu-collect) and it
will execute the analysis with ctu enabled (--ctu-analysis).
https://github.com/Ericsson/codechecker/blob/master/docs/analyzer/user_guide.md#ctu

Note that the Ericsson Clang fork is far more stable and usable with
the CTU analysis of C++ projects, you should try it:
https://github.com/Ericsson/clang

Gabor

The way you executed clang does not merge the AST files. (Still, this
is probably an error, but rather in ASTUnit. Do you get this error
when you pass only one .ast file as a parameter?)

No, I did not.
Besides, I also think checking files one by one can also be another choice, since I will always enable CTU analysis.

And thank you for your suggestions of Ericsson’s work.

Regards,
Ella