Handling static analyzer diagnostics

Hi all,

I've been playing around with the Static Analyzer in an external code tree and
came up with a couple of tests based upon GoogleTest. Inspired by the unit
tests found in the clang source tree I came up with a PathDiagnosticConsumer
attached at the end of this mail.

All that works. However, running the test suite also results in printing the
warning to stderr which somehow clutters the output of a test suite run. I'm
wondering whether it is possible to silence a warning which I handle via my
DiagConsumer explicitly whereas all other warnings are still printed to stderr?

Cheers,
Stefan

class DiagConsumer : public PathDiagnosticConsumer {
  llvm::raw_ostream &Output;
public:
  DiagConsumer(llvm::raw_ostream &Output) : Output(Output) {}
  void FlushDiagnosticsImpl(std::vector<const PathDiagnostic *> &Diags,
                            FilesMade *filesMade) override {
    for (const auto *PD : Diags)
      Output << PD->getCheckerName() << ":" << PD->getShortDescription() << '\n';
  }
  StringRef getName() const override { return "Test"; }
};

class TestAction : public AnalysisAction {
  llvm::raw_ostream &DiagsOutput;
public:
  TestAction(llvm::raw_ostream &DiagsOutput) : DiagsOutput(DiagsOutput) {}
protected:
  std::unique_ptr<clang::ASTConsumer> CreateASTConsumer(clang::CompilerInstance &CI,
                                                        clang::StringRef InFile) override {
    std::unique_ptr<AnalysisASTConsumer> AnalysisConsumer = CreateAnalysisConsumer(CI);
    auto analyzerOpts = CI.getAnalyzerOpts();
    analyzerOpts->CheckersAndPackages = {{"foobar", true}};
    AnalysisConsumer->AddCheckerRegistrationFn([](CheckerRegistry &Registry) {
      Registry.addChecker<Foobar>("foobar", "", "");
    });
    AnalysisConsumer->AddDiagnosticConsumer(new DiagConsumer(DiagsOutput));
    return std::move(AnalysisConsumer);
  }
};

TEST(X, Y) {
  std::string Diags;
  llvm::raw_string_ostream OS(Diags);
  auto Code = /* ... */
  EXPECT_TRUE(tooling::runToolOnCode(std::make_unique<TestAction>(OS), Code, "FileName.c"));
  EXPECT_EQ(Diags, "foobar:baz\n");
}

AnalysisConsumer automatically populates PathDiagnosticConsumers according to AnalyzerOptions and the default is not empty; you might as well be writing html or plist files with your tests (the actual default is different depending on how everything is invoked and i'm not sure what it is in your case). You might have to implement a way to clear the consumers before adding your consumer.

Why do you have to go that far though? Are llvm-lit tests not an answer to everything?

Good point. I wasn't aware of llvm-lit and how easy it is to use in
out-of-tree projects. I just gave it a try and it works wonderful
(especially in conjunction with VerifyDiagnosticConsumer which makes
testing for new diagnostics very easy).

Thanks for the hint,
Stefan