How to run the unit test for contributing a new clang-tidy check to LLVM?

Beginners
,
1

I am currently following the guide at Getting Involved — Extra Clang Tools 20.0.0git documentation. I built llvm with the flag clang-tools-extra, made my check with add_new_check.py, did ninja clang-tidy, and running ./bin/clang-tidy --checks=-*,bugprone-public-enable-shared-from-this ../clang-tools-extra/test/clang-tidy/checkers/bugprone/public-enable-shared-from-this.cpp works as expected.

Now, the problem becomes: how do I test out my check using the expected automated framework that upstream wants? Is it using check-clang-tidy.py, or is it using ./bin/llvm-lit? I can’t seem to run check-clang-tidy.py. If using llvm-lit, how can I pass in the header?

I’ve read and tried:

https://clang.llvm.org/extra/clang-tidy/Contributing.html

  • check-clang-tidy.py doesn’t work despite me giving it the input_file_name, check name, and output file. It just says no checks enabled and the python file shows this error
    subprocess.CalledProcessError: Command '['clang-tidy', 'temp_file.cpp', '-fix', '--checks=-*,bugprone-public-enable-shared-from-this', '--config={}', '--', '-std=c++11', '-nostdinc++']' returned non-zero exit status 1.

  • ./bin/llvm-lit /llvm-project/clang-tools-extra/test/clang-tidy/checkers/bugprone/public-enable-shared-from-this.cpp -v shows it fails because

Running ['clang-tidy', '/llvm-project/build/tools/clang/tools/extra/test/clang-tidy/checkers/bugprone/Output/public-enable-shared-from-this.cpp.tmp.cpp', '-fix', '--checks=-*,bugprone-public-enable-shared-from-this', '--config={}', '--', '-I', '/llvm-project/clang-tools-extra/test/clang-tidy/checkers/bugprone/Inputs/', '-std=c++11', '-nostdinc++']...
clang-tidy /llvm-project/build/tools/clang/tools/extra/test/clang-tidy/checkers/bugprone/Output/public-enable-shared-from-this.cpp.tmp.cpp -fix --checks=-*,bugprone-public-enable-shared-from-this --config={} -- -I /llvm-project/clang-tools-extra/test/clang-tidy/checkers/bugprone/Inputs/ -std=c++11 -nostdinc++ failed:
1 error generated.
Error while processing /llvm-project/build/tools/clang/tools/extra/test/clang-tidy/checkers/bugprone/Output/public-enable-shared-from-this.cpp.tmp.cpp.
/llvm-project/build/tools/clang/tools/extra/test/clang-tidy/checkers/bugprone/Output/public-enable-shared-from-this.cpp.tmp.cpp:2:10: error: 'memory' file not found [clang-diagnostic-error]
    2 | #include <memory>
      |          ^~~~~~~~
Found compiler errors, but -fix-errors was not specified.
Fixes have NOT been applied.

and

subprocess.CalledProcessError: Command '['clang-tidy', '/llvm-project/build/tools/clang/tools/extra/test/clang-tidy/checkers/bugprone/Output/public-enable-shared-from-this.cpp.tmp.cpp', '-fix', '--checks=-*,bugprone-public-enable-shared-from-this', '--config={}', '--', '-std=c++11', '-nostdinc++']' returned non-zero exit status 1
  • lit %check_clang_tidy %s bugprone-public-enable-shared-from-this also didn’t work; it says no test suite detected.

Other things I’ve tried:

  • ninja check-clang-tools … seems to build everything, so I don’t know if my test actually succeeded? and it fails on linking clangd for some reason.

I read through lit - LLVM Integrated Tester — LLVM 20.0.0git documentation which does explain the commented part in the test file…but not how to run lit in the command line for this singular test.

LLVM Testing Infrastructure Guide — LLVM 20.0.0git documentation this explains the meanings of %s…but it doesn’t touch running clang tidy tests with check-clang-tidy.py, which I assume is upstream’s way of automatically testing the file.

Contributing to LLVM — LLVM 20.0.0git documentation which only says “include a small unit test” and doesn’t say anything about running said unit test.

  • ninja check-all didn’t work and (coincidentally, apparently) crashed the infra at my workplace.

The following is a snippet of the test file public-enable-shared-from-this.cpp:

// RUN: %check_clang_tidy None%s bugprone-public-enable-shared-from-this %t
#include <memory>

class BadExample : std::enable_shared_from_this<BadExample> {
// CHECK-MESSAGES: :[[@LINE-1]]:7: warning: class BadExample is not public even though it's derived from std::enable_shared_from_this [bugprone-public-enable-shared-from-this]
// CHECK-FIXES: :[[@LINE-2]]:19 public 
        public:
        BadExample* foo() { return shared_from_this().get(); }
        void bar() { return; }
};

void using_not_public() {
        auto bad_example = std::make_shared<BadExample>();
        auto* b_ex = bad_example->foo();
        b_ex->bar();
}

class GoodExample : public std::enable_shared_from_this<GoodExample> {
        public:
        GoodExample* foo() { return shared_from_this().get(); }
        void bar() { return; }
};

void using_public() {
        auto good_example = std::make_shared<GoodExample>();
        auto* g_ex = good_example->foo();
        g_ex->bar();
}

And this was the expected output as given by running clang-tidy manually:

/llvm-project/build/../clang-tools-extra/test/clang-tidy/checkers/bugprone/public-enable-shared-from-this.cpp:4:7: warning: class BadExample is not public even though it's derived from std::enable_shared_from_this [bugprone-public-enable-shared-from-this]
    4 | class BadExample : std::enable_shared_from_this<BadExample> {
      |       ^
      |                    public 
Suppressed 2 warnings (2 with check filters).
Found compiler error(s).

As you can see, I’ve tried my best to do my own research, but I was quickly overwhelmed. Advice, pointers, or a guide to how to run llvm-lit for clang-tidy tests would be appreciated.

(Side note, but this would be my first open source contribution and interaction with the community; I read through some open community guides re: Googling but if there’s any pitfalls I may not be aware of, feel free to tell me.)

2

Update: calling ./bin/llvm-lit seems to work on other standard checks. Seems like the issue here is actually the header rather than anything on the commented RUN line.