I have a pass called “MyPass”. And I can run this pass in the LLVM test suite programs and produce a report with the statistics of the execution of this pass. Now, I would like to run the programs in LLVM test suite and generate a report comparing the results obtained without using the pass with the results obtained using the pass. Is there any way to do this?
Yes, you can use the "llc-beta" test suite option to do what you're looking for. That runs the tests with and without a set of command line options and provides a comparison.
1. Make your pass disabled by default and enabled via a command line option (e.g., --enable-mypass),
2. Tell the test-suite Makefiles to run your pass as the beta option*,. Look in <llvm/projects/test-suite/Makefile.programs*,
3. Run the testsuite in night test mode (the "simple" mode does not support beta options at this time) via something like "make TEST=nightly report.html",
4. Open the report to see your results.
* Look in <llvm/projects/test-suite/Makefile.programs. The options are enabled by target, search for "BETAOPTION" to find them. You'll see examples for setting the LLC beta option via LLCBETAOPTION. If your pass is a machine function pass, that's where you'll want to add it. If it's run by 'opt' instead, you'll want to use OPTBETAOPTION.