How to test my pass

Hi there,

I have written a PRE pass using LLVM. How can I test my pass? Is
there any standard test-cases or bench-marks to test a pass. Please
suggest accordingly.

Regards,
Chayan

I have written a PRE pass using LLVM. How can I test my pass? Is
there any standard test-cases or bench-marks to test a pass. Please
suggest accordingly.

Try running it on the LLVM testsuite:
   http://llvm.org/svn/llvm-project/test-suite/trunk

Ciao,

Duncan.

Chayan Sarkar wrote:

Hi there,

I have written a PRE pass using LLVM. How can I test my pass? Is
there any standard test-cases or bench-marks to test a pass. Please
suggest accordingly.

Use the LLVM nightly test suite: http://llvm.org/docs/TestingGuide.html#testsuiterun

Make a copy of an existing test like TEST.example.Makefile and modify it to run 'opt -yourpass -time-passes -stats', then 'make TEST=yourexample report' and read the report.raw.out it produces. Look for crashes first and foremost, then look at how many times it actually modified the user code, then finally make sure it didn't take show absurd compile-time problems. Fix all the crashers and O(n^2) behaviour.

Now you're ready to try your pass on real code. Modify include/llvm/Support/StandardPasses.html to add your pass to the list of defaults. Go back to the nightly test suite and 'make TEST=nightly report.html' which will not only use your pass when building the tests, but actually run the resulting executables and verify their results. Most of the * cells in the resulting HTML will be as a result of your pass, but not all. Your first step in debugging them is to cd to the correct source directory and run 'make bugpoint-opt'[1] to get a reduced testcase. Fix all the crashers and miscompiles.

If you have access to SPEC, that's also a great source of benchmarks. You should certainly try it out over spec if you at all possibly can, since spec turns up all sorts of miscompiles that the llvm test suite doesn't.

Finally, do a run of the nightly test with your pass enabled, and a reference run without it. How much did you pass really speed the programs up? How much did it slow the compiler down? Is your pass worth it to our users? If so, it's time to commit!

Nick

[1] In MultiSource/... that does the right thing, in SingleSource/... it will print out a message telling you what you should really run.

Thank you so much....