Hi,
Several clang analysis test cases written in ObjC contain the following comment line in their header: "These declarations were reduced using Delta-Debugging from Foundation.h on Mac OS X.". Are there some instructions about tools which should be used to generate such reduced headers? Being able to do so would make it possible to report bugs by using runnable test cases, and those test cases could be eventually included with the clang itself.
Hi,
Several clang analysis test cases written in ObjC contain the
following comment line in their header: "These declarations were
reduced using Delta-Debugging from Foundation.h on Mac OS X.". Are
there some instructions about tools which should be used to generate
such reduced headers?
If I'm not mistaken, the steps are just to generate a full file with
clang -E, then reduce the file using Delta (http://delta.tigris.org/).
But Ted should be able to comment with more complete steps.
Being able to do so would make it possible to
report bugs by using runnable test cases, and those test cases could
be eventually included with the clang itself.
Reducing out Foundation.h only really helps for non-runnable
testcases; the presence of Foundation.h is a pretty good indicator of
the presence of the Foundation classes. 
-Eli
Hi,
Several clang analysis test cases written in ObjC contain the
following comment line in their header: "These declarations were
reduced using Delta-Debugging from Foundation.h on Mac OS X.". Are
there some instructions about tools which should be used to generate
such reduced headers?If I'm not mistaken, the steps are just to generate a full file with
clang -E, then reduce the file using Delta (http://delta.tigris.org/).
But Ted should be able to comment with more complete steps.
That's exactly right.
The Delta tool requires a user-provided script that discerns a "test case" between good and bad, and tries to remove pieces from the test case (which is any text file) until a minimal test case is generated that is still "good".
In the case of header file reduction, I'm working with two files:
t.i (the preprocessed header goop that I want to reduce)
t.c (the file that includes t.i)
The "test case" is t.i, and the my script (called "detect_error.pl") classifies a version of t.i as good or bad by doing the following:
1) If "gcc -fsyntax-only t.c" fails, the test case is BAD.
2) if "clang -fsyntax-only t.c" fails, the case is BAD.
3) Otherwise, the test case is GOOD.
The trick is identifying what is failure: gcc emiting warnings, gcc or clang crashing, etc.
Delta-debugging then whittles down t.i until it cannot make it any smaller. Once it reduces t.i, I just include the contents of t.i inside t.c (or t.m, or whatever).
Being able to do so would make it possible to
report bugs by using runnable test cases, and those test cases could
be eventually included with the clang itself.
I can certainly clean up the detect_error.pl script and post it the website with directions.
Reducing out Foundation.h only really helps for non-runnable
testcases; the presence of Foundation.h is a pretty good indicator of
the presence of the Foundation classes.
Exactly. In general, header reduction might be good for filing any kind of test case against Clang that depends on system/library-specific APIs.
Here is a small guide to use delta to reduce test cases. The guide is GCC specific but you can use it to understand how to use delta to reduce clang test cases.
I'm not Ted :), but I've attached 2 script that I regularly use. I have more, which have other combinations.
For example, imagine you want to reduce a test case that crashes clang. After the preprocessing, you would run (several times):
multidelta -level=x ./crash_clang a.c
the level (x) is a number between e.g. 0 and 10. Check the delta manual for how to best pick up a number.
Now suppose clang triggers a error that gcc does not. Then you can use the clang_error script (provided you changed it to grep for your specific error).
Nuno
clang_error (319 Bytes)
crash_clang (231 Bytes)
Here is my scripts to add to the pile. Once must change the source file in "detect_error.pl" (this is the source that includes the header to be reduced) while "reduce.pl" takes as an argument the preprocessed header you want to reduce (this file is included by the source file).
Both scripts rely on the delta-debugging scripts being in your path.
detect_error.pl (1.29 KB)
reduce.pl (331 Bytes)
Thanks Ted and Nuno!
I'll try these scripts and report any results (hopefully by submitting some reduced test cases) in couple of days.
I've been able to reduce headers in a simple test case with detect_error.pl and reduce.pl scripts very easily (some manual clean-up of result source code was required, but it was pretty minor), so at least for me those scripts were useful. I've submitted the test case as bug 2542 (NSAnimation self-ownership).
I also saw that Nuno posted some scripts. Hopefully from all these scripts you'll find something that works for you. If these scripts prove useful, we should post them to the Clang website.
I've been able to reduce headers in a simple test case with detect_error.pl and reduce.pl scripts very easily
Cool!
(some manual clean-up of result source code was required, but it was pretty minor),
I have to do this every time as well. I'm sure that cleanup work could be automated one day as well.
I've submitted the test case as bug 2542 (NSAnimation self-ownership).
Wonderful test cases! I've posted some questions on the Bugzilla report.