Hi,
I'm doing daily runs of the analyzer on the DragonFly BSD source code. Find the results at http://yoyodyne.ath.cx/ccc-analyze/ if you're interested.
A couple of questions:
* There are a few crashes reported in the buildworld "Analyzer Failures" section. Should I file individual bug reports for those?
* Could the "Analyzer Failures" be made sortable by arbitrary column as well (like "Reports")?
* Interestingly, the "null dereference" count went down from 308 in http://yoyodyne.ath.cx/ccc-analyze/buildkernel/2008-08-10-1/ to 119 in http://yoyodyne.ath.cx/ccc-analyze/buildkernel/2008-08-21-2/ (we fixed a couple but not that many). Some of them were valid bugs, according to one guy from our team.
Regards,
Sascha
Hi,
I'm doing daily runs of the analyzer on the DragonFly BSD source code.
Find the results at http://yoyodyne.ath.cx/ccc-analyze/ if you're
interested.
A couple of questions:
* There are a few crashes reported in the buildworld "Analyzer Failures"
section. Should I file individual bug reports for those?
Hi Sascha,
Feel free to file one bugzilla report with an attached tar.bz2 of your crashes directory.
* Could the "Analyzer Failures" be made sortable by arbitrary column as
well (like "Reports")?
Sure thing. I'll try and include that fix in the next checker release.
* Interestingly, the "null dereference" count went down from 308 in
http://yoyodyne.ath.cx/ccc-analyze/buildkernel/2008-08-10-1/ to 119 in
http://yoyodyne.ath.cx/ccc-analyze/buildkernel/2008-08-21-2/ (we fixed a
couple but not that many). Some of them were valid bugs, according to
one guy from our team.
I've been trying to gradually add more path-sensitivity to the analyzer to suppress false positives. If you find that some bugs are disappearing, if you feel up to it, *please* submit preprocessed sources of the files that use to get flagged with errors and no longer are. (you can file these as bugzilla reports). Also please include the checker-# (both for when the bugs were originally found and when they disappeared). In general it's hard for us to detect false negatives; the only way to guard against this is to gradually build up a bigger regression test suite from bug reports like these.
Ted
Hi Sascha,
Another useful thing is if people want to provide us preprocessed sources that the analyzer correctly identified bugs in. This would allow us to incrementally add valid bug reports to our test suite.
This is something that we can actually add to scan-build. For every source file that the analyzer flags an error, we can also automatically generate preprocessed .i/.mi files. Users can then submit these to us if they wish for either the case that the bug is a false positive or that it’s valid. Building up a test suite of validated bugs would be invaluable.
Ted
I could send you some examples where clang didn't identify a bug
(another checkers found the bug, or it was discovered during a manual
code review).
Would that be useful?
Best regards,
--Edwin
Hi Edwin,
Those are definitely extremely useful, and would be very much appreciated. I want to build up a test suite of known bugs that we both currently can find and should try to find in the future. Preprocessed files are preferably, along with the target information needed to reproduce the error (e.g., 32-bit versus 64-bit, etc.).
I'm also very concerned about errors that we were finding before that we are no longer finding. Those are either regressions in the analyzer or in clang's parser/semantic analyzer.
Thanks!
Ted