DejaGnu test-suite coverage

Project Infrastructure LLVM Dev List Archives
1

Hello,

After a fun discussion regarding unit testing and coverage metrics, it came up that we don’t have coverage analysis for LLVM proper. With a certain amount of experience in this arena, I set about building a nice overview. I don’t have the toolset and commands fully integrated into the build system, and I’m not sure if doing so would be appropriate (its a somewhat fragile and annoying process, and would interfere with normal development/testing cycles), but I have a snapshot of the coverage posted online here:

http://chandlerc.net/llvm-coverage/

Please feel free to dig around, ask questions, etc. Also, I can very quickly re-calculate coverage if people would like to push up tests. I’ll be happy to regenerate the coverage on request, it only takes an hour or so, and isn’t a lot of actual work.

Hope you enjoy and we can spot a few notably suspect areas with this information.
-Chandler

PS: Yes, Clang is on the list for this, but I need to get a bit more viable setup for it running, and I also want to see if it will fit easily within any build infrastructure.

2

This is pretty spiffy. I’m (pleasantly) surprised we have 81.8% coverage for instcombine. It would be a specific case where it would be good to add regtests to try to get to 100%, given how tricky the code is. It would also be nice if your scripts covered autogenerated code like the tblgen produced files in the backend.

-Chris

3

This is pretty spiffy. I’m (pleasantly) surprised we have 81.8% coverage for instcombine. It would be a specific case where it would be good to add regtests to try to get to 100%, given how tricky the code is. It would also be nice if your scripts covered autogenerated code like the tblgen produced files in the backend.

Done. http://chandlerc.net/llvm-coverage/ is now updated, and coverage dropped to 62.1%, but thats spectacularly skewed, as generated code is by its nature bloated. =]

-Chandler