Hi jkv! Very interesting, thanks for sharing that.
LLVM has the benefit of allowing us to tie directly to the abstract syntax tree representation and instrument during LLVM IR lowering, meaning that optimizations won’t impact the accuracy of the coverage.
I haven’t dug deeply into your GCC patch. I also previously read SQLite’s description of MC/DC and had concerns about what they were doing with branch coverage to achieve that. It sounds like you are doing something different. They claim that MC/DC and branch coverage are “are very nearly the same thing”, but there are cases in which it is possible to have 100% branch condition coverage and yet not achieve MC/DC. I added branch condition coverage to LLVM in 2021 (Source-based Code Coverage — Clang 18.0.0git documentation), but I opted not to follow SQLite when implementing MC/DC.
For background (for the record), measuring a condition’s ability to independently impact a decision’s outcome requires the following to be shown:
This means that the MC/DC analysis really needs to track executed test vectors and analyze them to find pairs of test vectors in order to ascertain whether the MC/DC constraints are satisfied for each condition. This is the approach supported by other vendors, including LDRA and VectorCAST, and is what I have implemented for LLVM. The source-view visualization looks like this:
1| |#include "mcdc-demo.h"
2| |
3| 3|bool test(bool A, bool B, bool C) {
4| 3| return ((A && B) || C);
------------------
| Branch (4:12): [True: 2, False: 1]
| Branch (4:17): [True: 1, False: 1]
| Branch (4:23): [True: 0, False: 2]
------------------
|---> MC/DC Decision Region (4:11) to (4:24)
|
| Number of Conditions: 3
| Condition C1 --> (4:12)
| Condition C2 --> (4:17)
| Condition C3 --> (4:23)
|
| Executed MC/DC Test Vectors:
|
| C1, C2, C3 Result
| 1 { F, -, F = F }
| 2 { T, F, F = F }
| 3 { T, T, - = T }
|
| C1-Pair: covered: (1,3)
| C2-Pair: covered: (2,3)
| C3-Pair: not covered
| MC/DC Coverage for Decision: 66.67%
|
------------------
-Alan