Hello everyone,
I’m working on migrating to Source-based code coverage from gcov-compatible coverage w/ clang 13.0.1 for a large C++ project and we’ve been facing with a problem with llvm-cov reporting hash mismatches for methods implemented in header files that is causing incomplete coverage reports - the counter groups for these methods are completely skipped throughout all instatiations. Related issues to the problem are: #72786 and #32849.
I’ve compiled a minimal example and my understanding of it, and I would like to ask for advice on how to mitigate this.
We’re running code coverage builds to get coverage reports for libraries dynamically linked to test executables. The test executables themselves do not get instrumented, because we are not interested in numbers for the tests and the instrumentation slows down their build. The libraries are instrumented and only they are used when invoking llvm-cov.
Consider a mock library that consists of a class with most methods implemented inside a header file:
test_class.h
#ifndef __TEST_CLASS_H_
#define __TEST_CLASS_H_
class TestClass {
public:
TestClass(bool x) { if (x) a = 1; else a = 0; }
int TestMethod(int x) { if (a == 1) return x; else return 0; }
int NotInHeader(int a, int b, int c);
static int TestStaticMethod(int x) { return x + 2; }
private:
int a;
};
#endif
test_lib.cc
#include "test_class.h"
int TestClass::NotInHeader(int a, int b, int c) {
this->a = a + b + c;
return c;
}
We build this library with instrumentation:
$ clang -fPIC -shared -fprofile-instr-generate -fcoverage-mapping -o libtest_lib.so test_lib.cc
The only symbol from the class is the NotInHeader method:
$ readelf -s libtest_lib.so | grep TestClass
55: 00000000000015a0 55 FUNC GLOBAL DEFAULT 10 _ZN9TestClass11NotInHeade
241: 00000000000015a0 55 FUNC GLOBAL DEFAULT 10 _ZN9TestClass11NotInHeade
Consider another library which instantiates some of the methods defined in the header:
another_lib.cc
#include "test_class.h"
void utility(void) {
TestClass t(1);
t.TestMethod(0);
TestClass::TestStaticMethod(1); }
It also gets built instrumented:
$ clang -fPIC -shared -fprofile-instr-generate -fcoverage-mapping -o libanother_lib.so another_lib.cc
$ readelf -sW libanother_lib.so | grep TestClass
54: 0000000000001750 97 FUNC WEAK DEFAULT 10 _ZN9TestClassC2Eb
55: 0000000000001820 33 FUNC WEAK DEFAULT 10 _ZN9TestClass16TestStaticMethodEi
60: 00000000000017c0 83 FUNC WEAK DEFAULT 10 _ZN9TestClass10TestMethodEi
109: 0000000000209248 16 OBJECT LOCAL DEFAULT 24 __profc__ZN9TestClass10TestMethodEi
125: 0000000000209258 8 OBJECT LOCAL DEFAULT 24 __profc__ZN9TestClass16TestStaticMethodEi
142: 0000000000209238 16 OBJECT LOCAL DEFAULT 24 __profc__ZN9TestClassC2Eb
215: 0000000000001750 97 FUNC WEAK DEFAULT 10 _ZN9TestClassC2Eb
217: 0000000000001820 33 FUNC WEAK DEFAULT 10 _ZN9TestClass16TestStaticMethodEi
239: 00000000000017c0 83 FUNC WEAK DEFAULT 10 _ZN9TestClass10TestMethodEi
Here is our mock test executable that links both of these libraries and instantiates header-defined methods but doesn’t get instrumented:
executable.cc
#include "test_class.h"
#include <stdio.h>
extern void utility(void);
int main() {
TestClass t(1);
printf("%d\n",t.TestMethod(1));
t.NotInHeader(1,2,3);
utility();
}
$ clang -L./ -ltest_lib -lanother_lib -o exe executable_1.cc
Running it we get a profraw file with all counters of interest:
$ LD_LIBRARY_PATH=./ ./exe
$ llvm-profdata show default.profraw --all-functions --counts
Counters:
_ZN9TestClass11NotInHeaderEiii:
Hash: 0x0000000000000018
Counters: 1
Function count: 1
Block counts: []
_Z7utilityv:
Hash: 0x0000000000000000
Counters: 1
Function count: 1
Block counts: []
_ZN9TestClassC2Eb:
Hash: 0x00000000002924d1
Counters: 2
Function count: 0
Block counts: [0]
_ZN9TestClass10TestMethodEi:
Hash: 0x000000a7d2613611
Counters: 2
Function count: 0
Block counts: [0]
_ZN9TestClass16TestStaticMethodEi:
Hash: 0x0000000000000018
Counters: 1
Function count: 1
Block counts: []
Instrumentation level: Front-end
Functions shown: 5
Total functions: 5
Maximum function count: 1
Maximum internal block count: 0
$ llvm-profdata merge -o test.profdata default.profraw
Oddly, there’s no hits on the constructor _ZN9TestClassC2Eb and _ZN9TestClass10TestMethodEi.
Here is where the problem appears: we want to export this data from the two libraries:
$ llvm-cov export -format lcov -instr-profile test.profdata libtest_lib.so libanother_lib.so
warning: 3 functions have mismatched data
SF:/home/aleksa.markovic/projects/cc/llvm_reader_test/test_lib.cc
FN:3,_ZN9TestClass11NotInHeaderEiii
FNDA:1,_ZN9TestClass11NotInHeaderEiii
FNF:1
FNH:1
DA:3,1
DA:4,1
DA:5,1
DA:6,1
BRF:0
BRH:0
LF:4
LH:4
end_of_record
There is a mismatch for the header-defined methods and the counters from them are not exported. When this is scaled to dozens of libraries and instatiations, thousands of symbols end up mismatched and it’s not deterministic whether they’re present or not.
Now, we swap the libraries and there’s no mismatches reported:
$ llvm-cov export -format lcov -instr-profile test.profdata libanother_lib.so libtest_lib.so
SF:/home/aleksa.markovic/projects/cc/llvm_reader_test/another_lib.cc
FN:3,_Z7utilityv
FNDA:1,_Z7utilityv
FNF:1
FNH:1
DA:3,1
DA:4,1
DA:5,1
DA:6,1
DA:7,1
BRF:0
BRH:0
LF:5
LH:5
end_of_record
SF:/home/aleksa.markovic/projects/cc/llvm_reader_test/test_class.h
FN:6,_ZN9TestClassC2Eb
FN:8,_ZN9TestClass10TestMethodEi
FN:12,_ZN9TestClass16TestStaticMethodEi
FNDA:0,_ZN9TestClassC2Eb
FNDA:0,_ZN9TestClass10TestMethodEi
FNDA:1,_ZN9TestClass16TestStaticMethodEi
FNF:3
FNH:1
DA:6,0
DA:8,0
DA:12,1
BRDA:6,0,0,-
BRDA:6,0,1,-
BRDA:8,0,0,-
BRDA:8,0,1,-
BRF:4
BRH:0
LF:3
LH:1
end_of_record
There’s different behavior when the libraries are specified with --object:
$ llvm-cov export -format lcov -instr-profile test.profdata --object libanother_lib.so --object libtest_lib.so
warning: 3 functions have mismatched data
SF:/home/aleksa.markovic/projects/cc/llvm_reader_test/another_lib.cc
FN:3,_Z7utilityv
FNDA:1,_Z7utilityv
FNF:1
FNH:1
DA:3,1
DA:4,1
DA:5,1
DA:6,1
DA:7,1
BRF:0
BRH:0
LF:5
LH:5
end_of_record
SF:/home/aleksa.markovic/projects/cc/llvm_reader_test/test_class.h
FN:6,_ZN9TestClassC2Eb
FN:8,_ZN9TestClass10TestMethodEi
FN:12,_ZN9TestClass16TestStaticMethodEi
FNDA:0,_ZN9TestClassC2Eb
FNDA:0,_ZN9TestClass10TestMethodEi
FNDA:1,_ZN9TestClass16TestStaticMethodEi
FNF:3
FNH:1
DA:6,0
DA:8,0
DA:12,1
BRDA:6,0,0,-
BRDA:6,0,1,-
BRDA:8,0,0,-
BRDA:8,0,1,-
BRF:4
BRH:0
LF:3
LH:1
end_of_record
SF:/home/aleksa.markovic/projects/cc/llvm_reader_test/test_lib.cc
FN:3,_ZN9TestClass11NotInHeaderEiii
FNDA:1,_ZN9TestClass11NotInHeaderEiii
FNF:1
FNH:1
DA:3,1
DA:4,1
DA:5,1
DA:6,1
BRF:0
BRH:0
LF:4
LH:4
end_of_record
Mismatches are reported but the counters seem to be output correctly. However in our real use case with dozens of libraries, the number of lines output still varies.
In these situations, what is the best way to invoke llvm-cov to get the most complete and consistent results?
Thank you.
