Getting a combined llvm-cov report for a set of tests

Project Infrastructure LLVM Dev List Archives
1

I am working on a large software project with many thousands of tests. A common workflow using gcov as a coverage tool is:

  1. Build code with coverage instrumentation
  2. Run all the tests. Gcov aggregates the line coverage counters together into its .gcda files
  3. Generate a report for some set of files using those combined gcda files.
  4. Analyze the report to find files/sections of code that are not covered by any test.

Using llvm’s source-based code coverage, I don’t believe I can accomplish steps 3-4. Each binary produces a .profraw file which contains profile counts, but the mapping of those profile counts to files is contained inside the binary. I can use llvm-profdata merge to merge multiple .profraw files together, but the report can still only be generated for a single binary, and so only the files contained in that binary’s __llvm_covmap section will be produced as part of the report.

Am I correct in this thinking? Is there a way to get a single coverage report for a set of multiple binaries? I discovered the undocumented ‘llvm-cov convert-for-testing’ command which generates ‘covmapping files’, which appear to be the contents of __llvm_covmap with an additional header. Maybe it’s possible to merge multiple of these covmapping files and generate a report from that data (plus the merged profraw files)? Or perhaps I could use ‘llvm-cov export’ on each binary individually and manually merge the results somehow (although in that case, I would not be able to ‘import’ the merge to create the usual report) ? Are there other suggestions, or is this a planned feature in 4.0? I found some discussion on this thread, but I am not sure if this is asking the same thing:
https://groups.google.com/forum/#!topic/llvm-dev/r4qOdN-y9hc

Thanks!

2

Hi,

There isn't yet a way to generate a single coverage report for multiple
binaries. We're planning on adding this functionality to llvm-cov soon. The
patch is up for review here: ⚙ D25086 [llvm-cov] Add support for loading coverage from multiple objects.

Maybe it's possible to merge multiple of these covmapping files and generate a report from that data (plus the merged profraw files)?

It also isn't yet possible to merge multiple 'covmapping' files. Do you have
other use-cases for this functionality (apart from using it to generate a
single report given multiple binaries)?

I found some discussion on this thread, but I am not sure if this is asking the same thing:
Redirecting to Google Groups

Yes, that thread is about the same topic. The person who started it is the
author of the patch I linked to :).

best,
vedant