llvm-cov: Combined report for multiple executables

Project Infrastructure LLVM Dev List Archives
1

Hi! I am trying to get a combined coverage report from multiple
executables. Looking at earlier discussions [1, 2], it looks like this
is supposed to work. I am having some difficulty getting this to work
as I would expect it to work, however. Following is a simple case to
explain:

////////// shared.h
#include <string>

void Print1(const std::string& msg);
void Print2(const std::string& msg);
////////// end

////////// shared.cc
#include <iostream>
#include <string>

void Print1(const std::string& msg) {
  std::cout << "1" << msg;
}

void Print2(const std::string& msg) {
  std::cout << "2" << msg;
}
////////// end

////////// first.cc
#include "shared.h"

int main() {
  Print1("First\n");
  return 0;
}
////////// end

////////// second.cc
#include "shared.h"

int main() {
  Print2("Second\n");
  return 0;
}
////////// end

I use the following commands to build 'first' and 'second' binaries:
  clang++ -O0 -c -fprofile-instr-generate -fcoverage-mapping shared.cc
-o ./out/shared.o
  clang++ -O0 -fprofile-instr-generate -fcoverage-mapping
./out/shared.o first.cc -o ./out/first
  clang++ -O0 -fprofile-instr-generate -fcoverage-mapping
./out/shared.o second.cc -o ./out/second

I then use the following to run the binaries:
  LLVM_PROFILE_FILE='./out/first.profraw' ./out/first
  LLVM_PROFILE_FILE='./out/second.profraw' ./out/second

I then use the following commands to show the report:
  llvm-profdata merge -sparse ./out/first.profraw ./out/second.profraw
-o ./out/all.profdata
  llvm-cov show ./out/second -object ./out/first -instr-profile
./out/all.profdata

But the output only contains:

second.cc:
    1| |#include "shared.h"
    2| |
    3| 2|int main() {
    4| 2| Print2("Second\n");
    5| 2| return 0;
    6| 2|}

shared.cc:
    1| |#include <iostream>
    2| |#include <string>
    3| |
    4| 1|void Print1(const std::string& msg) {
    5| 1| std::cout << "1" << msg;
    6| 1|}
    7| |
    8| 1|void Print2(const std::string& msg) {
    9| 1| std::cout << "2" << msg;
   10| 1|}

The output does not include anything for first.cc If I run the last
command with ./out/second and ./out/first swapped, then the output
only contains first.cc and shared.cc, but does not have second.cc

Is my expectation that the report should include both first.cc and
second.cc correct? If it is, then I am probably holding something
wrong? Any suggestions on what I should try?

[1] Redirecting to Google Groups
[2] ⚙ D25086 [llvm-cov] Add support for loading coverage from multiple objects

Thank you!
Sadrul

2

Hi Sadrul,

Hi! I am trying to get a combined coverage report from multiple
executables. Looking at earlier discussions [1, 2], it looks like this
is supposed to work. I am having some difficulty getting this to work
as I would expect it to work, however. Following is a simple case to
explain:

////////// shared.h
#include <string>

void Print1(const std::string& msg);
void Print2(const std::string& msg);
////////// end

////////// shared.cc
#include <iostream>
#include <string>

void Print1(const std::string& msg) {
std::cout << "1" << msg;
}

void Print2(const std::string& msg) {
std::cout << "2" << msg;
}
////////// end

////////// first.cc
#include "shared.h"

int main() {
Print1("First\n");
return 0;
}
////////// end

////////// second.cc
#include "shared.h"

int main() {
Print2("Second\n");
return 0;
}
////////// end

I use the following commands to build 'first' and 'second' binaries:
clang++ -O0 -c -fprofile-instr-generate -fcoverage-mapping shared.cc
-o ./out/shared.o
clang++ -O0 -fprofile-instr-generate -fcoverage-mapping
./out/shared.o first.cc -o ./out/first
clang++ -O0 -fprofile-instr-generate -fcoverage-mapping
./out/shared.o second.cc -o ./out/second

I then use the following to run the binaries:
LLVM_PROFILE_FILE='./out/first.profraw' ./out/first
LLVM_PROFILE_FILE='./out/second.profraw' ./out/second

I then use the following commands to show the report:
llvm-profdata merge -sparse ./out/first.profraw ./out/second.profraw
-o ./out/all.profdata
llvm-cov show ./out/second -object ./out/first -instr-profile
./out/all.profdata

But the output only contains:

second.cc:
   1| |#include "shared.h"
   2| |
   3| 2|int main() {
   4| 2| Print2("Second\n");
   5| 2| return 0;
   6| 2|}

shared.cc:
   1| |#include <iostream>
   2| |#include <string>
   3| |
   4| 1|void Print1(const std::string& msg) {
   5| 1| std::cout << "1" << msg;
   6| 1|}
   7| |
   8| 1|void Print2(const std::string& msg) {
   9| 1| std::cout << "2" << msg;
  10| 1|}

The output does not include anything for first.cc If I run the last
command with ./out/second and ./out/first swapped, then the output
only contains first.cc and shared.cc, but does not have second.cc

Is my expectation that the report should include both first.cc and
second.cc correct? If it is, then I am probably holding something
wrong? Any suggestions on what I should try?

The coverage tool currently has limitations which require it to assume that the one definition rule holds over all of the tool inputs. That means that if you have multiple definitions of the same symbol (in this case "main"), the coverage tool will only report one of them.

You might try reorganizing the code so that there's only one function called main.

best,
vedant

3

Hi Sadrul,

Hi! I am trying to get a combined coverage report from multiple
executables. Looking at earlier discussions [1, 2], it looks like this
is supposed to work. I am having some difficulty getting this to work
as I would expect it to work, however. Following is a simple case to
explain:

////////// shared.h
#include <string>

void Print1(const std::string& msg);
void Print2(const std::string& msg);
////////// end

////////// shared.cc
#include <iostream>
#include <string>

void Print1(const std::string& msg) {
std::cout << "1" << msg;
}

void Print2(const std::string& msg) {
std::cout << "2" << msg;
}
////////// end

////////// first.cc
#include "shared.h"

int main() {
Print1("First\n");
return 0;
}
////////// end

////////// second.cc
#include "shared.h"

int main() {
Print2("Second\n");
return 0;
}
////////// end

I use the following commands to build 'first' and 'second' binaries:
clang++ -O0 -c -fprofile-instr-generate -fcoverage-mapping shared.cc
-o ./out/shared.o
clang++ -O0 -fprofile-instr-generate -fcoverage-mapping
./out/shared.o first.cc -o ./out/first
clang++ -O0 -fprofile-instr-generate -fcoverage-mapping
./out/shared.o second.cc -o ./out/second

I then use the following to run the binaries:
LLVM_PROFILE_FILE='./out/first.profraw' ./out/first
LLVM_PROFILE_FILE='./out/second.profraw' ./out/second

I then use the following commands to show the report:
llvm-profdata merge -sparse ./out/first.profraw ./out/second.profraw
-o ./out/all.profdata
llvm-cov show ./out/second -object ./out/first -instr-profile
./out/all.profdata

But the output only contains:

second.cc:
   1| |#include "shared.h"
   2| |
   3| 2|int main() {
   4| 2| Print2("Second\n");
   5| 2| return 0;
   6| 2|}

shared.cc:
   1| |#include <iostream>
   2| |#include <string>
   3| |
   4| 1|void Print1(const std::string& msg) {
   5| 1| std::cout << "1" << msg;
   6| 1|}
   7| |
   8| 1|void Print2(const std::string& msg) {
   9| 1| std::cout << "2" << msg;
  10| 1|}

The output does not include anything for first.cc If I run the last
command with ./out/second and ./out/first swapped, then the output
only contains first.cc and shared.cc, but does not have second.cc

Is my expectation that the report should include both first.cc and
second.cc correct? If it is, then I am probably holding something
wrong? Any suggestions on what I should try?

The coverage tool currently has limitations which require it to assume that the one definition rule holds over all of the tool inputs. That means that if you have multiple definitions of the same symbol (in this case "main"), the coverage tool will only report one of them.

Ah, I see. If I add some functions other than main() in first.cc and
second.cc, those all do correctly show up (except one of the 'main'
functions show no-coverage). That makes sense. Thank you very much for
the quick reply!

Sadrul