Bob, Justin,
I’ve just committed a poor man’s coverage implementation that works with asan.
http://llvm.org/viewvc/llvm-project?rev=194701&view=rev
http://llvm.org/viewvc/llvm-project?rev=194702&view=rev
It provides only function-level boolean coverage (i.e. no counters, just “visited or not”),
but is very fast and very simple (no extra sections to the binary file, etc)
I’ve tried it for Chrome’s content_shell (huge and heavy binary) and the overhead
is negligible at both run-time and shutdown-time.
We’ll be evaluating this implementation and collecting usage stats.
Maybe we want to implement something simple like this in the Clang coverage.
–kcc
Hi Kostya,
Thanks for the heads-up on this. I haven’t had a chance to look into the details yet, but it looks like these patches may be breaking our bootstrapped LTO build. Our buildbots have been failing all day, and we’re still trying to figure out the problem. I’m going to speculatively revert those changes, since they were the only patches on the buildbot blame list. I will either reapply the changes or help debug the problem.
—Bob
Hi Kostya,
Thanks for the heads-up on this. I haven’t had a chance to look into the
details yet, but it looks like these patches may be breaking our
bootstrapped LTO build. Our buildbots have been failing all day, and we’re
still trying to figure out the problem. I’m going to speculatively revert
those changes, since they were the only patches on the buildbot blame list.
I will either reapply the changes or help debug the problem.
How could this possibly affect your LTO build?
The option is off by default.
Do you have any details, logs, etc?
Also, when are you planing to "reapply the changes or help debug"?
I’m waiting to see if this fixes the buildbots. Unfortunately, because they were failing all day, there are a bunch of other regressions that have come up, and I’m still working through them. It takes quite a while to run a bootstrapped LTO clang build, so it will take a while longer.
I don’t have any other useful information at this point, and I share your puzzlement about how your changes could possibly break the compiler. My only hypothesis is some sort of memory corruption.
I will keep you posted.
The bit code file produced by the stage 1 compiler for one of the files in the clang driver is corrupt and causes the linker for stage 2 to crash.
The bit code file produced by the stage 1 compiler for one of the files in the clang driver is corrupt and causes the linker for stage 2 to crash.
Is AddressSanitizer involved in any of the stages of the LTO build?
No, not that I am aware of.
No, not that I am aware of.
So, if my commits did indeed trigger the failures it could be
something like binary size change that caused different code alignment
or some such
and which triggered a latent memory bug somewhere else.
It's already late evening for you now. Will you have a chance to
reapply changes today?
--kcc
I don’t know yet, but I will let you know as soon as I can.
FYI I’ve seen what looked like a memory corruption in (private) Clang bootstrap process long before Kostya’s changes were submitted. I hope I’ll have the chance to investigate it soon.
Yes. I’m seeing this as well and it predates Kostya’s change. Shows up as memory corruption or double free on Linux.
I was able to successfully build with r194701, so I have reapplied that along with the compiler-rt changes. Somehow we need to get to the bottom of the problem. If you can reproduce it, please help!!
I was able to successfully build with r194701, so I have reapplied that
along with the compiler-rt changes.
Good! When you have a chance, please take a look at the change.
I'd like to see if we can do something like this for clang coverage.
Somehow we need to get to the bottom of
the problem. If you can reproduce it, please help!!
Alexey sees some memory corruption which is blocking him too
--kcc
Hi,
Justin is making nice commits for llvm-cov, so I thought we may continue this discussion now.
The quick-and-dirty implementation of coverage (in asan) is getting some early users and they seem to be happy.
AsanCoverage allows to collect per-function or per-basic-block coverage (booleans only, no counters) at a very low cost:
1% for per-function and 20% for per-basic-block: https://code.google.com/p/address-sanitizer/wiki/AsanCoverage
The output format is very simple and does not require any external tool (other than 56 lines in python) to handle it.
So, I was wondering it some similar functionality could be included as part of llvm-cov effort.
Also, do you have some documentation about the llvm-cov (how it works, what is the performance, etc)?
BTW, would a 5-minute lightning talk about AsanCoverage be interesting at LLVM Euro 2014?
Thanks,
–kcc
Hi,
Justin is making nice commits for llvm-cov, so I thought we may continue this discussion now.
The quick-and-dirty implementation of coverage (in asan) is getting some early users and they seem to be happy.
AsanCoverage allows to collect per-function or per-basic-block coverage (booleans only, no counters) at a very low cost:
1% for per-function and 20% for per-basic-block: https://code.google.com/p/address-sanitizer/wiki/AsanCoverage
The output format is very simple and does not require any external tool (other than 56 lines in python) to handle it.
So, I was wondering it some similar functionality could be included as part of llvm-cov effort.
Also, do you have some documentation about the llvm-cov (how it works, what is the performance, etc)?
Our recent work on llvm-cov has been pretty limited in scope. We’ve gotten it to work as a drop-in replacement for the old version of gcov that we had from gcc-4.2. There are still a few minor differences, but it generally seems to be working well.
As we discussed at the dev meeting, I want to hook up our PGO instrumentation to be used for coverage as well, but so far, our focus has been on the PGO side and there is nothing implemented for coverage yet. Once we start on that, I plan to extend llvm-cov to be the command line interface for viewing the coverage data. If you would like to teach it to display the AsanCoverage data as well, that seems like a good idea. Just be careful not to break anything for the old-style gcov support. There are various coverage tools out there that wrap around gcov, and we need to be careful not to break them.
BTW, would a 5-minute lightning talk about AsanCoverage be interesting at LLVM Euro 2014?
I’d be interested in that, but I haven’t been involved at all with planning LLVM Euro, so someone else will have to answer that.
Hi,
Justin is making nice commits for llvm-cov, so I thought we may continue
this discussion now.
The quick-and-dirty implementation of coverage (in asan) is getting some
early users and they seem to be happy.
AsanCoverage allows to collect per-function or per-basic-block coverage
(booleans only, no counters) at a very low cost:
1% for per-function and 20% for per-basic-block:
GitHub - google/sanitizers: AddressSanitizer, ThreadSanitizer, MemorySanitizer
The output format is very simple and does not require any external tool
(other than 56 lines in python) to handle it.
So, I was wondering it some similar functionality could be included as
part of llvm-cov effort.
Also, do you have some documentation about the llvm-cov (how it works,
what is the performance, etc)?
Our recent work on llvm-cov has been pretty limited in scope. We’ve gotten
it to work as a drop-in replacement for the old version of gcov that we had
from gcc-4.2. There are still a few minor differences, but it generally
seems to be working well.
As we discussed at the dev meeting, I want to hook up our PGO
instrumentation to be used for coverage as well, but so far, our focus has
been on the PGO side and there is nothing implemented for coverage yet.
Once we start on that, I plan to extend llvm-cov to be the command line
interface for viewing the coverage data. If you would like to teach it to
display the AsanCoverage data as well, that seems like a good idea.
I'd rather prefer to have functionality similar to today's AsanCoverage in
the main "LLVM coverage", whatever that is.
Just be careful not to break anything for the old-style gcov support.
Hm. Is that the long term goal?
My impression was that gcov is broken by design and beyond repair and we
wanted something new that can scale.
Or you are talking only about the gcov file format which we want to keep
for legacy reasons even if the files will be generated somehow differently?
There are various coverage tools out there that wrap around gcov, and we
need to be careful not to break them.
BTW, would a 5-minute lightning talk about AsanCoverage be interesting at
LLVM Euro 2014?
I’d be interested in that, but I haven’t been involved at all with
planning LLVM Euro, so someone else will have to answer that.
I'll send a proposal.
--kcc
gcov and llvm-cov are just tools to view the coverage data.
We definitely want llvm-cov to continue to read the gcov-style file format, and we also need it to continue to support all the same things as gcov (at least the old gcc-4.2 version) for the sake of compatibility. I’m suggesting that we add new features to teach it to read the data files produced by AsanCoverage and (eventually) the -fprofile-instr-generate data.
Hi,
Justin is making nice commits for llvm-cov, so I thought we may continue
this discussion now.
The quick-and-dirty implementation of coverage (in asan) is getting some
early users and they seem to be happy.
AsanCoverage allows to collect per-function or per-basic-block coverage
(booleans only, no counters) at a very low cost:
1% for per-function and 20% for per-basic-block:
GitHub - google/sanitizers: AddressSanitizer, ThreadSanitizer, MemorySanitizer
The output format is very simple and does not require any external tool
(other than 56 lines in python) to handle it.
So, I was wondering it some similar functionality could be included as
part of llvm-cov effort.
Also, do you have some documentation about the llvm-cov (how it works,
what is the performance, etc)?
Our recent work on llvm-cov has been pretty limited in scope. We’ve
gotten it to work as a drop-in replacement for the old version of gcov that
we had from gcc-4.2. There are still a few minor differences, but it
generally seems to be working well.
As we discussed at the dev meeting, I want to hook up our PGO
instrumentation to be used for coverage as well, but so far, our focus has
been on the PGO side and there is nothing implemented for coverage yet.
Once we start on that, I plan to extend llvm-cov to be the command line
interface for viewing the coverage data. If you would like to teach it to
display the AsanCoverage data as well, that seems like a good idea.
I'd rather prefer to have functionality similar to today's AsanCoverage in
the main "LLVM coverage", whatever that is.
Just be careful not to break anything for the old-style gcov support.
Hm. Is that the long term goal?
My impression was that gcov is broken by design and beyond repair and we
wanted something new that can scale.
Or you are talking only about the gcov file format which we want to keep
for legacy reasons even if the files will be generated somehow differently?
gcov and llvm-cov are just tools to view the coverage data.
We definitely want llvm-cov to continue to read the gcov-style file
format, and we also need it to continue to support all the same things as
gcov (at least the old gcc-4.2 version) for the sake of compatibility. I’m
suggesting that we add new features to teach it to read the data files
produced by AsanCoverage and (eventually) the -fprofile-instr-generate data.
Ok, I see.
Then my question: will there be any objection if I disentangle AsanCoverage
from ASan and make it a separate LLVM phase with the proper clang driver
support?
Or it will be an unwelcome competition with the planned clang coverage?
Thanks!
--kcc