[LibFuzzer] Recent performance regression due to r270942

Project Infrastructure LLVM Dev List Archives
#1

Hi,

This started as an off hand comment in [1] but this appears to be a
real issue so I'm moving the discussion to the mailing list.

In r270942 the time taken to run LibFuzzer's test became noticeably
longer. I am building on

* Arch Linux (4.5.4-1-ARCH #1 SMP PREEMPT Wed May 11 22:21:28 CEST
2016 x86_64 GNU/Linux)
* I am building libFuzzer and running its tests like so

CC=<new_clang> CXX=<new_clang++> cmake -DLLVM_USE_SANITIZE_COVERAGE=1
-DLLVM_USE_SANITIZER=Address -DCMAKE_BUILD_TYPE=RELEASE ../src/ &&
make check-fuzzer -j8

where ``<new_clang>`` and ``<new_clang++>`` point to a recently built
clang (llvm: r270960 , clang: r270962 , compiler-rt: r270949) which
itself was compiled using gcc 6.1.1.

In r270937 (prior commit) the time to run the tests was ~25 seconds on
my machine but for the subsequent commit ( r270942 ) the tests took
~78 seconds.

If I run the tests

cd lib/Fuzzer/test
python2 ../../../bin/llvm-lit -v --time-tests -j1 .

then I can observe the time it is taking to run the tests.

# For r270942

```

Slowest Tests:

crash-log.txt.xz (40.2 KB)

#2

Reproduced, should be easy to fix. Will do it.
And thanks for noticing, on my machine this fails very fast and the test passes because it sees everything it wants to see.

–kcc

#3

Done. r271095

#4

Thanks that fixed the issue for me. But now ``fuzzer.test`` is failing
for me. Specifically

 not LLVMFuzzer-NullDerefTest -close_fd_mask=3 2>&1 | FileCheck %s
--check-prefix=NullDerefTest

However it looks like this is to be expected because this test is
relying on the symbol ``__sanitizer_set_report_fd`` to be defined
however my version compiler-rt is a slightly too old and doesn't have
this symbol. r271046 seems to add this so I will rebuild the version
of clang I use to build LibFuzzer.

#5

This will happen periodically.
libFuzzer tests are only expected to work with the same revision of
clang+compiler-rt.
We try to keep libFuzzer *mostly* compatible with the older versions of
clang
(thus the code is "if (weak_fun) weak_func()" and not just "weak_func()"),
but the tests need freshest clang+asan

--kcc