compiler-rt tests in cmake?

Greg_Fitzgerald · May 22, 2013, 7:38pm

Anybody working on porting the compiler-rt tests to cmake?

The online documentation shows a preference for cmake and ctest, but the CMakeLists file has the comment “Currently the tests have not been ported to CMake, so disable this directory.” How should we be running the test suite?

http://compiler-rt.llvm.org/

My immediate issue is that “make check-all” fails in the cmake build when compiler-rt is outside the projects directory. When I point to compiler-rt with LLVM_EXTERNAL_COMPILER_RT_SOURCE_DIR, lit still looks for lit.common.cfg within “projects/compiler-rt” instead of the external directory. I use similar CMake variables for Clang and Polly and have had no trouble. Any recommendations for how to fix?

Thanks,
Greg

Alexey_Samsonov · May 23, 2013, 4:39am

Hi!

The docs look strange to me - I don’t indeed see any CMake support for running compiler-rt tests.
Probably compiler-rt folks can comment on this…
I think you should run compilert-rt tests manually by smth. like compiler-rt/test/Unit/test.

CMake build system is able of running a bunch of sanitizer tests (AddressSanitizer, ThreadSanitizer etc.), and it assumes that
compiler-rt is checked out to llvm/projects/compiler-rt. Apparently, this is a problem. There was a patch that tried to address this, but
it never got committed.

Greg_Fitzgerald · May 23, 2013, 11:37pm

it assumes that compiler-rt is checked out to
llvm/projects/compiler-rt. Apparently, this is a problem.

I have a patch for this ready. I’ll send it to you and llvm-commits. Most of the tests pass with “make check-all” but the recently-added lsan tests are all failing. Do those fail for you as well? If so, can we XFAIL them for now and try to keep the “make check-all” build clean?

Thanks,
Greg

Alexey_Samsonov · May 24, 2013, 10:53am

> it assumes that compiler-rt is checked out to
> llvm/projects/compiler-rt. Apparently, this is a problem.

I have a patch for this ready. I'll send it to you and llvm-commits.
Most of the tests pass with "make check-all" but the recently-added lsan
tests are all failing. Do those fail for you as well? If so, can we XFAIL
them for now and try to keep the "make check-all" build clean?

Thanks! I'll take a look at the patch today. LSan tests work fine for me,
and we have them on our buildbot as well. What are the failures you see?

Sergey_Matveev · May 24, 2013, 1:10pm

I blame this line in lsan/lit_tests/lit.cfg:

Setup attributes common for all compiler-rt projects.

compiler_rt_lit_cfg = os.path.join(llvm_src_root, “projects”, “compiler-rt”,
“lib”, “lit.common.cfg”)

Greg_Fitzgerald · May 25, 2013, 12:12am

When I build compiler-rt with clang 3.2, all lsan tests pass. The only failing tests I see are in ubsan:

Failing Tests (6):
UndefinedBehaviorSanitizer :: Float/cast-overflow.cpp
UndefinedBehaviorSanitizer :: Integer/add-overflow.cpp
UndefinedBehaviorSanitizer :: Integer/div-zero.cpp
UndefinedBehaviorSanitizer :: Integer/sub-overflow.cpp
UndefinedBehaviorSanitizer :: Integer/uadd-overflow.cpp
UndefinedBehaviorSanitizer :: Integer/usub-overflow.cpp

When I build with gcc 4.4.3, I need the changes below to get the source to compile (and then the lsan tests fails). What are you all using to build compiler-rt? are you developing on linux, osx or windows? Using gcc or llvm? Which should I expect to work? Any buildbots running?

And lastly, are there build instructions to build the Android shared object? Is there any way to build an android static lib? How about for arm-linux?

Thanks,
Greg

diff --git a/lib/interception/interception.h b/lib/interception/interception.h
index d50af35…1771d4e 100644
— a/lib/interception/interception.h
+++ b/lib/interception/interception.h
@@ -27,8 +27,8 @@ typedef __sanitizer::uptr SIZE_T;
typedef __sanitizer::sptr SSIZE_T;
typedef __sanitizer::sptr PTRDIFF_T;
typedef __sanitizer::s64 INTMAX_T;
-typedef __sanitizer::OFF_T OFF_T;
-typedef __sanitizer::OFF64_T OFF64_T;
+//typedef __sanitizer::OFF_T OFF_T;
+//typedef __sanitizer::OFF64_T OFF64_T;

// How to add an interceptor:
// Suppose you need to wrap/replace system function (generally, from libc):
diff --git a/lib/lsan/lsan_allocator.cc b/lib/lsan/lsan_allocator.cc
index 9bf27b1…190dce8 100644
— a/lib/lsan/lsan_allocator.cc
+++ b/lib/lsan/lsan_allocator.cc
@@ -43,7 +43,7 @@ typedef CombinedAllocator<PrimaryAllocator, AllocatorCache,

Allocator;

static Allocator allocator;
-static THREADLOCAL AllocatorCache cache;
+static /THREADLOCAL/ AllocatorCache cache;

void InitializeAllocator() {
allocator.Init();
diff --git a/lib/msan/msan_allocator.cc b/lib/msan/msan_allocator.cc
index 7435843…3e6adb6 100644
— a/lib/msan/msan_allocator.cc
+++ b/lib/msan/msan_allocator.cc
@@ -33,7 +33,7 @@ typedef LargeMmapAllocator<> SecondaryAllocator;
typedef CombinedAllocator<PrimaryAllocator, AllocatorCache,

Allocator;

-static THREADLOCAL AllocatorCache cache;
+static /THREADLOCAL/ AllocatorCache cache;
static Allocator allocator;

static int inited = 0;

Evgenii_Stepanov · May 25, 2013, 8:17pm

When I build compiler-rt with clang 3.2, all lsan tests pass. The only
failing tests I see are in ubsan:

Failing Tests (6):
    UndefinedBehaviorSanitizer :: Float/cast-overflow.cpp
    UndefinedBehaviorSanitizer :: Integer/add-overflow.cpp
    UndefinedBehaviorSanitizer :: Integer/div-zero.cpp
    UndefinedBehaviorSanitizer :: Integer/sub-overflow.cpp
    UndefinedBehaviorSanitizer :: Integer/uadd-overflow.cpp
    UndefinedBehaviorSanitizer :: Integer/usub-overflow.cpp

When I build with gcc 4.4.3, I need the changes below to get the source to
compile (and then the lsan tests fails). What are you all using to build
compiler-rt? are you developing on linux, osx or windows? Using gcc or
llvm? Which should I expect to work? Any buildbots running?

I think we build compiler-rt with the host compiler, except for
Android. Which is usually a recently-built clang. n Linux, but we
exercise osx build on our buildbots.
We never build with compiler-rt not in projects/.
Our bot scripts are here:
https://code.google.com/p/address-sanitizer/source/browse/#svn%2Ftrunk%2Fbuild%2Fscripts%2Fslave
The bots itself are not publicly visible, we hope to change that soon.

And lastly, are there build instructions to build the Android shared object?
Is there any way to build an android static lib? How about for arm-linux?

Android runtime is special, we build it in a separate build tree configured with
-DCMAKE_TOOLCHAIN_FILE=$LLVM_CHECKOUT/cmake/platforms/Android.cmake
See buildbot_cmake.sh for details.

Alexey_Samsonov · May 27, 2013, 7:36am

> When I build compiler-rt with clang 3.2, all lsan tests pass. The only
> failing tests I see are in ubsan:
>
> Failing Tests (6):
> UndefinedBehaviorSanitizer :: Float/cast-overflow.cpp
> UndefinedBehaviorSanitizer :: Integer/add-overflow.cpp
> UndefinedBehaviorSanitizer :: Integer/div-zero.cpp
> UndefinedBehaviorSanitizer :: Integer/sub-overflow.cpp
> UndefinedBehaviorSanitizer :: Integer/uadd-overflow.cpp
> UndefinedBehaviorSanitizer :: Integer/usub-overflow.cpp
>
>
> When I build with gcc 4.4.3, I need the changes below to get the source
to
> compile (and then the lsan tests fails). What are you all using to build
> compiler-rt? are you developing on linux, osx or windows? Using gcc or
> llvm? Which should I expect to work? Any buildbots running?

As Evgeniy said, we build compiler-rt on Linux and on Mac OS X, and run
tests for various sanitizers on our buildbots.
Note that CMake build system of compiler-rt uses host compiler to build it.
On Linux we use:
1) gcc 4.6.3
2) tip-of-the-trunk Clang,
and tests pass under both of these.

gcc 4.4.3 seems way too old, so in some sense we've dropped support for it
(you have to make changes, as old gcc is not
smart enough to make some POD variables thread-local).

Greg_Fitzgerald · May 28, 2013, 6:26pm

Okay, dropping gcc 4.4.3 makes sense. How do you feel about using clang 3.2 (and the upcoming 3.3) instead of tip-of-the-trunk clang? It looks like everything works great, but that you just need to make those UB tests ‘unsupported’ since they fail with “libclang_rt.ubsan was built without __int128 support”.

Thanks,
Greg

Greg_Fitzgerald · May 29, 2013, 2:38am

Android runtime is special, we build it in a separate build tree configured with
-DCMAKE_TOOLCHAIN_FILE=$LLVM_CHECKOUT/cmake/platforms/Android.cmake

This worked great, thanks! Would you mind tweaking Android.cmake so that I can override the location of the C compiler? The current version forces me to use the just-built-clang and that the new build directory be in a sibling directory. But as you mentioned, we can build compiler-rt with a any recent version of gcc or clang. Here’s the change I’m looking for:

diff --git a/cmake/platforms/Android.cmake b/cmake/platforms/Android.cmake
index 72849b1…5f732ce 100644
— a/cmake/platforms/Android.cmake
+++ b/cmake/platforms/Android.cmake
@@ -11,8 +11,15 @@

make

SET(CMAKE_SYSTEM_NAME Linux)

Evgenii_Stepanov · May 29, 2013, 7:24am

Sure, r182831.

Alexey_Samsonov · May 29, 2013, 7:30am

UBsan tests work for me when I run “check-ubsan” in both build trees (the one with gcc 4.6.3 as a host compiler, and the one with fresh Clang).
It’s pretty convenient for us to use fresh Clang to configure LLVM and compiler-rt. One major reason is that autoconf/make build system always builds compiler-rt with just-built Clang.
There are other benefits, like keeping sanitizers code “-Werror”-clean under the fresh Clang, ability to catch regressions in compiler etc. Why would you need a fixed version?

Greg_Fitzgerald · May 29, 2013, 1:40pm

For me, UBsan fails with clang 3.2 and passes with clang 3.3.

Using a fixed version allows you to build all clang/llvm/compiler-rt with one compiler. It simplifies the build process quite a bit. Also better for isolating regressions in compiler-rt, especially if you use git-bisect.

Greg

Alexey_Samsonov · May 29, 2013, 2:28pm

For me, UBsan fails with clang 3.2 and passes with clang 3.3.

Cool, can you use clang 3.3 then? I think that the reason selected UBSan
tests fail under clang 3.2 is a bug in Clang, which was fixed (Richard may
correct me if I'm wrong).
I don't really want to mark these tests as "failing on a certain version of
host compiler".

Using a fixed version allows you to build all clang/llvm/compiler-rt with
one compiler. It simplifies the build process quite a bit. Also better
for isolating regressions in compiler-rt, especially if you use git-bisect.

I think you may fix a host compiler (it may be clang 3.3 or gcc 4.6.3),
build w/o -Werror and stay happy most of the time.
Bootstrap process (use system compiler to build Clang, use this Clang to
build LLVM+Clang+compiler-rt) isn't really scary, it adds just a few lines
to your build script.
AFAIK some developers actually do this every day. But, yes, this is not
user-friendly, and we probably should document it better (or even provide
the scripts) somewhere...

Greg_Fitzgerald · May 29, 2013, 11:40pm

Cool, can you use clang 3.3 then?

I can, but digging deeper I see that the compiler-rt sanitizer tests depend on just-built-clang for its object instrumentation. The next time the instrumentation changes, I’d expect those tests to break. If the lit tests that require -fsanitize were moved to the clang repo, then I think it’d be safe to build compiler-rt with clang 3.3 or gcc 4.6.3.

Back to Android, I built the ASan shared object and successfully ran this example:
https://code.google.com/p/address-sanitizer/source/browse/wiki/example_UseAfterFree.cc?r=1580

Kostya_Serebryany · May 30, 2013, 7:18am

+eugenis@, our Android expert.

Evgenii_Stepanov · May 30, 2013, 7:58am

Cool, can you use clang 3.3 then?

I can, but digging deeper I see that the compiler-rt sanitizer tests depend
on just-built-clang for its object instrumentation. The next time the
instrumentation changes, I'd expect those tests to break. If the lit tests
that require -fsanitize were moved to the clang repo, then I think it'd be
safe to build compiler-rt with clang 3.3 or gcc 4.6.3.

Back to Android, I built the ASan shared object and successfully ran this
example:
https://code.google.com/p/address-sanitizer/source/browse/wiki/example_UseAfterFree.cc?r=1580

But unlike these instructions:
http://www.chromium.org/developers/testing/addresssanitizer

the Android instructions don't mention asan_symbolize.py
https://code.google.com/p/address-sanitizer/wiki/Android

When I use asan_symbolize.py (from Linux), I see no symbols in the stack
trace and the following error messages:

addr2line: '/data/example_UseAfterFree': No such file
addr2line: '/data/libclang_rt.asan-arm-android.so': No such file
addr2line: '/system/lib/libstdc++.so': No such file

How can I decode those addresses in the stack trace? Is there a way to
configure asan_symbolize.py to find my binaries and
arm-linux-androideabi-addr2line?

You could try preprocessing your report with perl or sed to fix paths
to your binaries. It would be great to have an option for that in
asan_symbolize.py.

As for addr2line, we just install binutils-multiarch ubuntu package.

Alexey_Samsonov · May 30, 2013, 8:03am

> Cool, can you use clang 3.3 then?

I can, but digging deeper I see that the compiler-rt sanitizer tests
depend on just-built-clang for its object instrumentation. The next time
the instrumentation changes, I'd expect those tests to break. If the lit
tests that require -fsanitize were moved to the clang repo, then I think
it'd be safe to build compiler-rt with clang 3.3 or gcc 4.6.3.

Tests are different: certainly tests that depend on instrumentation are
built with clang from the same build tree (that is tests "depend" on Clang).
* when you run "make" in your build tree, you build Clang and runtimes with
a host compiler
* when you run "make check-all" in your build tree, you use this Clang and
runtimes to build/run tests.

Greg_Fitzgerald · May 30, 2013, 6:05pm

The sanitizer common and asan that mention 'thread' are failing for me
this morning. How are your bots looking? Last good commit here was
512c616cacf70ca029a2bf719a482b902f3687cd.

You could try preprocessing your report with perl or sed to fix paths
to your binaries. It would be great to have an option for that in
asan_symbolize.py.

As for addr2line, we just install binutils-multiarch ubuntu package.

Cool, that gets the job done, thanks. Looks like there's some effort
going into embedding the addr2line functionality into compiler-rt. Is
that something folks are actively working on?

Tests are different: certainly tests that depend on instrumentation
are built with clang from the same build tree (that is tests "depend" on Clang).

The compiler-rt library definitions have no dependency on the clang
instrumentation, correct? Only the other way around?

Thanks,
Greg

Alexey_Samsonov · May 30, 2013, 7:28pm

The sanitizer common and asan that mention 'thread' are failing for me
this morning. How are your bots looking? Last good commit here was
512c616cacf70ca029a2bf719a482b902f3687cd.

Hm, our bots seem to be green. Could you refer to guilty svn revision?

> You could try preprocessing your report with perl or sed to fix paths
> to your binaries. It would be great to have an option for that in
> asan_symbolize.py.
>
> As for addr2line, we just install binutils-multiarch ubuntu package.

Cool, that gets the job done, thanks. Looks like there's some effort
going into embedding the addr2line functionality into compiler-rt. Is
that something folks are actively working on?

Well, LLVM has llvm-symbolizer tool, which is analogous to addr2line.
For now sanitizer tools are able to use it for out-of-process symbolization
(e.g. on Linux you won't have to run report through asan_symbolize.py if
you provide env variable ASAN_SYMBOLIZER_PATH=/path/to/llvm-symbolizer).
We have plans to actually compile the symbolizer into the binary and do
in-process symbolization, but it's not there yet. Not sure if that's what
you mean
by "addr2line in compiler-rt".

> Tests are different: certainly tests that depend on instrumentation
> are built with clang from the same build tree (that is tests "depend" on
Clang).

The compiler-rt library definitions have no dependency on the clang
instrumentation, correct? Only the other way around?

I'm confused here. compiler-rt and clang/llvm instrumentation depend on
each other,
so the version of Clang you use should be the same as the version of
compiler-rt.