ASan testing on Android

I'm following these instructions with the latest code from llvm.org:

https://code.google.com/p/address-sanitizer/source/browse/trunk/build/scripts/slave/buildbot_cmake.sh

Of SanitizerTest, AsanTest and AsanNoinstTest, I cannot get any of the
3 to pass all its tests. I've tried using an Android device running
JellyBean as well as the latest Android Emulator (20130917, api level
18). What run-time environments are known to work?

Thanks,
Greg

+eugenis

ARM JellyBean works over here. There is some flakiness in SanitizerCommon.SpinMutex test.

Some updates:

First, thanks so much for creating these test suites. Troubleshooting
is going very quickly!

There is some flakiness in SanitizerCommon.SpinMutex test.

Yes, I'm seeing that too. SpinMutexTry as well. Should these be disabled?

ARM JellyBean works over here.

I found a configuration that works well on JB 4.1.2 and 4.2.2. The
trick is to build the ASan *runtime* with *CMake*. If you build the
runtime using the Make build, the test suite falls over.

Also, lots of tests fail when running on the Android Emulator. This
is a convenient environment for testing, but is there any hope of
getting all of ASan working on it? I see that simple cases such as
detecting use-after-free and stack-buffer-overflow work on the
emulator.

Regarding:
https://code.google.com/p/address-sanitizer/wiki/Android

I see that the Make build puts the ASan runtime into its install
directory, and the test suite from the CMake build uses the local copy
of the ASan runtime for testing.

To reproduce the error I'm seeing, run the Android CMake build in the
Release+Asserts directory of the Make build. Install the ASan runtime
from the Make build and then run the AsanTests test suite.

Thanks,
Greg

Some updates:

First, thanks so much for creating these test suites. Troubleshooting
is going very quickly!

> There is some flakiness in SanitizerCommon.SpinMutex test.

Yes, I'm seeing that too. SpinMutexTry as well. Should these be disabled?

There is an issue in the spinlock on ARM: it's implemented with plain
non-atomic loads and stores. It seems that android ndk does not have the
atomics we need for this, and we did not have time to do an asm
implementation yet.

> ARM JellyBean works over here.

I found a configuration that works well on JB 4.1.2 and 4.2.2. The
trick is to build the ASan *runtime* with *CMake*. If you build the
runtime using the Make build, the test suite falls over.

Also, lots of tests fail when running on the Android Emulator. This
is a convenient environment for testing, but is there any hope of
getting all of ASan working on it? I see that simple cases such as
detecting use-after-free and stack-buffer-overflow work on the
emulator.

We did not run the tests on emulator. Not sure what would it take to fix
the tests there. It can't be super hard, though.

Regarding:
https://code.google.com/p/address-sanitizer/wiki/Android

I see that the Make build puts the ASan runtime into its install
directory, and the test suite from the CMake build uses the local copy
of the ASan runtime for testing.

To reproduce the error I'm seeing, run the Android CMake build in the
Release+Asserts directory of the Make build. Install the ASan runtime
from the Make build and then run the AsanTests test suite.

Do you mean it's a compile-time issue with the path in Android.cmake?

No, I'm saying the Make build for Android generates a broken ASan
shared library.

-Greg