-fsanitize=undefined and shared libraries

When trying to use -fsanitize=undefined with shared libraries, I get:

/usr/bin/ld: a.out: hidden symbol `__ubsan_handle_float_cast_overflow’ in /home/clang/build-cmake/bin/…/lib/clang/3.2/lib/linux/libclang_rt.ubsan-x86_64.a(ubsan_handlers.cc.o) is referenced by DSO
/usr/bin/ld: final link failed: Bad value
clang-3: error: linker command failed with exit code 1 (use -v to see invocation)

This is with Clang 3.2 compiled from sources using cmake, on Ubuntu 12.10.

I’m compiling my files like this:

clang++ -fPIC -fsanitize=undefined undefined.cpp -c -o undefined.o
clang++ -shared -Wl,-soname,libfoo.so undefined.o -o libfoo.so
clang++ -fsanitize=undefined main.cpp -c -o main.o
clang++ -fsanitize=undefined main.o -L. -lfoo

$ cat undefined.cpp
int cast(float x) {
return int(x);
}

$ cat main.cpp
#include

extern int cast(float x);

int main(int argc, char *argv[])
{
if (argc > 1) {
return cast(atof(argv[1]));
}

return 1;
}

If I don’t use shared libraries, but instead like the two .o files (without the -fPIC), it works fine, I get:

./a.out 1e10

: fatal error: value 1e+10 is outside the range of representable values of type ‘int’
Illegal instruction (core dumped)

Any idea what I’m doing wrong?

Thanks,
Martin

Did you got compiler-rt?

Yes, otherwise it wouldn’t work when statically linked.

The attached patch should fix the issue for you.

Richard, okay to commit?

~Will

0001-ubsan-Give-entry-methods-default-not-hidden-visibili.patch (2.81 KB)

Please use SANITIZER_INTERFACE_ATTRIBUTE (defined in
include/sanitizer/common_interface_defs.h). Otherwise, this looks
great, thanks!

I have had a similar problem with integer checks. compiler-rt was not linked
to .so files and there were unknown references to
__ubsan_handle_add_overflow and so on.

I managed this issue adding "-Wl,--whole-archive
-L/usr/local/lib/clang/3.3/lib/linux/ -lclang_rt.ubsan-i386
-Wl,--no-whole-archive" to command line... and its not a beautiful solution.

When this issue will be fixed in trunk?

I have had a similar problem with integer checks. compiler-rt was not linked
to .so files and there were unknown references to
__ubsan_handle_add_overflow and so on.

Do you have a means for reproducing this failure on trunk? A short
example would be helpful :).

If you don't have one handy, I can probably put one together sometime
next week. I'm pretty sure I've seen this a few times, even after the
visibility fix.

I managed this issue adding "-Wl,--whole-archive
-L/usr/local/lib/clang/3.3/lib/linux/ -lclang_rt.ubsan-i386
-Wl,--no-whole-archive" to command line... and its not a beautiful solution.

This is what clang does for asan, and I suspect we could do something
similar for ubsan. To avoid requiring a C++ runtime, perhaps we could
build two ubsan libraries (ubsan, ubsan-cxx) and link them in as
appropriate. The ubsan code already builds the C++ handlers in
separate TU's, so hopefully this wouldn't take much work.

In a different direction, I notice that we don't link in the ubsan
runtime when building a shared library. Why is this? (to avoid code
duplication, issues with -fPIC, some static initializer/internal state
concern?) Removing this check on my local clang copy has resolved
this issue for me, and is presently how I'm building programs.
However, I'm reluctant to propose this as a solution without knowing
why things are the way they are first :).

When this issue will be fixed in trunk?

As soon as someone reports it (looks like you've kicked that off,
thanks!) and we understand the issue well enough to fix it (perhaps
others on the list already know a good solution).

Thanks for bringing this up, and here's hoping it's resolved soon.

~Will

Will Dietz wrote

In a different direction, I notice that we don't link in the ubsan
runtime when building a shared library. Why is this? (to avoid code
duplication, issues with -fPIC, some static initializer/internal state
concern?) Removing this check on my local clang copy has resolved
this issue for me, and is presently how I'm building programs.
However, I'm reluctant to propose this as a solution without knowing
why things are the way they are first :).

Yes without this crunch the .so library have "U __ubsan_handle_add_overflow"
symbol according to nm. And it is impossible to load and use this library
through dlopen() for example.
Your patch with __attribute__((visibility("default"))) didn't help :frowning: I have
recompiled my llvm clang compiler-rt and still have undefined
__ubsan_handle_add_overflow in .so
Code duplication is not a big problem, this handlers are so tiny.

I have had a similar problem with integer checks. compiler-rt was not linked
to .so files and there were unknown references to
__ubsan_handle_add_overflow and so on.

Do you have a means for reproducing this failure on trunk? A short
example would be helpful :).

If you don’t have one handy, I can probably put one together sometime
next week. I’m pretty sure I’ve seen this a few times, even after the
visibility fix.

I managed this issue adding “-Wl,–whole-archive
-L/usr/local/lib/clang/3.3/lib/linux/ -lclang_rt.ubsan-i386
-Wl,–no-whole-archive” to command line… and its not a beautiful solution.

This is what clang does for asan, and I suspect we could do something
similar for ubsan. To avoid requiring a C++ runtime, perhaps we could
build two ubsan libraries (ubsan, ubsan-cxx) and link them in as
appropriate. The ubsan code already builds the C++ handlers in
separate TU’s, so hopefully this wouldn’t take much work.

Using --whole-archive here would break “-fsanitize=address,undefined”, since you’d link in sanitizer_common twice. This change would be unnecessary if we were to always link to the ubsan runtime, as you suggest below.

In a different direction, I notice that we don’t link in the ubsan
runtime when building a shared library. Why is this? (to avoid code
duplication, issues with -fPIC, some static initializer/internal state
concern?) Removing this check on my local clang copy has resolved
this issue for me, and is presently how I’m building programs.
However, I’m reluctant to propose this as a solution without knowing
why things are the way they are first :).

The current approach matches the approach taken for the other sanitizers. In general, the main binary must be linked with all the -fsanitize= flags used to build the constituent parts, right now. This restriction makes some sense for ASan, TSan, and MSan (since they need to do invasive things to the program as a whole), but we can probably remove it for UBSan.

Great, that works! The next linking error I get is:

hidden symbol `__ubsan_vptr_type_cache’ in /home/martin/clang/build/bin/…/lib/clang/3.3/lib/linux/libclang_rt.ubsan-x86_64.a(ubsan_type_hash.cc.o) is referenced by DSO

It’s a little trouble for me to narrow down the test case, but I’m happy to do it if it helps.

Thanks,
Martin

Any word on this linking error? Would a test case help?

Thanks,
Martin

Any word on this linking error? Would a test case help?

Thanks,
Martin

Great, that works! The next linking error I get is:

hidden symbol `__ubsan_vptr_type_cache' in
/home/martin/clang/build/bin/../lib/clang/3.3/lib/linux/libclang_rt.ubsan-x86_64.a(ubsan_type_hash.cc.o)
is referenced by DSO

Just a random guess - would adding SANITIZER_INTERFACE_ATTRIBUTE
to __ubsan_vptr_type_cache definition help? Like this:
extern "C" SANITIZER_INTERFACE_ATTRIBUTE HashValue
__ubsan_vptr_type_cache[VptrTypeCacheSize];

Bah, missed one! :). Thanks Alexey, fixed in r172730.

Martin, can you update and try now? Let me know if you have any
further linking issues.

As an aside, are you using vanilla clang/compiler-rt, or did you need
to apply the "always link in ubsan" workaround discussed in this
thread?

Thanks, happy code checking! :slight_smile:

~Will

Hey Will,

I’m using vanilla clang/compiler-rt, not the “always link in ubsan” workaround.

It now compiles and runs my unit tests! However, using std::fixed like this:

#include

int main()
{
std::cout << std::fixed;
return 0;
}

Gives this error message:

$ clang++ -g -fsanitize=undefined foo.cpp && ./a.out

==15067== WARNING: Trying to symbolize code, but external symbolizer is not initialized!

/home/martin/cache/lib/a.out:0x402918: runtime error: load of value 4294967035, which is not a valid value for type ‘std::_Ios_Fmtflags’
/home/martin/cache/lib/a.out:0x402b44: runtime error: load of value 4294967035, which is not a valid value for type ‘std::_Ios_Fmtflags’

Two questions:

Should I ignore the warning, if not, how do I fix it?

Is there a way to suppress messages from the standard library? I’m guessing not, but I can write a quick script on my end to filter them out.

Best,
Martin

Hey Will,

I'm using vanilla clang/compiler-rt, not the "always link in ubsan"
workaround.

It now compiles and runs my unit tests! However, using std::fixed like
this:

#include <iostream>

int main()
{
    std::cout << std::fixed;
    return 0;
}

Gives this error message:

$ clang++ -g -fsanitize=undefined foo.cpp && ./a.out

==15067== WARNING: Trying to symbolize code, but external symbolizer is not
initialized!

/home/martin/cache/lib/a.out:0x402918: runtime error: load of value
4294967035, which is not a valid value for type 'std::_Ios_Fmtflags'
/home/martin/cache/lib/a.out:0x402b44: runtime error: load of value
4294967035, which is not a valid value for type 'std::_Ios_Fmtflags'

Two questions:

Should I ignore the warning, if not, how do I fix it?

Is there a way to suppress messages from the standard library? I'm guessing
not, but I can write a quick script on my end to filter them out.

This is a bug in libstdc++. You will be able to work around it with a
sanitizer blacklist file, once Will's patch for that lands, but for
now, filtering them out manually is likely to be your best option.

Here's a patch to fix it; I'll be looking into pushing this to
libstdc++ upstream in the next few days.

--- bits/ios_base.h
+++ bits/ios_base.h
@@ -87,7 +87,7 @@

   inline _GLIBCXX_CONSTEXPR _Ios_Fmtflags
   operator~(_Ios_Fmtflags __a)
- { return _Ios_Fmtflags(~static_cast<int>(__a)); }
+ { return _Ios_Fmtflags(static_cast<int>(__a) ^
static_cast<int>(_S_ios_fmtflags_end - 1)); }

   inline const _Ios_Fmtflags&
   operator>=(_Ios_Fmtflags& __a, _Ios_Fmtflags __b)
@@ -127,7 +127,7 @@

   inline _GLIBCXX_CONSTEXPR _Ios_Openmode
   operator~(_Ios_Openmode __a)
- { return _Ios_Openmode(~static_cast<int>(__a)); }
+ { return _Ios_Openmode(static_cast<int>(__a) ^
static_cast<int>(_S_ios_openmode_end - 1)); }

   inline const _Ios_Openmode&
   operator>=(_Ios_Openmode& __a, _Ios_Openmode __b)
@@ -165,7 +165,7 @@

   inline _GLIBCXX_CONSTEXPR _Ios_Iostate
   operator~(_Ios_Iostate __a)
- { return _Ios_Iostate(~static_cast<int>(__a)); }
+ { return _Ios_Iostate(static_cast<int>(__a) ^
static_cast<int>(_S_ios_iostate_end - 1)); }

   inline const _Ios_Iostate&
   operator>=(_Ios_Iostate& __a, _Ios_Iostate __b)

Sorry for bringing this thread up again...
I've updated clang and compiler-rt to current trunk. Now I have this error
trying to link against libMyLib.so which was linked with -fsanitize=integer
switch:

clang++ -fsanitize=integer ... -lMyLib -o ...
/.../libMyLib.so: undefined reference to `__mulodi4'
clang: error: linker command failed with exit code 1 (use -v to see
invocation)

I'm trying to localize the problem but have no big progress for now. It is
not reproduced on a simple shared lib project. Any ideas? Without -fsanitize
all works fine

See: http://llvm.org/bugs/show_bug.cgi?id=14469 , which I can
reproduce locally using -fsanitize=integer -m32 and by multiplying two
64bit integers together.

The issue is we emit overflow intrinsics that aren't neatly supported
on the platform and so a call to the supporting runtime is emitted
instead. Unfortunately libgcc doesn't implement __mulodi4
(compiler-rt does) resulting in the undefined reference you're seeing.

Unfortunately I don't know a good way to convince clang that
compiler-rt is available and to use that instead of libgcc, and it
seems Motsak in the bug report has the same question. As a kludge of
a workaround, you can manually add the libclang_rt library for your
architecture to your linker invocation (like:
/path/to/libclang_rt.i386.a), which I've used myself to get around
this issue in the past.

Perhaps someone more familiar with the issue can comment on the
challenges in resolving the linked bug?

Hope this helps,

~Will

See: http://llvm.org/bugs/show_bug.cgi?id=14469 , which I can
reproduce locally using -fsanitize=integer -m32 and by multiplying two
64bit integers together.

The issue is we emit overflow intrinsics that aren’t neatly supported
on the platform and so a call to the supporting runtime is emitted
instead. Unfortunately libgcc doesn’t implement __mulodi4
(compiler-rt does) resulting in the undefined reference you’re seeing.

Unfortunately I don’t know a good way to convince clang that
compiler-rt is available and to use that instead of libgcc, and it
seems Motsak in the bug report has the same question. As a kludge of
a workaround, you can manually add the libclang_rt library for your
architecture to your linker invocation (like:
/path/to/libclang_rt.i386.a), which I’ve used myself to get around
this issue in the past.

Perhaps someone more familiar with the issue can comment on the
challenges in resolving the linked bug?

Yes, I’ve seen this too. The easiest solution might be to include compiler-rt’s mulodi4.o in libclang_rt.ubsan.*.a. – that’s the approach I’ve been using when testing UBSan – although this doesn’t fix the problem for code not using UBSan.

The issue is we emit overflow intrinsics that aren't neatly supported
on the platform and so a call to the supporting runtime is emitted
instead. Unfortunately libgcc doesn't implement __mulodi4
(compiler-rt does) resulting in the undefined reference you're seeing.

Yes, I've seen this too. The easiest solution might be to include
compiler-rt's mulodi4.o in libclang_rt.ubsan.*.a. -- that's the approach
I've been using when testing UBSan -- although this doesn't fix the problem
for code not using UBSan.

ar rs libclang_rt.ubsan-i386.a mulodi4.o has solved the problem, thanks! If
ubsan is so dependent from this implementation may be it should be added to
the make script?