i386 -faddress-sanitizer

Is it expected that -faddress-sanitizer does not work on i386 and fails
to properly link resulting in errors like 'undefined symbol: __asan_init'
at runtime?

I have ./lib/clang/3.2/lib/linux/libclang_rt.asan-i386.a in llvm's installation,
but it doesn't look like it's linked automatically into the executable.

CFLAGS:
-faddress-sanitizer -O1 -fno-omit-frame-pointer -g $PROJECT_FLAGS

clang version 3.2
(http://llvm.org/git/clang.git 1802daf8539f3320c369e442d3c8471b10ffd3cb)
(http://llvm.org/git/llvm.git 324d96b9e265b0fd8bf63a28340910def64e2164)
Target: i386-pc-linux-gnu
Thread model: posix

You need to pass in -faddress-sanitizer both when compiling and when
linking an application. Try something like
"LDFLAGS=-faddress-sanitizer".

-Eli

Oh that makes a lot of sense and is kind of obvious in hindsight.
Yet it didn't work in a quick test. Any options I flags to pass for
verbose printfs to see what's happening you can suggest?
In the next test I'll also try to explicitly link the libasan archive
manually for ruling out issues with that archive.

Are you linking with ld or clang?
-faddress-sanitizer is a Clang flag, so if you're running `ld $LDFLAGS
...` to link your program, it won't work.

Are you linking with ld or clang?
-faddress-sanitizer is a Clang flag, so if you're running `ld $LDFLAGS
...` to link your program, it won't work.

clang.

Can you append -v at the place it's being invoked to check whether the
contents of LDFLAGS are being actually passed?

Can you append -v at the place it's being invoked to check whether the
contents of LDFLAGS are being actually passed?

$ clang <INPUT_OBJECTS> -lstdc++ -v -faddress-sanitizer -shared
<PROJECT_FLAGS> -o <SO_FILE>
clang
version 3.2 (http://llvm.org/git/clang.git
278eafa2cd8296f8128d13c6466a0ace3d03a872)
(http://llvm.org/git/llvm.git
2360b7ad99eedecaae512373e7be49c2143550cf)
Target: i386-pc-linux-gnu
Thread model: posix

"/usr/bin/ld" --eh-frame-hdr -m elf_i386 -shared -o <SO_FILE>
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/../../../crti.o
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/crtbeginS.o
-L/usr/lib/gcc/i686-pc-linux-gnu/4.7.2
-L/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/../../.. -L/lib -L/usr/lib
<INPUT_OBJECTS> -lstdc++ -lgcc --as-needed -lgcc_s --no-as-needed -lc
-lgcc --as-needed -lgcc_s --no-as-needed
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/crtendS.o
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/../../../crtn.o

I guess the libasan archive should be part of the above call to ld.

You are linking a shared object, asan.a should not be linked to it.
Are these all the flags passed to ld?

You are linking a shared object, asan.a should not be linked to it.

What should happen instead?

Are these all the flags passed to ld?

Yes it's all minus project-specific -l and *.a and *.o args
I have omitted for the email.

Just confirmed that manually passing the asan.a archive
in the ld command line fixes the issue but this should
of course happen automatically.

You are linking a shared object, asan.a should not be linked to it.

What should happen instead?

nothing. asan.a should be linked only to the main executable.

Are these all the flags passed to ld?

Yes it’s all minus project-specific -l and *.a and *.o args
I have omitted for the email.

don’t you have something like -Wl,-z,defs ?
The problem you describe was seen before in the presence of -Wl,-z,defs, which forces the definitions of all symbols in .so (and __asan* symbols are not defined).
The solution was to remove -Wl,-z,defs.

–kcc

> You are linking a shared object, asan.a should not be linked to it.

What should happen instead?

nothing. asan.a should be linked only to the main executable.

I don't build an executable. This is a shared object which gets
loaded at runtime. Specifically it's a .so implementing a native
interface binding used with the Erlang runtime.

> Are these all the flags passed to ld?

Yes it's all minus project-specific -l and *.a and *.o args
I have omitted for the email.

don't you have something like -Wl,-z,defs ?
The problem you describe was seen before in the presence of -Wl,-z,defs,
which forces the definitions of all symbols in .so (and __asan* symbols are
not defined).
The solution was to remove -Wl,-z,defs.

I can't find any of that in the verbose clang linker call's logs.

You have two issues here then.
First, for some reason the linker doesn't like the unresolved calls to
ASan runtime in your shared object. Can you please attach the full
command line used to invoke the linker (feel free to obfuscate the
file names if needed, but please keep the flags).
Second, you're going to have problems with unresolved symbols once you
load your shared object into a non-instrumented executable.
The only way to detect any errors in a shared library so far is to
rebuild the executable loading that library with ASan.
Please don't link libasan.a with the .so anyway, because you're likely
to have problems with the initialization order. ASan runtime needs to
be initialized at program startup.

You have two issues here then.
First, for some reason the linker doesn't like the unresolved calls to
ASan runtime in your shared object. Can you please attach the full
command line used to invoke the linker (feel free to obfuscate the
file names if needed, but please keep the flags).

clang c_src/<PROJECT>.o -lstdc++ -v -faddress-sanitizer
  -shared
  -L/usr/local/lib/erlang/lib/erl_interface-3.7.8/lib
  -lerl_interface -lei -o priv/<PROJECT>.so
clang version 3.2 (http://llvm.org/git/clang.git
278eafa2cd8296f8128d13c6466a0ace3d03a872)
(http://llvm.org/git/llvm.git
2360b7ad99eedecaae512373e7be49c2143550cf)
Target: i386-pc-linux-gnu
Thread model: posix
"/usr/bin/ld" --eh-frame-hdr -m elf_i386 -shared -o priv/<PROJECT>.so
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/../../../crti.o
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/crtbeginS.o
-L/usr/local/lib/erlang/lib/erl_interface-3.7.8/lib
-L/usr/lib/gcc/i686-pc-linux-gnu/4.7.2
-L/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/../../.. -L/lib -L/usr/lib
c_src/<PROJECT>.o -lstdc++
-lerl_interface -lei -lgcc --as-needed -lgcc_s
--no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/crtendS.o
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/../../../crtn.o

Second, you're going to have problems with unresolved symbols once you
load your shared object into a non-instrumented executable.
The only way to detect any errors in a shared library so far is to
rebuild the executable loading that library with ASan.
Please don't link libasan.a with the .so anyway, because you're likely
to have problems with the initialization order. ASan runtime needs to
be initialized at program startup.

This makes sense and I'll build an instrumented Erlang runtime
as a next step. With that in place do I still have to enable asan
in CFLAGS when building the object file? I would say yes.

Is this detail documented? If not maybe it should but I may have
been not reading carefully enough.

You have two issues here then.
First, for some reason the linker doesn’t like the unresolved calls to
ASan runtime in your shared object. Can you please attach the full
command line used to invoke the linker (feel free to obfuscate the
file names if needed, but please keep the flags).

clang c_src/.o -lstdc++ -v -faddress-sanitizer
-shared
-L/usr/local/lib/erlang/lib/erl_interface-3.7.8/lib
-lerl_interface -lei -o priv/.so

clang version 3.2 (http://llvm.org/git/clang.git
278eafa2cd8296f8128d13c6466a0ace3d03a872)
(http://llvm.org/git/llvm.git
2360b7ad99eedecaae512373e7be49c2143550cf)
Target: i386-pc-linux-gnu
Thread model: posix

“/usr/bin/ld” --eh-frame-hdr -m elf_i386 -shared -o priv/.so
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/…/…/…/crti.o
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/crtbeginS.o
-L/usr/local/lib/erlang/lib/erl_interface-3.7.8/lib
-L/usr/lib/gcc/i686-pc-linux-gnu/4.7.2
-L/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/…/…/… -L/lib -L/usr/lib
c_src/.o -lstdc++
-lerl_interface -lei -lgcc --as-needed -lgcc_s

–no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/crtendS.o
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/…/…/…/crtn.o

Hm. I don’t see any reason for the failure here… Anyone?

Second, you’re going to have problems with unresolved symbols once you
load your shared object into a non-instrumented executable.
The only way to detect any errors in a shared library so far is to
rebuild the executable loading that library with ASan.
Please don’t link libasan.a with the .so anyway, because you’re likely
to have problems with the initialization order. ASan runtime needs to
be initialized at program startup.

This makes sense and I’ll build an instrumented Erlang runtime
as a next step. With that in place do I still have to enable asan
in CFLAGS when building the object file? I would say yes.

You need to use -faddress-sanitizer while building the objects you want to test.
You don’t need to instrument Erlang sources (but most likely it won’t hurt), just make sure that
-faddress-sanitizer is passed to the link command.

BTW, we have the same problem with python.
In our private setting we solved it with “LD_PRELOAD=asan.so python x.py”,
but I would not recommend this as a generally good approach.

Is this detail documented?

Probably no. I think we explain how asan works in detail, from which you can deduct these limitations,
but that’s not trivial.

If not maybe it should

Mmm. Perhaps.

–kcc

> You have two issues here then.
> First, for some reason the linker doesn't like the unresolved calls to
> ASan runtime in your shared object. Can you please attach the full
> command line used to invoke the linker (feel free to obfuscate the
> file names if needed, but please keep the flags).

clang c_src/<PROJECT>.o -lstdc++ -v -faddress-sanitizer
  -shared
  -L/usr/local/lib/erlang/lib/erl_interface-3.7.8/lib
  -lerl_interface -lei -o priv/<PROJECT>.so
clang version 3.2 (http://llvm.org/git/clang.git
278eafa2cd8296f8128d13c6466a0ace3d03a872)
(http://llvm.org/git/llvm.git
2360b7ad99eedecaae512373e7be49c2143550cf)
Target: i386-pc-linux-gnu
Thread model: posix
"/usr/bin/ld" --eh-frame-hdr -m elf_i386 -shared -o priv/<PROJECT>.so
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/../../../crti.o
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/crtbeginS.o
-L/usr/local/lib/erlang/lib/erl_interface-3.7.8/lib
-L/usr/lib/gcc/i686-pc-linux-gnu/4.7.2
-L/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/../../.. -L/lib -L/usr/lib
c_src/<PROJECT>.o -lstdc++
-lerl_interface -lei -lgcc --as-needed -lgcc_s
--no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/crtendS.o
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/../../../crtn.o

Hm. I don't see any reason for the failure here... Anyone?

Is the probably special handling for .so files that automatically
prevents linking in asan.a?

> Second, you're going to have problems with unresolved symbols once you
> load your shared object into a non-instrumented executable.
> The only way to detect any errors in a shared library so far is to
> rebuild the executable loading that library with ASan.
> Please don't link libasan.a with the .so anyway, because you're likely
> to have problems with the initialization order. ASan runtime needs to
> be initialized at program startup.

This makes sense and I'll build an instrumented Erlang runtime
as a next step. With that in place do I still have to enable asan
in CFLAGS when building the object file? I would say yes.

You need to use -faddress-sanitizer while building the objects you
want to test. You don't need to instrument Erlang sources (but most
likely it won't hurt), just make sure that -faddress-sanitizer is
passed to the link command.

That's good to know if all I want to check are faults in my
shared object only.

I can confirm it basically works now with the rebuilt
Erlang runtime. Thanks everybody!

Speaking of finding errors, I never was able to find
errors in one or two projects I tried the clang static
analyzer on. Is there really nothings else to than
running the project's build wrapped with the scanner?

You have two issues here then.
First, for some reason the linker doesn't like the unresolved calls to
ASan runtime in your shared object. Can you please attach the full
command line used to invoke the linker (feel free to obfuscate the
file names if needed, but please keep the flags).

clang c_src/<PROJECT>.o -lstdc++ -v -faddress-sanitizer
-shared
-L/usr/local/lib/erlang/lib/erl_interface-3.7.8/lib
-lerl_interface -lei -o priv/<PROJECT>.so
clang version 3.2 (http://llvm.org/git/clang.git
278eafa2cd8296f8128d13c6466a0ace3d03a872)
(http://llvm.org/git/llvm.git
2360b7ad99eedecaae512373e7be49c2143550cf)
Target: i386-pc-linux-gnu
Thread model: posix
"/usr/bin/ld" --eh-frame-hdr -m elf_i386 -shared -o priv/<PROJECT>.so
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/../../../crti.o
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/crtbeginS.o
-L/usr/local/lib/erlang/lib/erl_interface-3.7.8/lib
-L/usr/lib/gcc/i686-pc-linux-gnu/4.7.2
-L/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/../../.. -L/lib -L/usr/lib
c_src/<PROJECT>.o -lstdc++
-lerl_interface -lei -lgcc --as-needed -lgcc_s
--no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/crtendS.o
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/../../../crtn.o

Hm. I don't see any reason for the failure here... Anyone?

Is the probably special handling for .so files that automatically
prevents linking in asan.a?

Second, you're going to have problems with unresolved symbols once you
load your shared object into a non-instrumented executable.
The only way to detect any errors in a shared library so far is to
rebuild the executable loading that library with ASan.
Please don't link libasan.a with the .so anyway, because you're likely
to have problems with the initialization order. ASan runtime needs to
be initialized at program startup.

This makes sense and I'll build an instrumented Erlang runtime
as a next step. With that in place do I still have to enable asan
in CFLAGS when building the object file? I would say yes.

You need to use -faddress-sanitizer while building the objects you
want to test. You don't need to instrument Erlang sources (but most
likely it won't hurt), just make sure that -faddress-sanitizer is
passed to the link command.

That's good to know if all I want to check are faults in my
shared object only.

BTW, we have the same problem with python.
In our private setting we solved it with "LD_PRELOAD=asan.so python x.py",
but I would not recommend this as a generally good approach.

Is this detail documented?

Probably no. I think we explain *how* asan works in detail, from which you
can deduct these limitations,
but that's not trivial.

If not maybe it should

Mmm. Perhaps.

I can confirm it basically works now with the rebuilt
Erlang runtime. Thanks everybody!

Speaking of finding errors, I never was able to find
errors in one or two projects I tried the clang static
analyzer on. Is there really nothings else to than
running the project's build wrapped with the scanner?

That should have worked. Make sure that your project is clean before analyzing. You can also try inserting an error and checking if it is caught (something like: int foo() {int x = 0; return 5/x;}).

Anna.

You have two issues here then.
First, for some reason the linker doesn’t like the unresolved calls to
ASan runtime in your shared object. Can you please attach the full
command line used to invoke the linker (feel free to obfuscate the
file names if needed, but please keep the flags).

clang c_src/.o -lstdc++ -v -faddress-sanitizer
-shared
-L/usr/local/lib/erlang/lib/erl_interface-3.7.8/lib
-lerl_interface -lei -o priv/.so
clang version 3.2 (http://llvm.org/git/clang.git
278eafa2cd8296f8128d13c6466a0ace3d03a872)
(http://llvm.org/git/llvm.git
2360b7ad99eedecaae512373e7be49c2143550cf)
Target: i386-pc-linux-gnu
Thread model: posix
“/usr/bin/ld” --eh-frame-hdr -m elf_i386 -shared -o priv/.so
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/…/…/…/crti.o
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/crtbeginS.o
-L/usr/local/lib/erlang/lib/erl_interface-3.7.8/lib
-L/usr/lib/gcc/i686-pc-linux-gnu/4.7.2
-L/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/…/…/… -L/lib -L/usr/lib
c_src/.o -lstdc++
-lerl_interface -lei -lgcc --as-needed -lgcc_s
–no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/crtendS.o
/usr/lib/gcc/i686-pc-linux-gnu/4.7.2/…/…/…/crtn.o

Hm. I don’t see any reason for the failure here… Anyone?

Is the probably special handling for .so files that automatically
prevents linking in asan.a?

In the clang driver we check what is being linked and pass libasan.a to ld only if the final executable is being linked.
So, a shared library just doesn’t get linked with asan.a, but it still has _asan_report* callbacks,
which may cause link errors if you are using -Wl,-z,defs (or something similar).
Looks like you have resolved the problem, right?

Let us know if the tool works for your case (and if you find bugs in Erlang :slight_smile:

BTW, I’ve extended the docs a bit: http://clang.llvm.org/docs/AddressSanitizer.html#usage

–kcc

Thanks Anna that confirmed scan-build and scan-view work.
How did I not think of inserting buggy code. I must have assumed
my code is buggy enough.