Shared libraries, msan and -z,defs

I recently tried to enabled building llvm libraries with -Wl,-z,defs.
The intention was to detect missing dependencies on ELF, so that we
don't
get into a situation where a .so builds, but the equivalent .dylib or
.dll fails.

This failed when building with msan because of undefined references to
functions like __msan_memcpy.

Unfortunately, I don't think elf linkers have a way of informing them
that a given symbol will be provided at runtime. It is possible to
pass a dummy executable build with msan, but that gets recorded in
DT_NEEDED.

My understanding is that the msan library provides two things: an
initialization code that has to run really early and auxiliary
functions like __msan_memcpy.

Would it be possible to split libclang_rt.msan-x86_64.a such that the
utility functions are available on a .so (or at least a -fPIC .a)?
That library could be linked with every .so. The early init logic
would still only be linked with executables.

Cheers,
Rafael

The sanitizer libc interceptors (malloc, etc) also need to be linked into the executable because the loader searches the executable first.

I think the sanitizers also rely on the initial executable TLS model.

Given the complexities, I don’t think it’s worth the effort to split. It might be worth it if there were more desire to support the shared library sanitizer build that GCC prefers.

What about creating an msan interface DSO for the purposes of satisfying -z defs? The executable will always be searched first, so the interface DSO could be empty or full of ud2a.

What about creating an msan interface DSO for the purposes of satisfying -z
defs? The executable will always be searched first, so the interface DSO
could be empty or full of ud2a.

I like the idea. The ideal solution would probably be for the linkers
to have a --not-needed command line option that prevents the .so from
going in the DT_NEEDED, but one with just stubs is probably fine.

Cheers,
Rafael

I think we did something in ASan to address this issue.

We need to chaise and kill all extra complexity, not add more complexity.
Pleeease, try finding another solution that does not involve splitting the run-times even further.

-z,defs does not solve any problem other than detecting “missing dependencies on ELF”,
so let’s just not use it with sanitizers.

–kcc

That just moves the complexity to the build system :frowning:

Cheers,
Rafael

What about creating an msan interface DSO for the purposes of satisfying -z
defs? The executable will always be searched first, so the interface DSO
could be empty or full of ud2a.

I like the idea. The ideal solution would probably be for the linkers
to have a --not-needed command line option that prevents the .so from
going in the DT_NEEDED, but one with just stubs is probably fine.

But then the program would fail to start if the (empty) interface
library is not available at runtime library search path.
Which is the common situation, unless you install clang as a
system-wide compiler. This will create lots of deployment problems.

True, but it's only a headache for people using -z defs.

I'm surprised there isn't a linker option we can use like '-u
__msan_memcpy' to say "yeah, this symbol is undefined, but it's expected to
be present at runtime".

But then the program would fail to start if the (empty) interface
library is not available at runtime library search path.
Which is the common situation, unless you install clang as a
system-wide compiler. This will create lots of deployment problems.

True, but it's only a headache for people using -z defs.

I'm surprised there isn't a linker option we can use like '-u __msan_memcpy'
to say "yeah, this symbol is undefined, but it's expected to be present at
runtime".

bfd ld has --ignore-unresolved-symbol, but it is missing from gold.

Cheers,
Rafael

bfd ld has --ignore-unresolved-symbol, but it is missing from gold.

For gold, there's --unresolved-symbols=ignore-all

-Greg

But that basically cancels -z,defs :slight_smile:

Cheers,
Rafael