llvm-strip creates unloadable shared objects on linux-armv7hf

Hello,

Recently we tried to streamline our toolchain by removing some GNU
tools with LLVM tools to avoid having multiple copies of strip, nm, ar
and similar tools. Today we ran into a really strange issue where
shared objects where not loadable.

We where able to track this down to llvm-strip with default arguments
creating a shared object that doesn't load correctly. It works if we
switch from not passing args to llvm-strip to pass -strip-all-gnu

I built bzip2 and libbz2.so to show this case here:

This is the error we got by using the stripped shared object with
llvm-strip -strip-all:

admin@Netgear-RN212:~/b$ LD_PRELOAD=./libbz2.so.all ./bzip2
ERROR: ld.so: object './libbz2.so.all' from LD_PRELOAD cannot be
preloaded (cannot open shared object file): ignored.
./bzip2: error while loading shared libraries: libbz2.so: cannot open
shared object file: No such file or directory

And with strip-all-gnu it works:
admin@Netgear-RN212:~/b$ LD_PRELOAD=./libbz2.so.all_gnu ./bzip2
bzip2: I won't write compressed data to a terminal.
bzip2: For help, type: `bzip2 --help'.

This only seems to happen on Linux-armv7hf - we haven't seen this on
any of the intel platforms.

Thanks,
Tobias

One thing I noticed is that llvm-strip seemed to remove a .ARM.attributes section. Can you try --keep-section=.ARM.attributes to tell to the command to keep the section?

Hello Rui,

Thanks for your reply. I tried with the keep-section argument and that
made the shared library work.

Should these sections be kept around by default maybe?

-- Tobias

I’m not ARM expert, but yes I guess so.

Hello Tobias,

Does your system happen to be using eglibc? I ran into this problem on
an Ubuntu 14.04 system and it was down to the eglibc dynamic loader
using the .ARM.attributes section when performing dlopen. Strictly
speaking the .ARM.attributes section is only defined for relocatable
objects, the ABI says that their presence in executables or dynamic
loaders is neither required or forbidden, so it is somewhat risky for
any portable program to depend on their presence.

Since eglibc was merged back into glibc this dependency on
.ARM.attributes went away. For LLD we took the position that it was
worth keeping the .ARM.attributes to placate eglibc, as this was more
likely to be encountered 2 years ago. I've not got a strong position
on llvm-strip, in theory it should be strippable from executable and
shared libraries, but there may be a case that eglibc is important
enough to not strip by default.

Peter

Hello Peter,

I was able to fix this issue with this simple patch against llvm-objcopy:

diff --git a/llvm/tools/llvm-objcopy/ELF/ELFObjcopy.cpp
b/llvm/tools/llvm-objcopy/ELF/ELFObjcopy.cpp
index dd6a7d7e14b..c0dfd3a9838 100644
--- a/llvm/tools/llvm-objcopy/ELF/ELFObjcopy.cpp
+++ b/llvm/tools/llvm-objcopy/ELF/ELFObjcopy.cpp
@@ -503,6 +503,8 @@ static Error replaceAndRemoveSections(const
CopyConfig &Config, Object &Obj) {
return false;
if (StringRef(Sec.Name).startswith(".gnu.warning"))
return false;
+ if (StringRef(Sec.Name).startswith(".ARM.attributes"))
+ return false;
if (Sec.ParentSegment != nullptr)
return false;
return (Sec.Flags & SHF_ALLOC) == 0;

Is this a good way of fixing the issue? Happy to read up on how to
submit patches if you think this is the way to go.

-- Tobias

Hello Tobias,

I think that looks reasonable to me, I think it will be down to the
llvm-objcopy team whether they want to make .ARM.attributes a special
case or not. The best way to find out is to submit a patch, citing the
problems with old versions of libc, I'd expect that you'll need to add
a test case for the patch to be accepted. To do that it is probably
best to look at the existing tests for llvm-strip and try and copy
them. The test could be as simple as generating a binary with a
section of type SHT_ARM_ATTRIBUTES and checking that strip didn't
remove it. These tests sometimes use yaml2obj to generate an ELF file
without needing a compiler and linker. Running the tests should be as
simple as ninja check-llvm or make check-llvm depending on whether you
used ninja or make when building llvm. If you want to run just one
test then you can use bin/llvm-lit -v -a /path/to/test.s (from your
build directory).

The instructions on how to contribute are in
LLVM Developer Policy — LLVM 16.0.0git documentation the people that I know have
been active in llvm-objdump are MaskRay (Fangrui Song), rupprect
(Jordan Rupprecht), grimar (George Rimar). If you include these people
on the reviewers then I'm sure they'll be able to add anyone else that
they think would be interested.

Hope this helps

Peter

Tobias,
I don’t have much experience with ARM, but from your report and Peter’s explanation of why LLD does it, I agree we should be consistent with LLD and keep the section.

Feel free to send a patch! (btw, my review handle is rupprecht, not rupprect).

Jordan,

I have sent the patch via Phabricator: https://reviews.llvm.org/D69188

Let me know if I got it right.

-- Tobias

Everyone,

Just to close this loop, the patch was accepted into master today:
https://github.com/llvm/llvm-project/commit/fb4a55010ee9bd03720609c8542f770775576fc8

Thanks for everyone guiding me!

Tobias

Thanks for your contribution! I’m sure I can speak for all of us when I say that we look forward to more contributions from you, if you see anything else you’d like to work on.

James

Same here. Thanks Tobias!