Common Symbols causing issues

Subprojects LLD
1

I have some code that crashes when using lld to link a shared object. So essentially it boils down to an object file having a common symbol, and a shared object having the definition of said object. Look here:

$ nm libfckit_lite.so | grep "\$data_init\$fckit_owned_object_module_"
0000000000008700 D $data_init$fckit_owned_object_module_

$ nm atlas_object.F90.o | grep "\$data_init\$fckit_owned_object_module_"
0000000000000238 C $data_init$fckit_owned_object_module_

When linking and creating a shared object this is the result(I also added the final executable):

$ nm libatlas_lite_lld.so  | grep  "\$data_init\$fckit_owned_object_module_"
000000000000e240 B $data_init$fckit_owned_object_module_

$ nm libatlas_lite_bfd.so | grep "\$data_init\$fckit_owned_object_module_"
                 U $data_init$fckit_owned_object_module_

$ nm exe_lld | grep "\$data_init\$fckit_owned_object_module_"
0000000000404700 B $data_init$fckit_owned_object_module_

$ nm exe_bfd | grep "\$data_init\$fckit_owned_object_module_"
                 U $data_init$fckit_owned_object_module_

It’s also worth noting that mold seems to behave like bfd, and leave the common symbol undefined. Is this an lld issue?

2

That is a difficult question to answer as far as I know there isn’t anything that I know of in the ELF specification that covers this case, which would put it in the realm of implementation defined behaviour. Whether this is an lld issue for not conforming to the same implementation defined behaviour as GNU ld and mold, or a bug in the program for relying upon it is a matter of debate. LLD normally likes to be compatible with GNU ld where possible so it could be worth raising a github issue for it.

Normally a non-common non-weak definition with the same name as a common symbol will take precedent. So by that logic a non-common definition in a shared object would take precedence over a common definition from a relocatable object. This does run counter to the usual symbol search path rules that a definition in an executable takes precedent over a definition in a shared library.

3

Thanks for the response! I did what you suggested and created an issue here: `COMMON` Symbols causing issues when linking with LLD · Issue #73693 · llvm/llvm-project · GitHub

4

I think we need the linker command line and input files to answer your question (use LLD_REPRODUCE or -Wl,--reproduce=, and upload the reproduce tarball), because (a) the order of libfckit_lite.so and atlas_object.F90.o and (b) whether atlas_object.F90.o is in --start-lib/--end-lib change the result. We need more information to know why exe_bfd keeps $data_init$fckit_owned_object_module_ undefined.

In ELF, for a relocatable object file definition, the precedence is STB_GLOBAL > COMMON > STB_WEAK.

If we have a defined symbol ins a relocatable (whether it is STB_GLOBAL, COMMON, or STB_WEAK), the shared object definition will be ignored except its st_size (see below). I think lld implements the correct semantics.

From All about COMMON symbols | MaskRay

When a common symbol is merged with a shared symbol, GNU ld and ld.lld (see D71161) increase st_size if the shared symbol has a larger st_size.

gold and mold don’t implement this rule.

5

The files are in the github issue, the link line isn’t too complicated:

"/home/users/saldivar/.local/bin/ld.lld" --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -shared -o libatlas_lite_lld.so /usr/lib64/gcc/x86_64-suse-linux/12/../../../../lib64/crti.o /usr/lib64/gcc/x86_64-suse-linux/12/crtbeginS.o -L/usr/lib64/gcc/x86_64-suse-linux/12 -L/usr/lib64/gcc/x86_64-suse-linux/12/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/lib -L/lib -L/usr/lib -rpath $PWD -soname libatlas_lite_lld.so atlas_object.cc.o atlas_object.F90.o atlas_object_c_binding.F90.o libfckit_lite.so -lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/lib64/gcc/x86_64-suse-linux/12/crtendS.o /usr/lib64/gcc/x86_64-suse-linux/12/../../../../lib64/crtn.o

In regards to mold (bfd code isn’t the easiest to follow, at least for myself), by the looks of this code, it doesn’t seem like they do that rule you mentioned. But this is the portion of code that switches the definition of the COMMON symbol from the .o file to the .so.

6

Thanks for the reproduce tarball in [LLD] `COMMON` Symbols causing issues when linking with LLD · Issue #73693 · llvm/llvm-project · GitHub . I have figured out GNU ld’s behavior and I believe it doesn’t make sense.

I have updated All about COMMON symbols | MaskRay to include the following text:

When a common symbol is merged with a dynamic symbol, GNU ld displays different behaviors depending on whether the dynamic symbol is relative to a BSS section.
See the definition of newdyncommon in bfd/elflink.c:_bfd_elf_merge_symbol (originally bfd/elflink.h from “19990502 sourceware import”).

as -o a.o <<< '.comm x,8,4'
as -o bss.o <<< '.bss; .globl x; x: .quad 0; .size x, .-x' && ld -shared bss.o -o bss.so
as -o data.o <<< '.data; .globl x; x: .quad 0; .size x, .-x' && ld -shared data.o -o data.so
ld.bfd a.o bss.so -o bfd.bss  # x is defined
ld.bfd a.o data.so -o bfd.data  # x is undefined
ld.lld a.o bss.so -o lld.bss  # x is defined
ld.lld a.o data.so -o lld.data  # x is defined

This special case does not make sense to me. In a relocatable object file, a COMMON symbol is stronger than a weak symbol.
Since a weak symbol takes precedence over a dynamic symbol, a COMMON symbol should as well.
gold and ld.lld unconditionally let the common symbol win.

Let’s say the st_size is expected. If the DSO’s x references bind to the executable (symbol interposition), whether or not x is defined or undefined in the executable does not matter.

7

Would you mind elaborating a bit more on symbol precedence regarding dynamic symbols? I’m just a bit confused because I thought this symbol would be a GLOBAL symbol based on this readelf output:

77: 0000000000008700   568 OBJECT  GLOBAL DEFAULT   19 $data_init$fckit_owned_object_module_

And based on your blog post, wouldn’t this have more precedence, because of: STB_GLOBAL > COMMON > STB_WEAK?

8

Also, thanks for pointing out the function in the bfd code!

9

My post says: “In ELF, for a definition in a relocatable object file, the precedence is STB_GLOBAL > COMMON > STB_WEAK.”

The interaction between a relocatable object file definition and a dynamic symbol (definition in a DSO) is not well specified in the generic ABI. They belong to implementation details. I have some notes on Symbol processing | MaskRay " Relocatable object file overriding shared object" has the relevant discussion.

If we have COMMON symbols in a.o and b.o, with st_size(x in a.o) > st_size(x in b.o). When linking a.o b.so, it’s dangerous to leave x undefined in the executable since there would likely be out-of-bounds access. If we keep x defined and apply the max st_size rule, with symbol interposition, accesses to x will be correct.

10

So, I created a minimal example, that kinda represents what the application I posted does:

$ cat build.sh
#!/usr/bin/env bash
set -x

echo '.comm x,8,4' | as -o a.o
echo '.data; .globl x; x: .quad 42; .size x, .-x' | as -o b.o

ld -shared b.o -o b.so
nm b.so

ld.lld -shared a.o b.so  -o a_lld.so
nm a_lld.so

ld -shared a.o b.so  -o a_bfd.so
nm a_bfd.so

cat main.c
clang -c main.c -fPIC

clang a_lld.so b.so main.o -Wl,-rpath,$PWD -o out_lld
./out_lld
echo $?

clang a_bfd.so b.so main.o -Wl,-rpath,$PWD -o out_bfd
./out_bfd
echo $?

$ ./build.sh
+ echo '.comm x,8,4'
+ as -o a.o
+ echo '.data; .globl x; x: .quad 42; .size x, .-x'
+ as -o b.o
+ ld -shared b.o -o b.so
+ nm b.so
0000000000002f40 d _DYNAMIC
0000000000003000 D x
+ ld.lld -shared a.o b.so -o a_lld.so
+ nm a_lld.so
0000000000002270 d _DYNAMIC
00000000000032f0 B x
+ ld -shared a.o b.so -o a_bfd.so
+ nm a_bfd.so
0000000000002f30 d _DYNAMIC
                 U x
+ cat main.c
extern int x;
int main(int argc, char *argv[]) { return x; }
+ clang -c main.c -fPIC
+ clang a_lld.so b.so main.o -Wl,-rpath,$PWD -o out_lld
+ ./out_lld
+ echo 0
0
+ clang a_bfd.so b.so main.o -Wl,-rpath,$PWD -o out_bfd
+ ./out_bfd
+ echo 42
42

From this, is what bfd is doing incorrect then?

11

I think GNU ld’s behavior is inconsistent (due it changing behavior whether x is in BSS or not), but we probably should not say its behavior is incorrect. This interaction with dynamic symbols is largely implementation defined. In this case we could say it’s a user fault to give linkers fishy input.

12

Looks like the behavior was added to bfd with this commit:

commit 77bd8dfa1f3678ea3c3d05f40de29a36802d21f5
Author: Ian Lance Taylor <ian@airs.com>
Date:   Thu May 15 21:38:26 1997 +0000

            * elflink.h (elf_link_add_object_symbols): Consistently treat
            uninitialized symbols in shared objects as common symbols.

diff --git a/bfd/ChangeLog b/bfd/ChangeLog
index 6bcd44a5b94..84f3d403a55 100644
--- a/bfd/ChangeLog
+++ b/bfd/ChangeLog
@@ -11,7 +11,8 @@ Thu May 15 14:31:28 1997  Nick Clifton  <nickc@cygnus.com>
 Thu May 15 16:40:20 1997  Ian Lance Taylor  <ian@cygnus.com>
 
        * elflink.h (elf_link_add_object_symbols): Force ELF symbol size
-       to common symbol size.
+       to common symbol size.  Consistently treat uninitialized symbols
+       in shared objects as common symbols.
 
 Fri May  9 10:15:27 1997  Nick Clifton  <nickc@cygnus.com>
 
diff --git a/bfd/elflink.h b/bfd/elflink.h
index 43425c02f8b..78f57ffc63e 100644
--- a/bfd/elflink.h
+++ b/bfd/elflink.h
@@ -913,26 +913,16 @@ elf_link_add_object_symbols (abfd, info)
              && definition
              && h->root.type == bfd_link_hash_common
              && (sec->flags & SEC_ALLOC) != 0
-             && (sec->flags & SEC_LOAD) == 0)
+             && (sec->flags & SEC_LOAD) == 0
+             && sym.st_size > 0
+             && bind != STB_WEAK
+             && ELF_ST_TYPE (sym.st_info) != STT_FUNC)
            {
-             if (! ((*info->callbacks->multiple_common)
-                    (info, h->root.root.string,
-                     h->root.u.c.p->section->owner, bfd_link_hash_common,
-                     h->root.u.c.size, abfd, bfd_link_hash_common,
-                     sym.st_size)))
-               goto error_return;
-
-             /* If the symbol in the shared library is smaller than
-                 the one we already have, then override it to stick
-                 with the larger symbol.  Set SIZE_CHANGE_OK because
-                 we only want to warn if requested with --warn-common.  */
-             if (sym.st_size < h->size)
-               {
-                 override = true;
-                 sec = bfd_und_section_ptr;
-                 definition = false;
-                 size_change_ok = true;
-               }
+             override = true;
+             sec = bfd_com_section_ptr;
+             definition = false;
+             value = sym.st_size;
+             size_change_ok = true;
            }
 
          /* Similarly, if we are not looking at a dynamic object, and

Not really any other context, but I guess it’s good to know when it was added.