Some thought on handling ELF shared libraries in lld

Project Infrastructure LLVM Dev List Archives
1

Most ELF shared libraries can be sliced in two ways. One is following
the information in the program headers (e_phoff). The other is
following the information in the section headers (e_shoff).

Regular relocatable objects only have the section header.

At runtime, the dynamic linker only uses the program headers. In fact,
the section headers is optional.

When given a shared library, how should the static linker handle it?

Note that, unlike the dynamic linker, the static one has to find all
the defined symbol is a shared library. It is not enough to just look
up the currently undefined symbols. To see that, consider

$ cat test.c
void f(void) { }
$ cat test2.c
void f(void);
void g(void) {
  f();
}

$ clang -c test.c test2.c -fPIC
$ clang -shared test.o -o test.so
$ rm -f test.a
$ ar rc test.a test.o
$ clang test.so test2.o test.a -o t.so -Wl,-t -shared
$ clang test2.o test.a -o t.so -Wl,-t -shared

The second link will include the archive member, the first one will not.

It is tempting to use the program headers in the static linker. Doing
so would let us support linking with shared libraries with no section
headers, but there are a few issues:

* The intention of the spec seems to be for everything static to use
the section headers and everything dynamic to use the program headers.
* Finding the number of symbols with the program header in a
traditional ELF file is a hack. One has to read the nchain field of
the hash table.
* It doesn't seem even possible to find that information in files
using the newer gnu hash format
(https://blogs.oracle.com/ali/entry/gnu_hash_elf_sections).

Given that, it looks like we should use the sections. For what it is
worth, it looks like that is what every other ELF linker does.

Cheers,
Rafael

2

From: "Rafael Espíndola" <rafael.espindola@gmail.com>
To: "LLVM Developers Mailing List" <llvmdev@cs.uiuc.edu>
Sent: Tuesday, July 21, 2015 7:46:58 AM
Subject: [LLVMdev] Some thought on handling ELF shared libraries in lld

Most ELF shared libraries can be sliced in two ways. One is following
the information in the program headers (e_phoff). The other is
following the information in the section headers (e_shoff).

Regular relocatable objects only have the section header.

At runtime, the dynamic linker only uses the program headers. In
fact,
the section headers is optional.

When given a shared library, how should the static linker handle it?

Note that, unlike the dynamic linker, the static one has to find all
the defined symbol is a shared library. It is not enough to just look
up the currently undefined symbols. To see that, consider

$ cat test.c
void f(void) { }
$ cat test2.c
void f(void);
void g(void) {
  f();
}

$ clang -c test.c test2.c -fPIC
$ clang -shared test.o -o test.so
$ rm -f test.a
$ ar rc test.a test.o
$ clang test.so test2.o test.a -o t.so -Wl,-t -shared
$ clang test2.o test.a -o t.so -Wl,-t -shared

The second link will include the archive member, the first one will
not.

It is tempting to use the program headers in the static linker. Doing
so would let us support linking with shared libraries with no section
headers, but there are a few issues:

* The intention of the spec seems to be for everything static to use
the section headers and everything dynamic to use the program
headers.
* Finding the number of symbols with the program header in a
traditional ELF file is a hack. One has to read the nchain field of
the hash table.
* It doesn't seem even possible to find that information in files
using the newer gnu hash format
(https://blogs.oracle.com/ali/entry/gnu_hash_elf_sections).

Why do you need the total number of symbols?

-Hal

3

Sounds reasonable. The sstrip tool that performs stripping of the
section header does state that this makes shared libraries unsuitable
for static linking. From the documentation:

A shared-object library stripped in this fashion will still be usable by the dynamic linker, but not by the static linker.

That said, it should be technically possible to statically link a
shared library with no section header table if we chose to support
that use case. Can't you use the symbol indices of dynamic relocations
to achieve this instead of counting the number of symbols?

4

To find all the defined ones. See the above example.

Cheers,
Rafael

5

It is technically possible. But we have to find the total symbols
early (symbol resolution) and so we would need to do an early scan of
another table just to find that number.

Cheers,
Rafael

6

It is technically possible. But we have to find the total symbols
early (symbol resolution) and so we would need to do an early scan of
another table just to find that number.

That shouldn't be too expensive. I would expect roughly as many
dynamic relocations as there are symbols as they will mostly just be
patching PLT/GOT entries for them.

You could also make it easier by creating a new tag defining the size
of the symbol table. That would at least allow shared libraries
produced by lld to be able to easily parse the dynamic symbol table.

Is statically linking against shared libraries that have been stripped
of their section header tables something that is likely to be
requested?

7

It is technically possible. But we have to find the total symbols
early (symbol resolution) and so we would need to do an early scan of
another table just to find that number.

That shouldn't be too expensive. I would expect roughly as many
dynamic relocations as there are symbols as they will mostly just be
patching PLT/GOT entries for them.

The dynamic relocation will cover every symbol. For example, a .so with just

void f(void) {}

in it has no relocations at all.

The hash table will cover them all. But it doesn't seem possible to easily find
the size of the gnu hash table.

You could also make it easier by creating a new tag defining the size
of the symbol table. That would at least allow shared libraries
produced by lld to be able to easily parse the dynamic symbol table.

Is statically linking against shared libraries that have been stripped
of their section header tables something that is likely to be
requested?

No, every other linker that I know of uses sections.

Cheers,
Rafael

8

I agree we should use sections. Also, there's no reason we diverge
from what other linker do unless there's a real reason.