Linking the FreeBSD base system with lld

Project Infrastructure LLVM Dev List Archives
1

I've been occasionally trying to build the FreeBSD base system (on
amd64 / x86_64) using lld in order to see how things are progressing.
With a few hacks in the FreeBSD build I can now link the base system
userland.

The missing functionality I had to work around is listed below. Some
of these should probably be addressed in our FreeBSD build
infrastructure, while others should be handled in lld. I'm listing
this here only as a snapshot of where lld stands today for our use.

1. -r / --relocatable (PR 23024)
There are four or five uses of -r in the FreeBSD base system, and at
least some of them cannot be handled by just using ar instead.

2. -E / --export-dynamic
This is used by our GDB build.

3. -Y [path]
Used by the 32-bit compat libraries. GNU ld documents this as "add
path to the default library search path." I don't understand exactly
what this option does, versus -L.

4. unrecognized reloc 22 (R_X86_64_GOTTPOFF)
Encountered in one of the libc tests.

5. --no-undefined vs _GLOBAL_OFFSET_TABLE_
A number of libraries are linked with --no-undefined. These fail wtih
an error of the form:
undefined symbol: _GLOBAL_OFFSET_TABLE_ in /<path>tmp/usr/lib/crtbeginS.o

6. emulation type "elf_i386_fbsd"
This is used by the 32-bit library build, which fails with "Unknown
emulation: elf_i386_fbsd"

7. static linking fails because _end is undefined (PR 25528)

8. Library search paths
The FreeBSD build infrastructure builds the toolchain twice: once for
the build host to build the target FreeBSD libraries and binaries, and
once for installation as the toolchain installed in the target. The
host toolchain is built with a set of baked-in library search paths
(.../tmp/lib and .../tmp/usr/lib).

I have a WIP patch to always apply --sysroot during the FreeBSD build,
but with this added it seems we do not get $SYSROOT/lib and
$SYSROOT/usr/lib as default search paths in lld. We also lack a
convenient way to debug this today in lld; with GNU ld the --verbose
option dumps a built-in linker script which includes the paths in use,
but lld does not have something equivalent.

Also, the kernel link fails because FreeBSD's kernel linker script is
too complicated for the current parser.

For reference, here are the workarounds I applied to build FreeBSD
userland with lld:

* Remove -Wl,--no-undefined from those library makefiles where it is set
* Disconnect lib/libpam/statc_modules from the build
* Disable the addition of -static in bsd.lib.mk
* Explicitly add the temporary build paths as -L options in various LDFLAGS
* Set the following options in src.conf:
WITHOUT_BOOT=yes
WITHOUT_RESCUE=yes
WITHOUT_LIB32=yes
WITHOUT_GDB=yes
WITHOUT_TESTS=yes
MODULES_OVERRIDE=

2

I've been occasionally trying to build the FreeBSD base system (on
amd64 / x86_64) using lld in order to see how things are progressing.
With a few hacks in the FreeBSD build I can now link the base system
userland.

The missing functionality I had to work around is listed below. Some
of these should probably be addressed in our FreeBSD build
infrastructure, while others should be handled in lld. I'm listing
this here only as a snapshot of where lld stands today for our use.

1. -r / --relocatable (PR 23024)
There are four or five uses of -r in the FreeBSD base system, and at
least some of them cannot be handled by just using ar instead.

2. -E / --export-dynamic
This is used by our GDB build.

3. -Y [path]
Used by the 32-bit compat libraries. GNU ld documents this as "add
path to the default library search path." I don't understand exactly
what this option does, versus -L.

4. unrecognized reloc 22 (R_X86_64_GOTTPOFF)
Encountered in one of the libc tests.

5. --no-undefined vs _GLOBAL_OFFSET_TABLE_
A number of libraries are linked with --no-undefined. These fail wtih
an error of the form:
undefined symbol: _GLOBAL_OFFSET_TABLE_ in /<path>tmp/usr/lib/crtbeginS.o

6. emulation type "elf_i386_fbsd"
This is used by the 32-bit library build, which fails with "Unknown
emulation: elf_i386_fbsd"

gold doesn't recognize that emulation name. Maybe you want to use elf_i386
instead?

7. static linking fails because _end is undefined (PR 25528)

8. Library search paths
The FreeBSD build infrastructure builds the toolchain twice: once for
the build host to build the target FreeBSD libraries and binaries, and
once for installation as the toolchain installed in the target. The
host toolchain is built with a set of baked-in library search paths
(.../tmp/lib and .../tmp/usr/lib).

I have a WIP patch to always apply --sysroot during the FreeBSD build,
but with this added it seems we do not get $SYSROOT/lib and
$SYSROOT/usr/lib as default search paths in lld. We also lack a
convenient way to debug this today in lld; with GNU ld the --verbose
option dumps a built-in linker script which includes the paths in use,
but lld does not have something equivalent.

Feel free to add that functionality. We have --verbose flag already, but it
prints out little information.

Also, the kernel link fails because FreeBSD's kernel linker script is

3

Option order, it comes after all -L path. For NetBSD, we have slightly
different linker scripts for i386-on-amd64, so we don't use -Y.

Joerg

4

GNU ld manual says that -Y is for Solaris compatibility. I'm sort of
reluctant to support that since it doesn't seems to be popular.

5

I'm not saying it is terribly useful, just that it differs from -L. I'm
not sure how the 32bit compat libraries are build on FreeBSD. For
NetBSD, we change the default SEARCH_DIR entries in the linker scripts.
For native binary format, it uses =/usr/lib, but for the -m32 equivalent
on amd64, it uses =/usr/lib/i386. That means all the search logic is
isolated in the linker script and no further command line changes are
necessary.

Joerg