Should we switch to --hash-style=both by default in LLD ?

Hi !

During linking LLD and other linkers builds a static hash table for dynamic symbols,
so that in runtime dynamic linker can use this table and find symbols faster.

--hash-style=style linker option is used to select the hash type:
"Set the type of linker's hash table(s). style can be either "sysv" for classic ELF ".hash" section,
"gnu" for new style GNU ".gnu.hash" section or "both" for both the classic ELF ".hash" and new
style GNU ".gnu.hash" hash tables. The default is "sysv"."

GNU_HASH format is a better version of hash table (https://sourceware.org/ml/binutils/2006-10/msg00377.html).

We have --hash-style=sysv currently set by default in LLD.
Though recently (see https://bugs.llvm.org//show_bug.cgi?id=34712) GNU linkers switched to use
".gnu.hash" section format in addition to normally used classic ".hash".
So they defaults --hash-style to "both" and this looks to be released with binutils 2.30.

I think we can switch LLD either to "both" or probably to "gnu" by default as well.
Initial version of patch that changes default to "both" is here: D38407

Any thoughts ?

Best regards,
George | Developer | Access Softek, Inc

Hi,

I initially wanted to switch the default to "gnu" as well in GNU linkers. However it was argued that this would generate ELF fiels that do not follow the ELF ABI. See here https://sourceware.org/ml/binutils/2017-07/msg00160.html Thus it has been chosen to use "both" rather than "gnu".

Cheers,
Romain

Fair enough, argument that we should not emit ELF files that aren't ABI compliant by default looks reasonable.
Use of "both" looks ideal for me then.

George.

+1 for both table formats. Then we’re compatible by default no matter what. If somebody cares about .o file size, they can explicitly specify their --hash-style.

Changing default to “both” is probably fine, but what does that change actually mean in practice? I believe clang already passes --hash-style=both to the linker by default.

Not exactly.

The current Linux driver (I haven’t checked other ELF-based systems) reads this (https://github.com/llvm-mirror/clang/blob/master/lib/Driver/ToolChains/Linux.cpp#L228)

// Do not use ‘gnu’ hash style for Mips targets because .gnu.hash
// and the MIPS ABI require .dynsym to be sorted in different ways.
// .gnu.hash needs symbols to be grouped by hash code whereas the MIPS
// ABI requires a mapping between the GOT and the symbol table.
// Android loader does not support .gnu.hash.
// Hexagon linker/loader does not support .gnu.hash
if (!IsMips && !IsAndroid && !IsHexagon) {
if (Distro.IsRedhat() || Distro.IsOpenSUSE() ||
(Distro.IsUbuntu() && Distro >= Distro::UbuntuMaverick))
ExtraOpts.push_back("–hash-style=gnu");

if (Distro.IsDebian() || Distro.IsOpenSUSE() || Distro == Distro::UbuntuLucid ||
Distro == Distro::UbuntuJaunty || Distro == Distro::UbuntuKarmic)
ExtraOpts.push_back("–hash-style=both");
}

Given that several libc implementation supports this in the Linux ecosystem (glibc, musl, uClibc, most likely others) I think lld should provide good defaults by default, instead of relying on what some distro maintainer have hardcoded for their own distro in clang.

Cheers,
ROmain

I read through the binutils mailing list thread, but I couldn’t find the exact reason why making --hash-style=gnu default except MIPS wasn’t a good idea.Do you mind if I ask you to explain it again for me?

Since lld is a new linker, we could make a bit more radical change than GNU ld can do, so I wonder if --hash-style=both is the right choice for us.

To be honest, I have no idea. I am much more on the user side of ld.bfd/ld.gold than the developer side. I know very little about the exact specific ELF ABI Michael Matz is referring to. I actually would expect you lld developers to have quite an good knowledge of it :wink:

On my side, I do configure and/or patch all my linkers (bfd, gold and lld) to use the "gnu" hash-style by default, and I can only say that with a classical glibc environment, it works. The fact that some Linux distro are doing that too seems to be a good arguments that it is safe, at least when using glibc. The question is more whether lld seeks (size) optimization over correctness (wrt the ELF specification) ?

Cheers,
Romain

>
> I read through the binutils mailing list thread, but I couldn't find the
exact reason why making --hash-style=gnu default except MIPS wasn't a good
idea.Do you mind if I ask you to explain it again for me?
>
> Since lld is a new linker, we could make a bit more radical change than
GNU ld can do, so I wonder if --hash-style=both is the right choice for us.

To be honest, I have no idea. I am much more on the user side of
ld.bfd/ld.gold than the developer side. I know very little about the exact
specific ELF ABI Michael Matz is referring to. I actually would expect you
lld developers to have quite an good knowledge of it :wink:

There are old systems that do not support .gnu.hash sections, but among the
systems lld support, I believe we can enable --hash-style=gnu except MIPS.

On my side, I do configure and/or patch all my linkers (bfd, gold and lld)
to use the "gnu" hash-style by default, and I can only say that with a
classical glibc environment, it works. The fact that some Linux distro are
doing that too seems to be a good arguments that it is safe, at least when
using glibc. The question is more whether lld seeks (size) optimization
over correctness (wrt the ELF specification) ?

I believe .hash section is optional, so an executable only with .gnu.hash
is still valid in terms of the ELF specification even on a system that
doesn't recognize .gnu.hash.

So, maybe making --hash-style=gnu a default option except MIPS is a
reasonable choice for us? Any objection?

>
> > Le 3 oct. 2017 à 00:09, Rui Ueyama <ruiu@google.com> a écrit :
> >
> > I read through the binutils mailing list thread, but I couldn't find the
> exact reason why making --hash-style=gnu default except MIPS wasn't a good
> idea.Do you mind if I ask you to explain it again for me?
> >
> > Since lld is a new linker, we could make a bit more radical change than
> GNU ld can do, so I wonder if --hash-style=both is the right choice for us.
>
> To be honest, I have no idea. I am much more on the user side of
> ld.bfd/ld.gold than the developer side. I know very little about the exact
> specific ELF ABI Michael Matz is referring to. I actually would expect you
> lld developers to have quite an good knowledge of it :wink:
>

There are old systems that do not support .gnu.hash sections, but among the
systems lld support, I believe we can enable --hash-style=gnu except MIPS.

We don't support it in NetBSD and I'm still not very impressed by
.gnu.hash...

> On my side, I do configure and/or patch all my linkers (bfd, gold and lld)
> to use the "gnu" hash-style by default, and I can only say that with a
> classical glibc environment, it works. The fact that some Linux distro are
> doing that too seems to be a good arguments that it is safe, at least when
> using glibc. The question is more whether lld seeks (size) optimization
> over correctness (wrt the ELF specification) ?
>

I believe .hash section is optional, so an executable only with .gnu.hash
is still valid in terms of the ELF specification even on a system that
doesn't recognize .gnu.hash.

I'm moderately sure that many dynamic linkers will reject at least
shared libraries without a supported hash.

Joerg

I don't think there is much value in being different from the gnu
linkers in here since the common case is to have the compiler driver
pass an explicit option.

So +1 to changing the default to "both".

Cheers,
Rafael

George Rimar <grimar@accesssoftek.com> writes:

I don't think there is much value in being different from the gnu
linkers in here since the common case is to have the compiler driver
pass an explicit option.

So +1 to changing the default to "both".

... except for MIPS, as the way how we need to sort .gnu.hash section
contents is not compatible with a MIPS ABI requirement.

Cheers,

To wrap up, looks like this is what we should do:

  • Use -hash-style=both as a default except MIPS.
  • For MIPS, use -hash-style=sysv as a default.

We’ll make a change accordingly.