[LLD] thunk implementation correctness depends on order of input section.

I've been working on supporting ARM/Thumb interworking thunks in LLD
and have encountered a limitation that I think it is worth bringing up
in a wider context. This is all LLD specific, apologies if I've abused
llvm-dev here.

TL;DR summary:
- Thunks in lld may not work if they are added to InputSections that
have already been scanned.
- There is a short term fix, but in the longer term I think that we
will want to allow multiple passes of the relocations.
- I'd like to know if there are any preferences on a solution.

The current thunk implementation scans for and adds Thunks to
InputSections within scanRelocations(). At present a Thunk for a
RegularSymbol is always added to the InputSection that defines the
RegularSymbol. I think that this can cause a problem when the
InputSection with the relocation that needs to be indirected via the
thunk is processed after the InputSection that the thunk is added to.

For reference the important code snippet in Writer.cpp is:

  // Scan relocations. This must be done after every symbol is declared so that
  // we can correctly decide if a dynamic relocation is needed.
  for (OutputSectionBase<ELFT> *Sec : OutputSections) {
    Sec->forEachInputSection([&](InputSectionBase<ELFT> *S) {
      if (auto *IS = dyn_cast<InputSection<ELFT>>(S)) {
        // Set OutSecOff so that scanRelocations can use it.
        uintX_t Off = alignTo(Sec->getSize(), S->Alignment);
        IS->OutSecOff = Off;

        scanRelocations(*IS);

        // Now that scan relocs possibly changed the size, update the offset.
        Sec->setSize(Off + S->getSize());
      } else if (auto *EH = dyn_cast<EhInputSection<ELFT>>(S)) {
        if (EH->RelocSection)
          scanRelocations(*EH, *EH->RelocSection);
      }
    });
  }

Adding a thunk to an InputSection that has already been processed will
mean that its size will not be updated by Sec->setSize(Off +
S->getSize()); this leads to corrupt or missing thunks being generated
as they are written to invalid buffer location.

To reproduce the problem I've taken the existing
test/ELF/mips-npic-call-pic.s and reversed the order of the input
objects so that the relocations needing thunks are processed last.
I've attached the patch I've made to the test case and the ouputs of
llvm-objdump for convenience. In the mips-thunk-correct.txt output all
thunks are present, and of the correct size. In the
mips-thunk-corrupt.txt not all thunks are present and some appear to
be corrupt.

Given that scanRelocations depends on IS->OutSecOff it isn't as simple
as just finding the InputSection and correcting its size as this will
affect the value of IS->OutSecOff of all sections that follow it,
potentially invalidating any decision made on the basis of
IS->OutSecOff.

The simplest short-term fix is to add the thunks to the InputSection
that contains the relocation needing the Thunk (The current IS). In
the longer term, with range-extension thunks in mind, I think it would
be wise to allow multiple passes through the relocations. For example
one possibility for the example above is that adding a Thunk to a
section already processed triggers another pass of the loop, updating
anything created that relies upon IS->OutSecOff.

My current thought is that to get a simple implementation of ARM/Thumb
interworking thunks working, including thunks to PLT entries [*], I
will need to add support for a Thunk to be added to a different
InputSection than the target SymbolBody anyway. So I can add all
Thunks to the current InputSection fix as part of the ARM/Thumb
interworking. Support for range extension thunks are a way down the
priority list at the moment so I don't propose to make any radical
changes to scanRelocations at this point, however I'm willing to do so
if others would prefer?

[*] The interworking can be inlined into the ARM PLT entries, at the
cost of requiring multiple entry points for every ARM PLT entry. My
preference is to implement support for thunks to PLT entries.

Kind regards

Peter

mips-thunk-corrupt.txt (1.07 KB)

mips-thunk-correct.txt (1.58 KB)

mips-npic-call-pic.diff (574 Bytes)

The first think I’d want to say is to not try too hard to use the existing “framework” in LLD. The current thunk mechanism was made just for MIPS non-PIC/PIC function calls and are not proven to be generic or useful for other purposes. We are cool with that because it just works for MIPS (except the bug you found) and satisfies our needs. And the amount of code for MIPS thunk is so small that we are not serious about reusing it. Probably the name given to it (“thunk”) is too generic – maybe we should rename it MipsLA25Thunk or something like that. I think you want to create a new type of thunk for ARM.

The bug that we sometimes create broken MIPS thunks seems to have introduced in r265673 which Rafael made. Before that patch, we didn’t assume that section VAs are available in scanRelocs. I think we want to revert that change (although it cannot simply be reverted because the patch was submitted in April, and many changes has been made on it since then.)

Rafael, can you take at that change?

First of all thanks for finding the bug.

I agree with Rui that right now we can manage without general thunk
infrastructure. Let's provide at least a few "thunk" implementation
and after that reconsider necessity of common thunk framework. As to
MIPS there is one more type of thunk (keyword is .MIPS.stubs) and one
more optimization of current thunk (putting a thunk in the beginning
of the section if the section contains the only function required the
thunk). But I do not plan to implement these features in the near
future.

Thank you both for the feedback.

I think that there will be a good deal of commonality between
architectures that need thunks. I agree that it will be easier to
generalise later when we need to.

ARM also has an optimisation of a thunk at the start of the section.
In effect change state and fall through to a function at the start of
the section. It is also quite a bit down the priority list.

Peter

Not exactly fall through :slight_smile: It has to form the address of the other entry point (+1 for Thumb) in a temp register and then BX to it. It just means the constant added to the PC is very small…