[RFC] Use SHT_DWARF64 for lld DWARF32/DWARF64 debug section ordering

Hi all,

While working on our DWARF64 investigation, we ran into the same DWARF32/DWARF64 mixed-input ordering problem discussed in this earlier thread:

I would like to continue that discussion with a concrete proposal based on SHT_DWARF64.

Proposal

This RFC proposes using a new ELF section type, SHT_DWARF64, as an explicit producer/linker contract.

The proposed approach is:

  1. LLVM producers emit DWARF64 .debug_* sections with sh_type = SHT_DWARF64.
  2. LLVM ELF tools and MC recognize and round-trip SHT_DWARF64.
  3. lld adds an opt-in option, currently drafted as --dwarf32-before-dwarf64, to place DWARF32 debug input sections before DWARF64 ones.

The main advantage is that lld does not need to infer DWARF32 vs DWARF64 by parsing debug contents, inspecting relocations, or relying on build-system ordering. With SHT_DWARF64, lld can simply stable-partition .debug_* input sections: non-SHT_DWARF64 sections first, then SHT_DWARF64 sections, preserving relative order within each group.

Draft: [DebugInfo][LLD] Use SHT_DWARF64 for lld DWARF32/DWARF64 debug section ordering by Sockke · Pull Request #206690 · llvm/llvm-project · GitHub

Open issues for discussion

The main open question is the ABI/tooling impact of introducing a new ELF section type.

SHT_DWARF64 gives lld an explicit and cheap signal, but it is still an ELF ABI extension. I would like feedback from ELF/generic-abi/tooling maintainers on whether this direction is acceptable, what value should be used, and what compatibility impact this may have on existing ELF tools that currently expect .debug_* sections to be SHT_PROGBITS.

The current draft treats SHT_DWARF64 as producer-provided metadata and does not add fallback DWARF-content scanning in lld.

Mixed DWARF32/DWARF64 within one input section

One corner case is that a single input debug section may itself contain both DWARF32 and DWARF64 units. Section-level ordering cannot handle that case because the section is no longer an indivisible sorting unit.

A possible design is to split such sections by DWARF unit contribution, then apply the same DWARF32-before-DWARF64 ordering to the resulting pieces. This requires parsing unit boundaries and updating the affected relocations/section references after reordering. It may also need special handling for related sections whose entries are referenced from `.debug_info` rather than directly classifiable from local section contents.

The current draft keeps section type as the sorting granularity. I would like feedback on whether this mixed-within-one-section corner case is important enough to support in the initial implementation, or should be handled as a follow-up.

@ayermolo @dblaikie @MaskRay @igorkudrin @jh7370

I wonder if there’s any useful generalization of this SHF_ kind beyond DWARF - I think @aeubanks might’ve had similar issues with sorting code using the large code model?

Also might be worth cross posting in: https://groups.google.com/g/generic-abi

2 Likes

To me, this feels like it should be a section flag rather than a section type, mostly because then you don’t end up in the land of the linker needing to mutate the SHT_DWARF64 type into a SHT_PROGBITS type at link time, when it merges with a 32-bit DWARF section in order to end up with a single DWARF section in the final binary.

If we were to have a section type, unless there was acceptance in the gABI for it, we’d want a SHT_LLVM_* type and I reckon we’d want it for both DWARF32 and DWARF64, or possibly generically DWARF (to allow for “unknown/mixed” values) and then DWARF64.

My preference for a flag is that it is more clearly generalisable, i.e. you could have two flags, one for ordering things first and the other for ordering things last. To my knowledge, there is no current gABI flag for either of these, but I do seem to recall there being other platform-specific flags that do this (I could easily be hallucinating this though). The linker behaviour would be to group sections as normal, so 32-bit and 64-bit DWARF sections would end up in the same output section, but with SHF_FIRST (or whatever) sections being ordered before other sections and SHF_LAST (or whatever) being ordered last. Sections without the specification would then be placed in between the sections with those flags.

1 Like

Also might be worth cross posting in: https://groups.google.com/g/generic-abi

There is an old thread from 2020: https://groups.google.com/g/generic-abi/c/i2Xio-47QdQ/m/Qq8-76noAwAJ (“Reserve a section type value for DWARF64”)

IIRC Ali and Cary didn’t like a new section type for DWARF64.

3) A new section flag. I agree with Ali on this point. This idea has a larger blast radius.
(We have 0x60000000 section types while only 8 remaining generic section flags.)
I don't see the existing SHT_*_LARGE and SHT_*_SMALL convincing because they actually use different output section names.
(Please correct me as I also think they are some legacy stuff)
Should the output section have the flag if any of the input sections have the flag?
For many other flags this rule applies but for SHF_*_LARGE the flag can cause a problem
with relocatable links (as I mentioned above).

4) A section prefix. To be fair, as a linker person I like it in two ways:

* It is immediately obvious whether DWARF64 is used and whether
DWARF32 is used along with DWARF64.
* In a relocatable link mixing DWARF32 and DWARF64 sections, DWARF32
and DWARF64 sections will naturally not get mixed. We don't even need another linker feature
to match input sections by type.

On the other hand,

* It is non-conforming due to different section names.
James Henderson
(https://lists.llvm.org/pipermail/llvm-dev/2020-November/146721.html): "However,
conformance is still a concern to me as we cannot really retrofit the existing
standard versions, and the section names themselves are in the standard. That
means that tools that otherwise would work might stop working when presented
with a "new" DWARFv3/4/5 output that it in theory could otherwise handle."

This is just one problematic section, we also have .eh_frame which has the same issue of precompiled small code model libraries using PC32 relocations and large code model libraries using PC64 relocations. Given that .eh_frame and .debug_info sections are both placed on the same side of text and having this type of intra-section sort won’t solve both .eh_frame and .debug_info, IMO we should have a solution that works for multiple arbitrary sections that have both a small precompiled 32-bit reference portion and 64-bit reference portion.

Previously I was hoping to ignore the problem of .eh_frame by assuming it was small enough to not contribute too much to relocation pressure, but with this thread I don’t think we can keep hoping that.

The obvious answer is the different section name with a “l” prefix, just like the x86-64 medium/large code model’s .ldata. But as mentioned, that doesn’t conform to the existing spec. Perhaps we can update the DWARF spec? .zdebug_info for compressed debug info is already special cased. For .eh_frame we could emit a separate .leh_frame, and like PT_GNU_RELRO, we could add support for multiple PT_GNU_EH_FRAME segments. This requires updating a lot of the ecosystem, but otherwise I don’t see a nice way of supporting multiple sections that have this type of problem.

1 Like

We did move away from zdebug style naming, instead with a section flag. But yeah, fine if a couple of objects were ld -r’d together (two .debug_* sections, one with the flag and one without) but less common to have multiple sections with the same name in a linked executable (less common? But is it unheard of?).

& is ordering insufficient? (linking small sections first, then large sections after)