Compact section header table for ELF

LLVM Project
,
1

In ELF files, the section header table is an array of 64-byte Elf64_Shdr structures (or 40-byte Elf32_Shdr for 32-bit). Each structure describes name, type, flags, address, offset, size, link, info, alignment, and entry size.

When building llvm-project with -O3 -ffunction-sections -fdata-sections -Wa,--crel,--allow-experimental-crel , the section header tables occupy 17.6% of the total .o file size. In a -g -Wa,--crel,--allow-experimental-crel build, the section header tables occupy 13.9% of the total .o file size.

Last year I developed this format along with CREL and mentioned it in a few places. Recently refined it with CLEB128 and posted at https://groups.google.com/g/generic-abi/c/9DPPniRXFa8/m/32BVkrLrAgAJ , which received a lot of support.

Prototype https://github.com/MaskRay/llvm-project/tree/demo-cshdr

  • Core functionality requires <200 lines, making clang, lld, llvm-readelf, llvm-ar, llvm-objdump work with compact section header table.
  • yaml2obj and tests need a lot of lines :slight_smile:
  • CLEB128 (counted LEB128) llvm/lib/Support/LEB128.cpp . https://groups.google.com/g/generic-abi/c/9DPPniRXFa8/m/MJ3jetzZAAAJ suggested that we adopt a canonical name for this variable-length integer encoding. CLEB128 (counted LEB128) looks fantastic to me.

2024 prototype using unsigned LEB128 https://github.com/MaskRay/llvm-project/tree/demo-cshdr-2024

Detailed write-up: https://maskray.me/blog/2024-03-31-a-compact-section-header-table-for-elf

Note: The unsigned prefix-based variable-length integer encoding is identical to the one in MLIR bytecode, rather than the more common Little Endian Base 128 (LEB128) encoding used in DWARF and WebAssembly. This prototype doesn’t add variable-length signed integers. If we ever introduce them, we should use sign extension (MSB as sign) instead of zig-zag encoding (LSB as sign). MLIR, as sign extension actually generates faster code on most architectures.

Combining crel and cshdr, the object file is usually smaller than the WebAssembly binary format:

clang -c -Wa,--crel,--allow-experimental-crel,--cshdr -fno-asynchronous-unwind-tables -fno-exceptions -fno-ident -fno-addrsig -fno-unique-section-names a.cc
clang -c --target=wasm64 a.cc
2 Likes
LLVM Weekly - #619, November 10th 2025
2

Generally in favor, thank you for the detailed proposal and analysis.

I do have more thoughts about subsections-via-symbols, though. From your write-up:

Mach-O’s subsection feature imposes restrictions on label differences.
Coalescing symbols or adjusting values in sections would change subsection boundaries, leading to complexity in binary manipulation tools.
Additionally, there is a loss of flexibility as subsection properties (type, flags, linked-to section) cannot be changed at least.

I personally still consider this to be a more elegant solution, where we’d get -ffunction-sections at ~no section header cost, as we already have most information in the symbol table. I imagine we could store the subsections-via-symbols flag as a section flag. Linkers and other tools should treat such sections as being really multiple sections (one per symbol) (e.g., while reading the object, immediately produce multiple input section structures). Solutions would be needed for linked-to (new relocation type?) and comdat (maybe just use linker GC here?). The biggest argument against this is obviously the implementation complexity and the non-trivial change of linking semantics, which, I guess, would make this a non-starter…

3

For CLEB128, you mention the advantage of avoiding non-canonical LEB128 encodings. LLVM uses those non-canonical encodings to avoid shrinking section sizes in some cases, which could otherwise lead to size oscillations (e.g. ⚙ D42722 [MC] Fix assembler infinite loop on EH table using LEB padding). That shouldn’t be a problem for section headers, but how would CLEB128 handle that in general?

4

We sort of already considered and rejected bringing atomization to the ELF world when we abandoned the old LLD codebase in favor of the COFF rewrite (initial port). Atomization was sort of at the core of the original design philosophy, but in a section-oriented ecosystem, it mostly acts as overhead.

As you say, when you want to do fancy stuff (linked-to), you have to start encoding additional information into the file, and the linker has to learn how to interpret it. I wasn’t able to find a reference, but I seem to recall that the Darwin linker has to do some funny stuff to discover the association between globals and ASan global metadata in __asan_globals, but I could not find a reference. Given that there is already a very general pre-existing way to express all of this stuff, it makes sense to lean into it and make the general functionality more efficient.

1 Like
5

FWIW, LLD for Mach-O necessarily has to work with atoms. It’s generally self-contained; you split up the sections in the input files when reading them in, and operate on those split-up sections in the rest of the linker. Mach-O doesn’t have COMDATs IIRC; linked-to works via S_ATTR_LIVE_SUPPORT, which is a section attribute, so I guess you can’t get too granular?. CC @bd1976bris, who’s previously expressed interest in .subsections_via_symbols for Mach-O.

Overall I think .subsections_via_symbols is a clever solution for Mach-O being limited to 255 sections, but for ELF it seems cleaner to stick to explicit sections but make their representation more efficient. In particular, Mach-O ends up needing .alt_entry to prevent atomization in certain cases, which is a bit inelegant.

1 Like