[RFC] Compressed SHT_SYMTAB/SHT_STRTAB for ELF

[MaskRay]

In ELF executables and DSOs, .symtab and .strtab sections can consume a significant portion of the file size (10+% or even 20+%). In many scenarios, we cannot remove them due to symbolizers (crash reporters, Linux perf, etc) and other analysis tools. However, it’s natural to compress them using the standard SHF_COMPRESSED feature. We need linker and binary utilities support.

• ld.lld: I have a generalized patch for non-SHF_ALLOC sections in July 2023 and have now posted it at [ELF] Add --compress-section to compress matched non-SHF_ALLOC sections by MaskRay · Pull Request #84855 · llvm/llvm-project · GitHub
• llvm-objcopy: --compress-nonalloc-sections <section-glob>=[none|zlib|zstd]. none can decompress an already compressed section. (--compress-nonalloc-sections ... zstd may choose to recompress a zlib-compressed section using zstd?)
• llvm-readobj: -s
• llvm-nm

With the latest lld, we can compress .strtab, but the non-NUL terminated .strtab is not accepted by llvm-readelf

% myclang -fuse-ld=lld a.c -Wl,--compress-sections=.strtab=zstd -o a
% llvm-readelf -s a
...
warning: 'a': unable to get the string table for the SHT_SYMTAB section: SHT_STRTAB string table section [index 29] is non-null terminated

Symbol table '.symtab' contains 36 entries:
   Num:    Value          Size Type    Bind   Vis       Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT   UND <?>
     1: 00000000000002c4    32 OBJECT  LOCAL  DEFAULT     2 <?>

[pogo59]

Does this include .debug_* sections?

[MaskRay]

Some use cases include .debug_* (at least -gmlt, possibly also -fdebug-info-for-profiling) while some don’t.

[pogo59]

So llvm-dwarfdump would also need to learn about this compression? I know there is some provision already for compressed .debug sections, if it all Just Works with both zlib/zstd then that’s great.