[LLD] Should --compress_debug_sections be enabled (=zlib) by default ?

Hi,

In the file lld/ELF/Driver.cpp in function getCompressDebugSections we can see that the current default for lld is no debug section compression. It looks like tools like gdb, valgrind, elfutils, gcc's backtrace lib currently support compressed symbols. Since perf can use libdw from elfutils, I guess it supports it too.

Do you think it's time to enable compressed debug section by default ? On some of my big binaries (several hundreds of MB) this could save 50% of the overall binary size.

Cheers,
Romain

From: llvm-dev [mailto:llvm-dev-bounces@lists.llvm.org] On Behalf Of
Romain GEISSLER via llvm-dev
Sent: Friday, May 03, 2019 12:53 PM
To: llvm-dev@lists.llvm.org
Subject: [llvm-dev] [LLD] Should --compress_debug_sections be enabled
(=zlib) by default ?

Hi,

In the file lld/ELF/Driver.cpp in function getCompressDebugSections we can
see that the current default for lld is no debug section compression. It
looks like tools like gdb, valgrind, elfutils, gcc's backtrace lib
currently support compressed symbols. Since perf can use libdw from
elfutils, I guess it supports it too.

Do you think it's time to enable compressed debug section by default ? On
some of my big binaries (several hundreds of MB) this could save 50% of
the overall binary size.

I think we'd want a way to make this customizable for different platforms.
--paulr

We can compress debug sections in compiler-generated object files as well as linker-generated executables. So the question whether we should compress debug sections is not only relevant to lld but to clang, and I think it’s actually a whole toolchain-level decision.

If we decide to compress debug sections for some target, we should make a change to the compiler driver to flip the default for a compiler flag to turn on compression. lld is invoked by the compiler driver, so a change for the linker would be added to the compiler driver too.

I think I’m mildly in favor of making such change – I guess the cost of compress doesn’t too high, and that might even make things faster by reducing the amount of IO. I’d guess people wouldn’t even care whether debug sections are compressed or not, as their tools already support it. But I don’t have an evidence that the latter is actually the case.

Hi,

We can compress debug sections in compiler-generated object files as well as linker-generated executables. So the question whether we should compress debug sections is not only relevant to lld but to clang, and I think it’s actually a whole toolchain-level decision.

If we assume the compiler is clang. However in my case it would be gcc (I agree this is not a very ordinary mix of compiler/linker). When you use the full GNU toolchain, this choice of default value is done purely in binutils (you can configure it with the flag --enable-compressed-debug-sections={all,gas,gold,ld,none}).

Today as far as I know in binutils, to reduce IO, it’s enabled by default in “gas” but not in “ld.bfd” (and by the way, it’s not even implemented in “ld.gold”, I have to submit a patch for that). This may be true for some target only, I am considering Linux x64. The fear in enabling it in “ld.bfd” is that it might break tools. For the tool I use (gdb, valgrind, (gcc) sanitizers (with gcc’s libbacktrace for the stack printing)) it works. There might be other tools broken by this change. So yes it make sense to make the change at the whole toolchain level, in both the assembler + the linker. I don’t know if clang uses it’s own LLVM assembler or GNU gas.

I think I’m mildly in favor of making such change – I guess the cost of compress doesn’t too high, and that might even make things faster by reducing the amount of IO. I’d guess people wouldn’t even care whether debug sections are compressed or not, as their tools already support it. But I don’t have an evidence that the latter is actually the case.

Ok. I will just patch lld myself then, only for my own toolchain.

Thanks,
Romain

It depends whether you use index (Apple index or .gdb_index or .debug_names).
If not then compressed debug is probably always a win.
.gdb_index is not going to be supported by LLDB (it is supported by GDB),
.debug_names (at LLD level) is going to be supported by LLDB in some time.

If you do use index then loading such file will be slower as this type of
compression cannot decompress only small section of the file addressed by the
index. Then some filesystem level (or block device level) compressed can be
a better compromise between disk size and debug info access performance.

But it sure needs more numbers.

Jan