workaround to force LLD to make dwarf info sections mappable/loadable?

Context: https://bugs.llvm.org/show_bug.cgi?id=39862

I'm trying to enable stack traces in an OS kernel. To do that the kernel
needs access to its own debug info. I have this in my linker script:

    .debug_info : {
        __debug_info_start = .;
        KEEP(*(.debug_info))
        __debug_info_end = .;
    }
    .debug_abbrev : {
        __debug_abbrev_start = .;
        KEEP(*(.debug_abbrev))
        __debug_abbrev_end = .;
    }
    .debug_str : {
        __debug_str_start = .;
        KEEP(*(.debug_str))
        __debug_str_end = .;
    }
    .debug_line : {
        __debug_line_start = .;
        KEEP(*(.debug_line))
        __debug_line_end = .;
    }
    .debug_ranges : {
        __debug_ranges_start = .;
        KEEP(*(.debug_ranges))
        __debug_ranges_end = .;
    }

But LLD maps these sections to address 0:

00000000000003ac g .debug_abbrev 0000000000000000 __debug_abbrev_end
0000000000000000 g .debug_abbrev 0000000000000000
__debug_abbrev_start
0000000000009da5 g .debug_info 0000000000000000 __debug_info_end
0000000000000000 g .debug_info 0000000000000000 __debug_info_start
0000000000006aba g .debug_line 0000000000000000 __debug_line_end
0000000000000000 g .debug_line 0000000000000000 __debug_line_start
0000000000001bf0 g .debug_ranges 0000000000000000 __debug_ranges_end
0000000000000000 g .debug_ranges 0000000000000000
__debug_ranges_start
000000000000399f g .debug_str 0000000000000000 __debug_str_end
0000000000000000 g .debug_str 0000000000000000 __debug_str_start

is there a workaround to force LLD to make these sections
mappable/loadable so that the executable can access them?

Regards,
Andrew

Unwind info is usually emitted into an allocatable section, so you can compute a stack trace without looking at debug info. Of course, that doesn't include any symbol information, but you can compute the corresponding symbols offline.

IIRC the Linux kernel uses a custom format for symbol information; the build system post-processes the object files to compute the relevant information, and uses objcopy to insert the information into an allocatable section.

-Eli

I think Eli made a good point. In addition to that, if you are writing a OS kernel, I guess you are also writing a loader, so you can map anything as you want, no?

I think Eli made a good point.

Eli's point seems to be "your use case is not valid", which I will
consider, but it doesn't really work towards solving the problem as stated.

In addition to that, if you are writing a
OS kernel, I guess you are also writing a loader, so you can map
anything as you want, no?

Well, I don't have a loader written yet, and I want to use this stack
trace feature while working on it. But even if I did, the addresses in
the ELF file produced by LLD are incoherent, and so it would not be
possible to map them.

Another alternative would be to ship debug symbol files separately on a
file system. I also don't have a file system written yet.

The bottom line is that, there's no fundamental reason this cannot work;
it is a quirk of conventions that is unnecessarily preventing the debug
info from being mapped into memory.

Yes, there’s no fundamental reason this cannot work. Rather, it should work. It does not immediately mean that lld needs to work for such linker script, since we intentionally try to not work too hard to support 100% linker script compatibility with GNU ld, as it’s not strictly doable due to the lack of a formal specification, and as it is sometimes extremely tricky. That said, it is probably worth taking a look at why this doesn’t work.

One more thing - this project as it currently exists is a proof of
concept of how a bare bones embedded system executable running on bare
metal can still have the same advanced debugging capabilities as when
using, e.g. Linux. If the debug info can be directly mapped, and thus
eliminate a dependency on a file system or on a special bootloader, it's
a great demo.

I think this use case is valid.

Here's a working proof of concept.

Source:
https://github.com/andrewrk/clashos/tree/bf8e57ac220715d0698ab910d337ea590c4b4e33

Screenshot: https://i.imgur.com/8C1tYys.png

Copy pasted text output:
[nix-shell:~/dev/clashos]$ zig build qemu
ClashOS 0.0

!KERNEL PANIC!
integer overflow
/home/andy/dev/clashos/src/serial.zig:99:7: 0x1b10 in ??? (clashos)
    x += 1;
      ^
/home/andy/dev/clashos/src/main.zig:58:16: 0x1110 in ??? (clashos)
    serial.boom();
               ^
/home/andy/dev/clashos/src/main.zig:67:18: 0xecc in ??? (clashos)
    some_function();
                 ^
???:?:?: 0x1c in ??? (???)

linker script snippet:

    .rodata : ALIGN(4K) {
        *(.rodata)
        __debug_info_start = .;
        KEEP(*(.debug_info))
        __debug_info_end = .;
        __debug_abbrev_start = .;
        KEEP(*(.debug_abbrev))
        __debug_abbrev_end = .;
        __debug_str_start = .;
        KEEP(*(.debug_str))
        __debug_str_end = .;
        __debug_line_start = .;
        KEEP(*(.debug_line))
        __debug_line_end = .;
        __debug_ranges_start = .;
        KEEP(*(.debug_ranges))
        __debug_ranges_end = .;
    }

LLD patch (submitted upstream at https://reviews.llvm.org/D55276):

--- a/deps/lld/ELF/OutputSections.cpp
+++ b/deps/lld/ELF/OutputSections.cpp
@@ -95,7 +95,7 @@ void OutputSection::addSection(InputSection *IS) {
     Flags = IS->Flags;
   } else {
     // Otherwise, check if new type or flags are compatible with
existing ones.
- unsigned Mask = SHF_ALLOC | SHF_TLS | SHF_LINK_ORDER;
+ unsigned Mask = SHF_TLS | SHF_LINK_ORDER;
     if ((Flags & Mask) != (IS->Flags & Mask))
       error("incompatible section flags for " + Name + "\n>>> " +
toString(IS) +
             ": 0x" + utohexstr(IS->Flags) + "\n>>> output section " +
Name +