Unwinding call frames with separated data and return address stacks

I'm adding LLDB support for an unconventional platform which uses two
stacks: one purely for return addresses and another for frame context
(spilled registers, local variables, etc). There is no explicit link
between the two stacks, i.e. the frame context doesn't include any
pointer or index to identify the return address: the epilog for a
subroutine amounts to unwinding the frame context then finally popping
the top return address from the return stack. It has some resemblance
to the Intel CET scheme of shadow stacks, but without the primary
stack having a copy of the return address.

I can extend the emulation of the platform to better support LLDB. For
example while the real hardware platform provides no access to the
return address stack the emulation can expose it in the memory map,
provide an additional debug register for querying it, etc, which DWARF
expressions could then extract return addresses from. However doing
this seems to require knowing the frame number and I haven't found a
way of doing this (a pseudo-register manipulated by DWARF expressions
worked but needed some LLDB hacks to sneak it through the existing
link register handling, also seemed likely to be unstable against LLDB
implementation changes)

Is there a way to access the call frame number (or a reliable proxy)
from a DWARF expression? Or an existing example of unwinding a shadow
stack?

Thanks,
Tom

Hi Tom, interesting problem you're working on there.

I'm not sure any of the DWARF expression operators would work here. You want to have an expression that works for a given frame, saying "to find the caller's pc value, look at the saved-pc stack, third entry from the bottom of that stack." But that would require generating a different DWARF expression for the frame each time it shows up in a backtrace - which is unlike lldb's normal design of having an UnwindPlan for a function which is computed once and reused for the duration of the debug session.

I supposed you could add a user-defined DW_OP which means "get the current stack frame number" and then have your expression deref the emulated saved-pc stack to get the value?

lldb uses an intermediate representation of unwind information (UnwindPlan) which will use a DWARF expression, but you could also add an entry to UnwindPlan::Row::RegisterLocation::RestoreType which handled this, I suppose.

Hi Jason

Thanks for the advice - I've been surprised overall how capable DWARF
expressions are so wouldn't have been surprised to learn that there is
also a category of pseudo-variables (not that I can think of any
others, or other circumstances where it would be useful: the usual
combined code/data stack is ubiquitous). The RestoreType suggestion is
interesting as it might be a less-intrusive change.

Cheers,
Tom

I'm not sure I fully understood your setup, but it seems to me that this could be easily fixed if, in addition to the "fake" memory map, you could provide a fake "stack pointer" register which points to it.

Then, it should be possible to express the unwind info in regular debug_frame syntax:
previous_IP := [ fake_SP ]
previous_fake_SP := fake_SP +/- sizeof(IP)

regards,
pavel

Yeah, if you don't need to find a way to express this in DWARF, then adding a type to RestoreType would be very simple. lldb maps all the different unwind sources (debug_frame, eh_frame, arm index, compact unwind, assembly instruction scanning) into its internal intermediate representation (UnwindPlan) - so if you had an assembly-scanning unwind implementation for your target, you could add the appropriate RestoreType's. There are also architectural default unwind plans that are provided by the ABI plugin, both a default one (usually appropriate for frames up the stack) and an unwind plan that is valid at the first instruction of a function. These are good starting points for a new port, where you won't step through the prologue/epilogue correctly, but once you're in the middle of a function they can do a correct unwind on most architectures.

J

Hi Pavel

Thanks for that very useful notion - adding a fake stack pointer to
the target was indeed the least-intrusive approach (I had tried the
RestoreType approach suggested by Jason and it seemed likely to work
but required backporting 7.0 fixes into the regrettably-6.0-based
codeline).

I suspect it may be a mild guideline for avoiding further pain on this
unconventional target: when necessary use the platform emulation to
fake its being conventional.

Cheers,
Tom