-msave-args backend support for x86_64

Project Infrastructure LLVM Dev List Archives
1

ola,

ive been looking at adding support for an -msave-args option for
use on x86_64. the short explanation of it is that it makes x86_64
function prologues store their register arguments on the stack. the
purpose of this is to make the arguments trivially accessible for
things like stack traces with arguments.

as per
https://blogs.oracle.com/sherrym/entry/obtaining_function_arguments_on_amd64,
this was originally implemented by sun in the various compilers
they use to support the debugging facilities in their system. ive
been looking at doing the same thing on openbsd for the same reasons
which you can see at
'add -msave-args to gcc on amd64' - MARC and
'arguments in amd64 ddb via -msave-args' - MARC.

there's a strong possibility that openbsd will switch to clang and
llvm on amd64, so i had a look at implementing this in clang. i
know the illumos community is interested in this functionality,
presumably as a way forward from the old gcc theyre still using.

i am a fair way along but i wanted to ask for advice on how to
proceed from this point. ive only been hacking on llvm for a day
or so, so id appreciate some advice from people with experience
before i head too far down what could be the wrong path.

its not obvious to me what what the etiquette is for sending diffs
so it's inline below. it is also available at
https://mild.embarrassm.net/~dlg/diff/llvm.msave.trunk

this does enough that it generally works. it basically shoves some
extra PUSHes into the prologue, and tries to account for them
properly so other uses of the frame and the epilogue works.

so there are some issues with the code:

- it (probably) doesnt handle functions that return structs
- it doesnt realign the stack pointer after consuming space on it
- it doesnt restrict the use of -msave-args to generation of 64bit
  code.

so my questions are:

1. my understanding is if a function returns a struct, the caller
is responsible for allocating space for the struct and passes a
pointer to the callee via RDI, which takes a register away from
arguments.

is that true? what's the best way to detect that in
X86FrameLowering::emitPrologue()?

2. i copied get64BitArgumentGPRs from X86ISelLowering.cpp.

i need this so i know which registers to push onto the stack and
in which order.

should i move it to X86RegisterInfo.cpp? could someone give me a
pointer on following the FIXME, ie, how could i get that stuff from
tblgen?

if anyone has some tips for me, it would be greatly appreciated.

thanks in advance,
dlg

Index: lib/Target/X86/X86.td

2

This seems like a fragile and heavy hammer if your goal is simply to allow debuggers to find the arguments. Wouldn’t it be simpler to mark the arguments as live for the entire call, so that the back end will either kepp them in registers or spill them, depending on register pressure, and update the DWARF frame info so that you can find them?

David

3

This seems like a fragile and heavy hammer if your goal is simply to allow debuggers to find the arguments. Wouldn’t it be simpler to mark the arguments as live for the entire call, so that the back end will either kepp them in registers or spill them, depending on register pressure, and update the DWARF frame info so that you can find them?

that assumes that shipping dwarf is reasonable and handling dwarf is simple. in my situation im trying to make an in kernel debugger more useful. the debugger is the thing that happens when the kernel crashes. adding dwarf to the kernel increases its size by more than a factor of 4 (44.7M vs 10.6M), and adding code for dwarf handling would increase that size further. shipping dwarf in a .debuginfo style file makes it hard to access that info during a crash, since the thing that failed is the thing you need to use to get the .debuginfo file.

alternatively, emitting a limited series of pushes and adjusting the stack pointer is simple. i’d say it's roughly equivalent to the cost of stack probes, but simpler. it’s also simple to pull apart. the code to handle reading arguments the callee pushed onto the stack is literally an additional 9 lines, before falling through to the existing code that reads the rest of the arguments out of the callers frame.

it has been robust in deployment for over a decade in solaris and derivatives like illumos. if anyone could have justified shipping dwarf, it would have been the solaris team with their emphasis on debugging and introspecting their system. the fact that they didn’t is telling in my opinion.

cheers,
dlg

4

It is certainly missing frame info annotation, but the basic approach is
similar to what is already done for variadic functions on many
architectures with register-based argument handling. That leads me to
the question on whether this shouldn't be a generic code generation
flag.

Joerg

5

This doesn't need full debug data, just .eh_frame when emitting the
proper markers. From a debugger perspective it means unwinding into the
caller's frame and asking for the value of the registers at the call
site. The advantage is that code generation can pick different options
based on the individual requirements, i.e. it no longer has to be a
strict binary choice but can accomodate different use cases ("I will use
a debugger, so skip redundant instructions if it is still in a
register").

Joerg

6

FWIW MSVC has a similar flag called /homeparams:
https://msdn.microsoft.com/en-us/library/6exwh0y6.aspx

So, I think there is some value in this feature, even ignoring vector and floating point arguments, which won’t fit into these general-purpose register parameters. Win64 doesn’t have this problem because they don’t pass values larger than 8 bytes by value, including vectors. On Win64, we should also store arguments into the 32 byte shadow space allocated by the caller for varargs purposes to match tools expecting MSVC’s behavior.

Your patch doesn’t appear to update the dwarf or seh call frame info, which is very important. I might grab this at some point, since I have some experience with that.