RFC: New IR attribute incoming-stack-align

Hi,

I'm trying to fix an issue with clang's __force_align_arg_pointer__
attribute. The problem is described here:
https://llvm.org/bugs/show_bug.cgi?id=26662

The problem is affecting Wine (https://www.winehq.org/) where we have
a function that is an entry-point for the x86 Win32 abi. That
function may then call functions in the host's (linux, OS/X, ...) abi.
The issue is that the Win32 abi guarantees stack alignment of four
bytes while the host abi is typically sixteen.

This is supposed to be solved by clang's __force_align_arg_pointer__
attribute. What this does is to set the IR attribute alignstack = 16.
While this does correctly realign the stack it does not change the
assumption about the alignment of the incoming stack - which is still
assumed to have an alignment of sixteen. We need a way to signal to
the stack layout generator that the incoming alignment can only be
assumed to be four.

In contrast, gcc realigns the stack but also reduces the assumed
alignment of the incoming stack to four.

To fix this I'm proposing a new IR attribute "incoming-stack-align"
that would be set by clang in addition to the alignstack attribute.
I'm attaching patches to llvm and clang that implement this. The
patches are most certainly not correct, but I'd appreciate any
feedback about how to fix this.

Many thanks,
Huw.

add_incoming.patch (838 Bytes)

clang_add_incoming.patch (611 Bytes)

LLVM managed to survive the 16 byte stack alignment apocalypse on 32-bit Linux without adding concepts of incoming and outgoing stack alignment, so I’m hesitant to go back and this extra complexity now.

If we’re willing to tolerate the extra complexity of having yet another way to fiddle with stack alignment, then I agree separating the concepts of incoming and outgoing SP alignment is the way to go.

I'm open to other suggestions. The problem is that these functions
service an incoming abi stack alignment that differs from the host's
abi alignment.

An alternative would be to have the 'alignstack' IR attribute itself
reduce the incoming stack alignment assumption. i.e. if it's
specified then it's fair to assume that the incoming stack does not
have the correct alignment, so we should assume it has a smaller
alignment - which we would probably just pick as 4. Would that be any
better?

Huw.

I'd be in favor of that.

Wouldn't a better way be to say that if alignstack is specified,
arguments have their minimum required alignment and that's it?

Joerg