[RFC] Introducing an explicit calling convention

Hi All,

TLDR: Allow calling conventions to be defined on-the-fly for functions
in LLVM-IR, comments are requested on the mechanism and syntax.

Summary

I'm not convinced by this part of the proposal. It is valid for x86, for calls that use a normal call instruction and so the return address is pushed onto the stack, but on pretty much any other architecture this is not the case: the return address is in a register and it's the responsibility of the callee prolog to spill it (or to not spill it, if this is a leaf function), so may well not be at the start of the return area. It's also unlikely to interact well with safe stack and similar techniques.

Perhaps more importantly, this works only for calling conventions where the caller is responsible for cleaning up the stack. Some calling conventions (e.g. Windows' stdcall) require the callee to clean up the stack, supporting these would require that we be able to find on-stack parameters and move the stack pointer from IR.

Supporting any of these probably requires more explicit prolog and epilog instructions in the IR.

I'm not opposed to this in principle, and actually I'd quite like to move in this direction and remove our reliance on undocumented and inconsistent conventions between the back end and the front end for conveying information about ABIs. For example, returning two 32-bit integers or a pair of pointers on x86-32 requires returning the result in a single i64 in LLVM IR (on platforms where small structs are returned in registers, not on Linux), which is not particularly helpful for analysis or consistent with any other architecture. Given that front ends have to be aware of calling conventions, it would be nice if they could express them in the same way that the ABI references do...

David

David Chisnall via llvm-dev <llvm-dev@lists.llvm.org> writes:

I'm not opposed to this in principle, and actually I'd quite like to
move in this direction and remove our reliance on undocumented and
inconsistent conventions between the back end and the front end for
conveying information about ABIs. For example, returning two 32-bit
integers or a pair of pointers on x86-32 requires returning the result
in a single i64 in LLVM IR (on platforms where small structs are
returned in registers, not on Linux), which is not particularly
helpful for analysis or consistent with any other architecture. Given
that front ends have to be aware of calling conventions, it would be
nice if they could express them in the same way that the ABI
references do...

+1. Coordinating ABI semantics between the frontend and LLVM is tricky.
It would be super helpful to have a formal way of expressing ABI
semantics in the IR.

                           -David

I agree with that, but I think most users want LLVM to go the other way from what the OP is suggesting. I think non-clang frontends would prefer it if we pushed more calling convention details down into LLVM, not the other way around.

Regarding the original proposal, sure, we could probably support a calling convention that explicitly assigns arguments to same-sized registers. But I doubt that it will ever be expressive enough to handle the corner cases of HVA and struct passing. Those are the areas where other frontends typically run into ABI compatibility problems. So, we can definitely add a convention that explicitly assigns registers, but we should document that it’s really only going to handle the easy cases of assigning register-sized arguments to appropriately sized registers.

I suggest that the convention use string attributes, instead of attributes that point to metadata. We don’t have any facilities to allow attributes to point to metadata, but we already support string attributes on functions, and it should be easy to extend the IR parsing to support an argument attribute that looks like hwreg("rax,rdx"). I don’t think it will impact bitcode serialization, but I could be wrong. For the list of CSRs, you can add an attribute like "calleesavedregs"="rbx,rcx" to the function’s attribute set. That won’t require any bitcode or IR parser modifications.

Hope that helps!

Reid Kleckner via llvm-dev <llvm-dev@lists.llvm.org> writes:

    +1. Coordinating ABI semantics between the frontend and LLVM is
    tricky.
    It would be super helpful to have a formal way of expressing ABI
    semantics in the IR.

I agree with that, but I think most users want LLVM to go the other
way from what the OP is suggesting. I think non-clang frontends would
prefer it if we pushed more calling convention details down into LLVM,
not the other way around.

That would be great, but I'm not even sure how it would be possible
given the multitude of frontend languages and their ABI expectations.
Wouldn't we have to carry quite a lot of language semantic information
around? For example, AFAIK right now LLVM can't distinguish between an
ordinary struct containing two doubles and double complex. The various
vector ABIs may treat those two types differently, for example.

                        -David

Yep, we definitely would. Personally, I think we already pass so much frontend info down in the form of argument attributes that we might as well design a way to just directly say “these two doubles are C99 _Complex” instead of saying inventing ad-hoc conventions like “a struct of two doubles means one thing, but an array of two doubles means something else” or repurposing attributes like byval and inreg.

[+CC Kai Nacke, who I remember talking about this topic at a FOSDEM 2014 talk]

David Chisnall via llvm-dev <llvm-dev@lists.llvm.org> writes:

> I'm not opposed to this in principle, and actually I'd quite like to
> move in this direction and remove our reliance on undocumented and
> inconsistent conventions between the back end and the front end for
> conveying information about ABIs. For example, returning two 32-bit
> integers or a pair of pointers on x86-32 requires returning the result
> in a single i64 in LLVM IR (on platforms where small structs are
> returned in registers, not on Linux), which is not particularly
> helpful for analysis or consistent with any other architecture. Given
> that front ends have to be aware of calling conventions, it would be
> nice if they could express them in the same way that the ABI
> references do...

+1. Coordinating ABI semantics between the frontend and LLVM is tricky.
It would be super helpful to have a formal way of expressing ABI
semantics in the IR.

I agree with that, but I think most users want LLVM to go the other way
from what the OP is suggesting. I think non-clang frontends would prefer it
if we pushed more calling convention details down into LLVM, not the other
way around.

I agree that handling the calling conventions in one place (e.g. the frontend) would be a good idea.

Sometimes people complain that ABI handling in the frontend is hard and that they have to duplicate (in their language’s frontend) what Clang does. However, I don’t think that pushing more of this complexity into LLVM would be a good idea. What I suggest is that there should be a helper library for ABI lowering (ideally that library should be used by Clang eventually). What do you think?

-Manuel

I generally support the goal - we have the same problem described in a bit of detail just below - but I'm really not sure about the framing of the solution here.

Our variation on the problem is that we have many distinct calling conventions which are slight variation for each other. We're adapting a legacy collection of hand written assembly stubs each which had a slightly different calling convention. The typical difference is that one stub might kill a register that another preserves. At the moment, we've solved this through a mixture of a bunch of custom calling conventions declared downstream, and normalizing stubs where possible. The former is tedious; the later is rather error prone. The key thing for us is that variation between stubs is primarily small and mostly in the callee saved lists.

On the framing piece, the ABI is really a property of the callee, not of the arguments. As such, I think this really deserves to either be a first class syntax for spelling a calling convention, or an attribute. I'd suggest framing the description of the calling convention as "like this existing calling convention XXX, but w/o this callee saved register" or "like this existing calling convention XYZ, but with one argument register defined". Possibly spellings might include:

declare "C" {i64, i64} @example(i64 %a, i64 %b) ccoverride(noclobber={rax, rbx}, arg_reg_order={rcx, rdx}, ret_arg_regs={rdx, rcx))
declare "C" {i64, i64} @example(i64 %a, i64 %b) ccoverride="(noclobber={rax, rbx}, arg_reg_order={rcx, rdx}, ret_arg_regs={rdx, rcx))"

(The second one abuses String attributes horribly, but would be the easiest to implement.)

An alternate way of framing this would be to provide a clean interface for plugging in an externally defined calling convention w/o needing to rebuild LLVM. This would require a custom driver, but would avoid the need to build LLVM. This would solve our problem cleanly - and is probably what I'd get around to implementing someday - but I'm not sure how it matches your original use case.

Philip

One way to see if this mechanism is suitably flexible would be to check if it’s possible to implement #pragma parameter from 68K-based Mac and Palm compilers. This syntax allowed specifying specific registers to be used for arguments in a similar way.

Alex

I would love to see this and I was hoping that some of the Swift work would lead to a refactored clang library that allowed you to generate IR corresponding to C/C++ functions.

I think we have four use cases for front ends:

1. JIT'd language, minimal FFI. This doesn't care what the calling convention is, doesn't care that it's stable, does care that it's efficient. Except at well-defined external calls, is very happy for LLVM to pick a different calling convention for every function.

2. Language that has only an LLVM implementation, but does support static compilation and linking across different compiler versions. Doesn't care what the calling convention is, but it has to be stable. LLVM is free to pick a different calling convention for every type signature, but the convention must be stable and doesn't have to match the platform ABI.

3. Language that is adding a new LLVM implementation that is expected to interoperate with an existing implementation. There are a lot of these out there and they all end up suffering. Typically they either fork LLVM and add a custom calling convention, or they try to upstream a calling convention. When they want to support a new architecture, then need to coordinate their releases with LLVM to get their calling conventions supported by the new back end. They make live even harder for the next person to want to use their calling convention by adding more undocumented informal contracts between how the front end generates LLVM IR and how the back end maps this to register / stack layout.

4. Language that wants to interoperate with C/C++. These sometimes embed a big chunk of clang. This probably should be the preferred solution, because it will invariably miss corner cases (packed structs, for example) if it does anything simpler. Unfortunately, clang isn't really designed for this kind of embedding and the APIs that consumers need change quite frequently. Alternatively, they implement their own partial ABI support. For the core C ABI (ignoring GNU extensions and C++), it is *much* easier to do this in a JIT that generates assembly directly than one that targets LLVM IR. ABI specs describe a fairly simple (even amd64, though that's stretching it somewhat) set of rules for mapping C constructs to registers / stack offsets. These are well defined and well documented. The corresponding set of mappings from C constructs to IR constructs are undocumented outside of the code and the only way that you can reliably find them is to run clang and see what it emits (I have done this - it's not fun). If you need only basic C interop, doing this for each platform that you support ought to be simpler than embedding clang, but it isn't.

Ideally, we'd have both a clean set of APIs for other front ends to embed clang (including a lot of what libclang exposes: parsing C headers, walking types, and then generating interoperable code), built on top of a clean way for the front end to either specify the exact ABI requirements or leave them entirely up to later optimisations or the back end.

David

David Chisnall via llvm-dev <llvm-dev@lists.llvm.org> writes:

Only parameters passed in registers are considered as the
llvm.addressofreturnaddress intrinsic can be used to calculate the
location of values on the callers stack.

I'm not convinced by this part of the proposal. It is valid for x86,
for calls that use a normal call instruction and so the return address
is pushed onto the stack, but on pretty much any other architecture
this is not the case: [...]

You're right, that only works for x86. A cleaner way would be to require
that if the stackpointer is associated with a parameter, the value of
that parameter should be the caller's stackpointer. The explicit CC
would then have to take this into account while producing its prologue,
but that shouldn't be worse than what we are already doing on return
[see below].

Perhaps more importantly, this works only for calling conventions
where the caller is responsible for cleaning up the stack. Some
calling conventions (e.g. Windows' stdcall) require the callee to
clean up the stack, supporting these would require that we be able to
find on-stack parameters and move the stack pointer from IR.

In our prototype implementation, used for an Erlang JIT, we have call
chains (not tail calls) of explicitcc functions where the leaf does a
direct return back to the initial call. For that we have implemented a
stack-adjustment function attribute which allows us to adjust the stack
pointer on return (from the stackmap we know the size of each activation
record on the stack and our runtime system knows the call tree).

--Frej

Reid Kleckner via llvm-dev <llvm-dev@lists.llvm.org> writes:

Regarding the original proposal, sure, we could probably support a calling
convention that explicitly assigns arguments to same-sized registers. But I
doubt that it will ever be expressive enough to handle the corner cases of
HVA and struct passing. Those are the areas where other frontends typically
run into ABI compatibility problems. So, we can definitely add a convention
that explicitly assigns registers, but we should document that it's really
only going to handle the easy cases of assigning register-sized arguments
to appropriately sized registers.

I understand and support the desire for having a declarative way to
describe calling conventions but remember that the original proposal is
only for the callee-side of the calling convention. Its intended use is
for producing code which slots into, for example the shadow of a
stackmap or a manually analyzed binary. In this case, the assignment of
values to registers has already been done by something else, that is why
the proposal does not cover calls. Normally the user is responsible for
both the arguments to the stackmap intrinsics and building the function
called from its shadow. So matching up registers to types is in practice
not a problem.

I suggest that the convention use string attributes, instead of attributes
that point to metadata. We don't have any facilities to allow attributes to
point to metadata, but we already support string attributes on functions,
and it should be easy to extend the IR parsing to support an argument
attribute that looks like `hwreg("rax,rdx")`. I don't think it will impact
bitcode serialization, but I could be wrong. For the list of CSRs, you can
add an attribute like `"calleesavedregs"="rbx,rcx"` to the function's
attribute set. That won't require any bitcode or IR parser modifications.

That's a good suggestion. Right now our prototype just uses raw register
numbers (it was first made when llvm.read_register used a raw register
number), switching to string attributes will probably make the patch
less intrusive.

--Frej

Philip Reames <listmail@philipreames.com> writes:

An alternate way of framing this would be to provide a clean interface
for plugging in an externally defined calling convention w/o needing
to rebuild LLVM. This would require a custom driver, but would avoid
the need to build LLVM. This would solve our problem cleanly - and is
probably what I'd get around to implementing someday - but I'm not
sure how it matches your original use case.

I like this solution a lot. It would provide everything I need for my
use case and would sidestep the discussions about the syntax and exactly
what it should express. As far as I understand, it could also be used by
the ABI-lowering library suggested by Manuel Jacob and David Chisnall in
[1,2].

I'll wait a day or so to see if I get more suggestions before writing an
updated/alternative proposal.

--Frej

[1] http://lists.llvm.org/pipermail/llvm-dev/2019-January/129184.html
[2] http://lists.llvm.org/pipermail/llvm-dev/2019-January/129207.html

David Chisnall via llvm-dev <llvm-dev@lists.llvm.org> writes:

Sometimes people complain that ABI handling in the frontend is hard
and that they have to duplicate (in their language’s frontend) what
Clang does. However, I don’t think that pushing more of this
complexity into LLVM would be a good idea. What I suggest is that
there should be a helper library for ABI lowering (ideally that
library should be used by Clang eventually). What do you think?

I would love to see this and I was hoping that some of the Swift work
would lead to a refactored clang library that allowed you to generate
IR corresponding to C/C++ functions.

Such a library would still have to have a way to send the low-level
information about register and stack slot mappings to the back-end. The
original proposal would have to be extended to support calling functions
with the explicit CC, but then it could serve as the target of this ABI
lowering library.

I quite like Philip Reames suggestion[1] (elsewhere in this thread) of
just providing a way to plug in an externally defined calling
convention. The ABI lowering library could produce such plugins and we
don't have to worry about syntax and the expressibility of the CC
description.

--Frej

[1] http://lists.llvm.org/pipermail/llvm-dev/2019-January/129189.html

Philip Reames via llvm-dev <llvm-dev@lists.llvm.org> writes:

On the framing piece, the ABI is really a property of the callee, not
of the arguments.

The *calling convention* might be so, but the ABI is definitely not.
For a given language, the ABI is a property of the target. The ABI
covers a lot more ground than just the calling convention. With more
complex argument types, the "calling convention" depends on a lot of ABI
information beyond caller-save/callee-save/argument/return register
specifications. It has to know what the types look like in memory.

As such, I think this really deserves to either be a first class
syntax for spelling a calling convention, or an attribute. I'd suggest
framing the description of the calling convention as "like this
existing calling convention XXX, but w/o this callee saved register" or
"like this existing calling convention XYZ, but with one argument
register defined". Possibly spellings might include:

declare "C" {i64, i64} @example(i64 %a, i64 %b)
ccoverride(noclobber={rax, rbx}, arg_reg_order={rcx, rdx},
ret_arg_regs={rdx, rcx))
declare "C" {i64, i64} @example(i64 %a, i64 %b)
ccoverride="(noclobber={rax, rbx}, arg_reg_order={rcx, rdx},
ret_arg_regs={rdx, rcx))"

(The second one abuses String attributes horribly, but would be the
easiest to implement.)

This might work for simple scalar arguments but quickly breaks down in
the presence of aggregates. Describing, for example, the layout of C
struct types and their mapping to registers in the calling convention is
non-trivial.

An alternate way of framing this would be to provide a clean interface
for plugging in an externally defined calling convention w/o needing
to rebuild LLVM. This would require a custom driver, but would avoid
the need to build LLVM. This would solve our problem cleanly - and is
probably what I'd get around to implementing someday - but I'm not
sure how it matches your original use case.

This is an interesting idea. Keeping in mind what Manuel Jacob[1] and
David Chisnall[2] have said, what I've really wanted in my work is
something that, given a source-level function signature, a set of
argument Values and an optional Value in which to store the returned
value, would tell me how to generate the LLVM IR to pass the arguments
and store the return value. I've also wanted something that, given a
source type, would generate an LLVM type that correctly implements the
ABI layout. Finally, I'd want something that, given a source type, a
Value of the correspoding LLVM type and a source field access specifier,
would generate the IR to read from or write to the field.

StructType kinda-sorta provides some of the latter two, but not really.

The above is written with a C/C++ lens and other languages may need
more/different things from an ABI library.

                         -David

[1] http://lists.llvm.org/pipermail/llvm-dev/2019-January/129184.html
[2] http://lists.llvm.org/pipermail/llvm-dev/2019-January/129207.html

If anyone would like to organize a round table at EuroLLVM to talk about this topic, I’d love to participate. I have lots of thoughts and opinions here (but can’t really explain it all without a whiteboard) and they may be interesting to discuss.

-Chris

I'd also be interested in attending, but unfortunately this year EuroLLVM was scheduled for the week immediately after Brexit, when travelling to the EU from the UK is expected to be problematic.

David

A bit off-topic, but that seems .. unlikely.

As a New Zealand passport holder, I can visit the EU for 90 days
without a visa. Ok, perhaps the UK won't have such an agreement in
place immediately.

BUT! NZ still has all the pre-EU/EEC bilateral agreements in place. So
I believe I can visit Finland for 90 days, then go to Sweden for 90
days, then Denmark for 90 days, Netherlands for 90 days, Belgium,
France, Germany .. and on for 90 days in each of 20 or so countries.
It is necessary only to find someone to stamp my passport when I go
from one country to another. I believe a police station will suffice.

It would be surprising if the UK didn't still have the pre-1973
bilateral agreements on the books as well.