assumptions about varargs ABI

Hi,

Various parts of LLVM seem to assume that the ABI for a varargs
function is compatible with the ABI for a non-varargs function, so
that code like this:

define void @f(i32 %x) {
    ...
}
...
    call void (...)* bitcast (void (i32)* @f to void (...)*)(i32 42)

will work.

(I don't think C guarantees that this will work, at least not in the
version of the C99 standard I checked.)

I'm trying to target a (proprietary) ABI where the ABI is rather
different for varargs and non-varargs functions.

The particular bit of llvm-gcc that is causing me trouble at the
moment is TreeToLLVM::StartFunctionBody():

  // If the function has no arguments and is varargs (...), turn it into a
  // non-varargs function by scanning the param list for the function. This
  // allows C functions declared as "T foo() {}" to be treated like
  // "T foo(void) {}" and allows us to handle functions with K&R-style
  // definitions correctly.

This gives rise to cases where functions are being called as if they
had varargs, but this optimisation has kicked in so the function is
defined without varargs.

Would it be reasonable to do the same optimisation at a call? That is,
given some code that calls a function declared with no prototype:

    void f();
    ...
    f(42);

to bitcast the callee into a function type based on what the actual
arguments being passed in are, as if the source had been:

  ((void (*)(int))&f)(42);

?

Jay.

Various parts of LLVM seem to assume that the ABI for a varargs
function is compatible with the ABI for a non-varargs function, so

This is due to 'K&R' C function handling. In K&R and ANSI C, you can do stuff like this:

void foo();

void bar() {
    foo(1, 2, 3);
}

void foo(int a, int b, int c) {}

and it needs to work.

(I don't think C guarantees that this will work, at least not in the
version of the C99 standard I checked.)
I'm trying to target a (proprietary) ABI where the ABI is rather
different for varargs and non-varargs functions.

I am pretty sure it is required to work, otherwise you can't call a function with no prototype.

Would it be reasonable to do the same optimisation at a call? That is,
given some code that calls a function declared with no prototype:

   void f();
   ...
   f(42);

to bitcast the callee into a function type based on what the actual
arguments being passed in are, as if the source had been:

((void (*)(int))&f)(42);

Yes, I believe that would be safe.

-Chris

> Various parts of LLVM seem to assume that the ABI for a varargs
> function is compatible with the ABI for a non-varargs function, so

This is due to 'K&R' C function handling. In K&R and ANSI C, you can do
stuff like this:

void foo();

void bar() {
    foo(1, 2, 3);
}

void foo(int a, int b, int c) {}

and it needs to work.

OK, but that's nothing to do with varargs. In C, "void foo()" gives
foo "a function type that does not include a prototype", which is
nothing to do with a varargs function type ("a function type that
includes a protoype which ends with an ellipsis"). It's only an LLVM
convention that you represent the declaration "void foo()" by using a
varargs function type.

I am pretty sure it is required to work, otherwise you can't call a
function with no prototype.

Calling a function via a declaration without a prototype is a bit like
calling it via a void * pointer which you then cast to the proper
function pointer type before calling it: the important thing is that
the type of the expression you call matches the type of the function
you end up at, regardless of any casting that might have gone on in
between.

In the case of calling a function declared without a prototype, the
important thing is that the types of the actual arguments you pass in
match the types of the parameters of the function you end up at.

Calling a varargs function through a pointer to non-varargs function
type, or vice versa, always gives you undefined behaviour.

(Chapter and verse: C99+TC1 6.5.2.2:9 "If the function is defined with
a type that is not compatible with the type (of the expression)
pointed to by the expression that denotes the called function, the
behaviour is undefined", 6.7.5.3:15 "For two function types to be
compatible, [...] the parameter type lists [...] shall agree [...] in
use of the ellipsis terminator".)

> Would it be reasonable to do the same optimisation at a call? That is,
> given some code that calls a function declared with no prototype:
>
> void f();
> ...
> f(42);
>
> to bitcast the callee into a function type based on what the actual
> arguments being passed in are, as if the source had been:
>
> ((void (*)(int))&f)(42);

Yes, I believe that would be safe.

OK, I'll have a go at implementing this.

Thanks,
Jay.