lowering varargs, rewriting types


I’m looking at writing a custom backend targetting a proprietary virtual machine. I’m basing it on the existing C and MSIL backends.

The VM has a defined ABI for varargs functions: the caller allocates some memory to hold the arguments and passes a pointer to this memory as an extra parameter in the call. I’d like to write a varargs lowering pass that represents this ABI explicitly in the LLVM code, replacing varargs intrinsics with explicit operations on pointers. Does a pass like this already exist?

I think part of the work of this pass would be rewriting function types throughout the entire module. For example, if a global variable has a pointer-to-varargs-function type (or even a structure type with a pointer-to-varargs-function buried deep within it) then that type will have to be rewritten so that the function type has an extra pointer parameter instead of the “…” at the end of the parameter list. As I understand it you can’t change the type of a global variable in place, so I’d have to add a new variable with the tweaked type and delete the original one. It seems like quite a lot of work to have to rewrite large chunks of the module like this when all I want to do is make a small change to function types. Is there any support for doing this kind of tweaking of types across a whole module?

An alternative to tweaking types across the whole module would be to only change the types of varargs function definitions, and to insert bitcasts at every use of a varargs function and every call with a varargs function type. But that doesn’t sound very clean!

I’d appreciate any thoughts on this.