Using sret AND thiscall calling convention

Hi all,

I am trying to generate LLVM IR that calls an external C++ function
returning a structure by copy:
vec3 vec3::Cross(const vec3& iV) const;

Here is the LLVM IR that I am generating for win32 ABI, which says that the
first parameter should be a pointer to the return value:

%vec3 = type <{ float, float, float }>

define void @CPP_Return_Struct(%vec3* %v1, %vec3* %v2) inlinehint
alignstack(16) {
ENTRY:
  %"@0" = alloca %vec3, align 4
  call x86_thiscallcc void @vec3_Cross(%vec3* sret %"@0", %vec3* %v1, %vec3*
%v2)
  ret void
}

declare x86_thiscallcc void @vec3_Cross(%vec3* sret, %vec3*, %vec3*) align 2

It seems that the code generated by the JIT takes the first parameter for
the this value because of the X86_thiscall calling convention. However, in
this particular case, the first parameter is the return value which is
tagged with the sret attribute.

I have manually modified the X86GenCallingConv.inc to treat the second
parameter as this value when the first has been tagged with sret attribute,
and it is working fine.

But I am not sure if it is a (known?) bug or if I do not generate the
correct LLVM IR.

I found nothing about this on the mailing list, I apologize if I missed any
important information.

Damien

I would suggest avoiding any complicated calls from LLVM-generated
code into Visual Studio-generated code. For example,
http://llvm.org/bugs/show_bug.cgi?id=5058 ,
http://llvm.org/bugs/show_bug.cgi?id=5064 . Not sure about the
particular issue you're running into; looks like a bug at first
glance. The file that mostly controls this sort of thing is
lib/Target/X86/X86CallingConv.td .

-Eli

Thank you for the answer.

I am not really sure, but I do not think I have the issue described here
http://llvm.org/bugs/show_bug.cgi?id=5058, because according to the attached
CC document, when using thiscall the caller is not responsible for stack
cleaning.

Seems to be a bug, I created a new entry here
http://llvm.org/bugs/show_bug.cgi?id=10645, I hope I done it correctly :slight_smile:

Thanks again,
Damien

Eli Friedman-2 wrote: