Lowering Call Return


it seems like SelectionDAGBuilder expects returning of vectors (structures/arrays) to be lowered in either of the two ways:

  1. Flatten the complex data types to simple data types, and return them using registers (done by TargetLowering::LowerCallTo)
  2. sret demotion: return the address of the complex data type via a stack pointer

Is there an option to do sret demotion via a register? if yes, do we have a Target to see an example implementation?


Hi Khaled,

Check out X86TargetLowering::LowerReturn, and the call to getSRetReturnReg.

The SRetReturnReg looks like a hack (each target that uses it, declares this variable individually), but that seems to be the current way of handling it.


How about vector parameters?

define internal fastcc <4 x float> @add(<4 x float> %a.val, <4 x float> %b.val) nounwind {
%tmp4 = fadd <4 x float> %a.val, %b.val
ret <4 x float> %tmp4

a and b are flattened by SelectionDAGISel::LowerArguments(const BasicBlock *LLVMBB) before letting the target handle it.

SDValue NewRoot = TLI.LowerFormalArguments(DAG.getRoot(), F.getCallingConv(),
F.isVarArg(), Ins,
dl, DAG, InVals);

The “Ins” have all the parameters flattened to primitive types. I would like the vector parameters to be lowered using vector registers.

That I don't know, but I'd look around TargetCallingConv.td, CallingConvLower.h, etc.


Thanks for the help.
Vector parameters were flattened due to absence of vector register class. I was working with Pseudo vector registers, meant for lowering calls args and returns only. The target does not support vector registers. So the approach of using Pseudo vector register seemed to complicate the lowering!

Are there any other ways of handling vectors from FE, if the target does not support vector registers? Has anyone considered converting the vectors to arrays rather than flattening? May be, do a llvm pass which converts all the llvm IR vectors to llvm IR arrays.