SplitVecRes with SIGN_EXTEND_INREG unsupported

I have code that is generating sign extend in reg on a v8i32, but the backend does not support this data type. This then asserts in LegalizeVectorTypes.cpp:389 because there is no function to split this vector into smaller sizes. Would a correct solution be to add this case so to trigger the SplitVecRes_BinaryOp function?

This asserts on both my backend and x86 and TOT does not seem to have any code to handle this.

Thanks,

Micah

SIGN_EXTEND_INREG isn't a binary operation; the correct expansion is
expanding the first operand the same way SplitVecRes_BinaryOp does,
while passing through the second operand untouched. But yes, adding a
case to DAGTypeLegalizer::SplitVectorResult is the right idea.

If anyone else is curious, here's a testcase which crashes on x86 when
llc is run over it:
define <8 x i32> @a(<8 x i32> %a) {
  %b = trunc <8 x i32> %a to <8 x i16>
  %c = sext <8 x i16> %b to <8 x i32>
  ret <8 x i32> %c
}

-Eli

Thanks Eli,
I'll see if I can get something working and submit a patch.

Micah

Eli,
I have a simple SplitVecRes function that implements what you mentioned, splitting the LHS just as in BinaryOp, but passing through the RHS. The problem is that the second operand is MVT::Other, but when casted to an VTSDNode reveals that it is a vector length of the same size as the LHS SDValue. This causes a split on the LHS side to work correctly, but then it fails instruction selection because of Other. I have not been able to figure out how to split the MVT::Other node yet, any idea how to do this?

Instruction selection fails because it cannot match the pattern:
v4i8 = sign_extend_inreg <v4i8 node>, <Other node(v8i8 VTSDNode)>

The reason being my initial implementation based on the advice given takes as input:
v8i8 = sign_extend_inreg <v8i8 node>, <Other node(v8i8 VTSDNode)>
and generates two:
v4i8 = sign_extend_inreg <v4i8 node>, <Other node(v8i8 VTSDNode)>

Instead it should generate:
v4i8 = sign_extend_inreg <v4i8 node>, <Other node(v4i8 VTSDNode)>

So, how would I be able to split the Other node so that it will match the resulting data type?

My function looks like this:
void DAGTypeLegalizer::SplitVecRes_SIGN_EXTEND_INREG(SDNode *N, SDValue &Lo,
                                         SDValue &Hi) {
  SDValue LHSLo, LHSHi;
  GetSplitVector(N->getOperand(0), LHSLo, LHSHi);
  SDValue RHS = N->getOperand(1);
  DebugLoc dl = N->getDebugLoc();

  Lo = DAG.getNode(N->getOpcode(), dl, LHSLo.getValueType(), LHSLo, RHS);
  Hi = DAG.getNode(N->getOpcode(), dl, LHSHi.getValueType(), LHSHi, RHS);
}
Thanks,
Micah

You should be able to split the contained type with GetSplitDestVTs,
then recreate the node using SelectionDAG::getValueType(), I think.

That said, it could possibly be considered a bug in DAGCombine that
the second operand is a vector type; someone want to comment on that?

-Eli

Eli,
I don't see how this helps with the splitting of the Other node as it isn't the Dest that is the problem, but the second source value. Any place in the code that I can look at on how to split a VTSDNode?

Thanks,
Micah

Eli,
I think I was able to get it working. Thanks for the help, does this look correct to you?

void DAGTypeLegalizer::SplitVecRes_SIGN_EXTEND_INREG(SDNode *N, SDValue &Lo,
                                         SDValue &Hi) {
  SDValue LHSLo, LHSHi;
  GetSplitVector(N->getOperand(0), LHSLo, LHSHi);
  DebugLoc dl = N->getDebugLoc();
  EVT LoVT, HiVT;
  GetSplitDestVTs(N->getValueType(1), LoVT, HiVT);

  Lo = DAG.getNode(N->getOpcode(), dl, LHSLo.getValueType(), LHSLo, DAG.getValueType(LoVT));
  Hi = DAG.getNode(N->getOpcode(), dl, LHSHi.getValueType(), LHSHi, DAG.getValueType(HiVT));
}

Thanks,
Micah

Ok, It doesn't work. The problem is LLVM then asserts later on in SelectionDAG:2642 because it is checking to see whether the second operand is an Integer, and if not it assumes it is floating point and asserts with the method Cannot *_EXTEND_INREG FP types.

So, it seems that the root problem here is the 'MVT::Other' still hanging around. How do I convert this SDValue to an int vector that LLVM won't complain about?

Thanks,
Micah

After more digging, it seems that the SIGN_EXTEND_INREG is getting generated in DAGCombiner.cpp:3033.
// fold (sext (truncate x)) -> (sextinreg x).
    if (!LegalOperations || TLI.isOperationLegal(ISD::SIGN_EXTEND_INREG,
                                                 N0.getValueType())) {
      if (Op.getValueType().bitsLT(VT))
        Op = DAG.getNode(ISD::ANY_EXTEND, N0.getDebugLoc(), VT, Op);
      else if (Op.getValueType().bitsGT(VT))
        Op = DAG.getNode(ISD::TRUNCATE, N0.getDebugLoc(), VT, Op);
      return DAG.getNode(ISD::SIGN_EXTEND_INREG, N->getDebugLoc(), VT, Op,
                         DAG.getValueType(N0.getValueType()));
    }

One question is, should this code be executed on vectors? The description of SIGN_EXTEND_INREG mentions that this should be an atomic operation. I don't know of any mainstream hardware that can do atomic updates on a vector at once, as most have 32 or 64 bit atomics.

A question that has more impact, should the DAG combiner generate any combination of instructions that it cannot undo? I would say that it shouldn't, and this is a bug in LLVM.

This is one situation where it generates code that causes it to assert, whereas the original code works if this combination is disabled or only generated on scalar types.

Idea's?

Micah

I'd be fine with simply disabling the transformation in question on vectors.

-Eli