Crash bug for i1 vectors in X86ISelLowering

This crash occurs at least on 15.0.6 and 16.0.0 due to a bug in combinePredicateReduction

    EVT VecSVT = Match.getOperand(0).getValueType().getScalarType();
    if (VecSVT != MVT::i8) {
      NumElts *= VecSVT.getSizeInBits() / 8;
      EVT CmpVT = EVT::getVectorVT(*DAG.getContext(), MVT::i8, NumElts);
      MatchVT = EVT::getVectorVT(*DAG.getContext(), MVT::i1, NumElts);
      Match = DAG.getSetCC(
          DL, MatchVT, DAG.getBitcast(CmpVT, Match.getOperand(0)),
          DAG.getBitcast(CmpVT, Match.getOperand(1)), ISD::CondCode::SETEQ);

If VecSVT.getSizeInBits() returns 1, which is the case for i1 vectors, then the number of elements will be multiplied by 0. Clearly the expectation in this code is that the bit size is a multiple of 8. Further up this code is clearly only run on bit sizes 1, 8, 16, 32, 64.

Consequently I believe the correct fix here is to change if (VecSVT != MVT::i8) to if (VecSVT != MVT::i8 && VecSVT != MVT::i1)

Note that this bug only occurs when this optimization can be applied. First of all, the code is not called on vectors that does not have a power of 2 element length. Secondly, it only tends to happen if you first do a icmp on bool vectors, then immediately use that result to apply llvm.vector.reduce.and. If the comparison is between i8 vectors or greater, this doesn’t happen. Or the reduce is not immediately applied. I will try to investigate when the bug was introduced.

This seems to be the commit causing it: [X86] combinePredicateReduction - always use PMOVMSKB(PCMPEQB()) for … · llvm/llvm-project@f1305f2 · GitHub

Should be fixed by

Really fast fix @RKSimon. Awesome!