Using VEX_4V may not be the right fix since Intel guide says “Operand 3: VEX.vvvv”
I was talking about this section of code:
void RecognizableInstr::handleOperand(
…
while (operandMapping[operandIndex] != operandIndex) {
Spec->operands[operandIndex].encoding = ENCODING_DUP;
Spec->operands[operandIndex].type =
(OperandType)(TYPE_DUP0 + operandMapping[operandIndex]);
++operandIndex;
}
…
}
void RecognizableInstr::emitInstructionSpecifier(DisassemblerTables &tables) {
…
case X86Local::MRMSrcMem:
// Operand 1 is a register operand in the Reg/Opcode field.
// Operand 2 is a memory operand (possibly SIB-extended)
// - In AVX, there is a register operand in the VEX.vvvv field here -
// Operand 3 (optional) is an immediate.
…
HANDLE_OPERAND(roRegister)
if (HasVEX_4VPrefix)
// FIXME: In AVX, the register below becomes the one encoded
// in ModRMVEX and the one above the one in the VEX.VVVV field
HANDLE_OPERAND(vvvvRegister)
if (HasMemOp4Prefix)
HANDLE_OPERAND(immediate)
HANDLE_OPERAND(memory)
if (HasVEX_4VOp3Prefix)
HANDLE_OPERAND(vvvvRegister)
…
}
For GATHER with 5 operands (dst, mask_wb, src1, mem, mask), the operandMapping is “0 to 0, 1 to 1, 2 to 0, 3 to 3, 4 to 1”, operand 2 is tied to operand 0, operand 4 is tied to operand 1.
So operand 2 and 4 (src1, mask) are treated as DUP, and the physical operands are 0,1,3(dst, mask_wb, mem), while MRSrcMem assumes Reg, Mem, Reg.vvvv if HasVEX_4VOp3Prefix is true.
Same situation happens in X86MCCodeEmitter.cpp, where TIED_TO for operand 2 is operand 0. We can only increment CurOp once for operand 2, since TIED_TO of operand 1 is -1.
What we really want is to reverse the direction of TIED_TO for mask and mask_wb.
We can probably hack this to handle the special case of two tied-to operands, if we have 2 tied-to operands, handle operand vvvvRegister first, then handle memory operand in MRMSrcMem.
In X86MCCodeEmitter.cpp, we increase CurOp twice if we have 2 tied-to operands. But it is kind of ugly.
- // FIXME: Handle the special case for GATHER:
- // For GATHER with 5 operands (dst, mask_wb, src1, mem, mask), src1 is tied
- // to dst and mask is tied to mask_wb. The operandMapping is "0 to 0,
- // 1 to 1, 2 to 0, 3 to 3, 4 to 1". The 2nd physical operand is mask_wb, and
- // it is before mem, so we need to explicitly handle vvvvRegister first.
- if (HasVEX_4VOp3Prefix && tiedOperandsCount >= 2)
- HANDLE_OPERAND(vvvvRegister)