Taking over an x86 opcode for my own instruction

tl;dr, I’d like to add my own instruction, but I’m running into problems due to my lack of x86 encoding/decoding understanding.

Hello all. Currently, I’m working on adding my own x86 instruction. I have done this once before; the basic process I used was:

  1. Find an unused opcode, e.g. 0xF1 in this table: http://ref.x86asm.net/coder32.html
  2. Insert an instruction into lib/Target/X86/X86InstrInfo.td using this opcode.
    In this case, I used 0xF1, and created the following instruction:

def CACHE_OPERAND_B64i : RIi64<0xF1, AddRegFrm, (outs GR64:$unused),
(ins i64imm:$b), “cache_operand_b_i64\t$b”,
[(int_cache_operand_b_i64 i64imm:$b)]>,

However, when I compile, I’m getting errors of the form:

Error: Primary decode conflict: TEST64ri32 would overwrite CACHE_OPERAND_B64i
ModRM 199
Opcode 247
Context IC_64BIT_REXW

When I look at the definition of TEST64ri32, I see:

let Predicates = [In64BitMode] in
def TEST64ri32 : BinOpRI_F<0xF6, “test”, Xi64, X86testpat, MRM0r>;

It looks like TEST64ri32 is using opcode 0xF6 – why is there a conflict with 0xF1?

Even if I comment out TEST64ri32 (which I recognize is not a good idea), I still get a similar error:

Error: Primary decode conflict: NOT64r would overwrite CACHE_OPERAND_B64i
ModRM 215
Opcode 247
Context IC_64BIT_REXW

Which corresponds to the definition:

def NOT64r : RI<0xF7, MRM2r, (outs GR64:$dst), (ins GR64:$src1),
[(set GR64:$dst, (not GR64:$src1))], IIC_UNARY_REG>;

Again, why does 0xF7 conflict with 0xF1?

Gus Smith

P.S. If this looks hacky, that’s because it is; I’m not sure what the correct way to add an instruction into x86 is, if there even is one. Pointers along that line are appreciated.

Forgot to reply all.

Your format says “AddRegFrm” which means a register is encoded in bits 2:0 of the opcode. This is used by instructions like the 0xB0-0xB7(MOV immediate byte into byte register) row of the one-byte opcode table. The disassembler generator automatically takes the opcode and generates 8 different opcodes from it.