I’m using the MCDisassembler for x86, and I’m having a really hard time handling instruction prefixes properly.
Here’s a concise example of the types of problems I am having:
66 F2 AF REPNE SCASW WORD PTR ES:[EDI]
F2 66 AF REPNE SCASW WORD PTR ES:[EDI]
llvm-mc does fine when the REPNE prefix is first, but gives incorrect disassembly when it follows the OPSIZE override.
With ToT, this is what I get:
scasl %es:(%edi), %eax ← incorrect (ignored osz)
scasw %es:(%edi), %ax ← correct
I have followed through the getID function, where it uses the attrMask of ATTR_XD, and then ands in the ATTR_OPSIZE to see if there is an equivalent.
However, that second call to getIDWithAttrMask returns PHI. For some reason there isn’t a way I’ve found to indicate that certain instructions are allowed in that IC_XD_OPSIZE context.
I’ve tried a couple different things to change this, but it seems to break things all over the place. I’m currently pretty much stymied by this, butI’m willing to put in effort to make this better if someone can help me point in the right direction.