I took a look at all the OPC_SwitchOpcode operators in the matching tables. Almost every target starts with a large one, which is now cached by the interpreter. Instead, TableGen could build the opcode -> offset table.
But there are also many other opcode switches with between 10 and 70 cases. To speed these up, we could embed a sorted opcode -> offset table right in the matcher code. Or, if that seems like overkill, we could quite easily sort the opcodes and embed the opcode and offset of the middle one. A quick comparison would start the search in the correct half of the opcode sequence, halving the search time.