RFC building a target MCAsmParser

Hi everyone. We’re interested in contributing a Hexagon assembler to MC and we’re looking for comments on a good way to integrate the grammar in to the infrastructure.

We rely on having a robust assembler because we have a large base of developers that write in assembly due to low power requirements for mobile devices. We put in some C-like concepts to make the syntax easier and this design is fairly well received by users.

The following is a list of grammar snippets we’ve had trouble integrating in to the asm parser framework.

Instruction packets are optionally enclosed in braces.

{ r0 = add(r1, r2) r1 = add(r2, r0) }

Register can be the beginning of a statement. Register transfers have no mnemonic.

r0 = r1

Double registers have a colon in the middle which can look like a label

r1:0 = add(r3:2, r5:4)

Predicated variants for many instructions

if(p1) r0 = add(r1, r2)

Dense semantics for DSP applications. Complex multiply optionally shifting result left by 1 with optional rounding and optional saturation

r0 = cmpy(r1, r2):<<1:rnd:sat

Hardware loops ended by optional packet suffix

{ r0 = r1 }:endloop0:endloop1

We found the Hexagon grammar to be straight forward to implement using plain lex / parse but harder within the MCTargetAsmParser.

We were thinking a way to get the grammar to work would involve modifying tablegen and the main asm parser loop. We’d have to make tablegen break down each instructions in to a sequence of tokens and build a sorted matching table based on the set of these sequences. The matching loop would bisect this sorted list looking for a match. We think existing grammars would be unaffected; all existing instructions start with a mnemonic so their first token would be an identifier followed by the same sequence of tokens they currently have.

Let us know if we’re likely to run in to any issues making these changes or if there are other recommendations on what we could do. Thanks!

Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Frankly, the MC assembler infrastructure is pretty weak. I’ve spoken personally to some of the developers who wrote it, and they said they know it’s bad and suggested that it be rewritten. My experience from attempting to make MC parse Intel x86 asm confirms this.

Can you go into more detail what exactly you find difficult? X86 already
has the problem of integrating two quite different assembler variants.
Your grammar also seems to lack a number of things like how to specify
sections and similar meta data?


Hexagon assembly syntax doesn't have mnemonics in the same sense as other architectures. Take "add" as an example:

   r0 = add(r1, r2)
   r0 = add(r1, #2)
r1:0 = add(r3:2, r5:4)
r1:0 = add(r2, r5:4)

There is more:

   r0 = add(r1, add(r2, #5))
r1:0 = add(r3:2, r5:4):sat
   r0 += add(r1, #3)

predicated versions:

   if (!p0) r0 = add(r1, #4)


In the architecture definition they all have different tags, which we use in the .td files, but for the assembly programmer there is "add" stuck in various places in the instruction string. Most architectures use the syntax where the mnemonic comes first, followed by operands. Ours is very different.


Sure, the first immediate problem we run in to is statements not beginning
with mnemonics. The assembly syntax uses C-like assignments so most
statements start with a register token, then an equals token, and then the
mnemonic followed by the remaining tokens.

The next issue is with creating instruction bundles. We enclose a certain
number of instructions in curly braces to signify packet begin/end so this
doesn't fit in to the one line = one instruction rule.

The way to describe sections and metadata is defined though we didn't have
many problems so they're omitted.

I see the X86 target has a matcher for each syntax. Were there other things
in the grammar that were hard to integrate cleanly?

An assembler frontend for converting to MCInsts is a possibility. The parser for that would probably be generated using a third party parser generator. At that point we could just substitute use of the TableGen match tables for the generated library. Do you know if anyone’s considered including a parser lib like this before?

The TableGen changes I alluded to would get us close to parsing without hacks though it would still be deficient compared to a full parser. These changes would probably be the least amount of deviation from other targets depending on what everyone wants long-term.

One possibility for which we’d be interested in getting feedback is allowing a target to fully handle the parsing process.

We have a generated parser that can output other compiler IRs and this could be changed to output MCInsts. If we could get the input text stream and an output MC stream we could have a target specific way of doing all parsing. Perhaps this would be useful to other targets that have difficulty?

A parser generator isn’t distributed with the project so we could publish the parser generator input and output.

One possibility for which we'd be interested in getting feedback is allowing
a target to fully handle the parsing process.

How many of the problems that you have encountered come from the c++ code that
is generated by TableGen? If you ignore all the Tablegen'd code, does the
MCAssembler interface give you enough freedom to do what you want?