Assembly Dialects Roundtable

MLIR
1

Organisers: Renato Golin, Nicolas Vasilache, Sasha Lopoukhine

At EuroLLVM 2026, Renato, Nicolas, and I (Sasha) organized a roundtable aimed at structuring the assembly and ISA level efforts that are gaining traction in the MLIR ecosystem.
Our objective was to characterize the different IR encodings and their tradeoffs, and to gather input from the community as to the suitability of the different encoding choices across use cases.
The number of assembly-level MLIR projects that we gathered in preparation of this roundtable (see below) and the high attendance of the roundtable itself (40-50 people) suggests a broad interest in the community.
We identified several encoding choices across projects, for example, whether SSA def-use chains are maintained.
During the roundtable, I (Sasha) presented the encoding choices for my work on RISC-V and x86 dialects and also summarized the work of Renato and Nicolas (who were partially absent due to scheduling conflicts).
One particularly interesting tradeoff are encoding differences between my work, the upstream x86 dialect, and Aster.
Over the next few months, Nicolas, Renato, and I plan to iterate on the designs, aiming to eventually submit an RFC to gather community input on ISA-level abstractions and their use in MLIR.

Assembly-level Representations

Here are the projects that I’m aware of that have assembly-level concepts like registers in MLIR:

If you’re aware of other projects with such representations I’d be interesting in hearing about them!

Roundtable discussions

I presented a trivial example of how one could represent assembly-level operations in MLIR, to serve as a reference for the discussion:

// Input
%a = arith.addi %b, %c : index
%d = arith.addi %b, %a : index

// Lowered
%a = riscv.add %b, %c : (!riscv.reg, !riscv.reg) -> !riscv.reg
%d = riscv.add %b, %a : (!riscv.reg, !riscv.reg) -> !riscv.reg

// Allocated
%a = riscv.add %b, %c : (!riscv.reg<a0>, !riscv.reg<a1>) -> !riscv.reg<a1>
%d = riscv.add %b, %a : (!riscv.reg<a0>, !riscv.reg<a1>) -> !riscv.reg<a0>

// asm
add a1, a0, a1
add a0, a0, a1

This is the representation that I have been using in my work over the last few years. Importantly, this is not the same structure as the one used in Aster, where the destination register is passed in, similarly to Destination-Passing Style (DPS) in linalg. Another key distinction is that we keep the SSA relationships for the register-allocated form, whereas Aster destructs the SSA relationships and represents register reads and writes as effects.

I presented the register allocation approach that we currently use in our backend dialects. We discussed the possibility of multiple register allocation approaches, and how the encoding above can is unusual and may lead to some confusion. One of the things we discussed was the ability to verify whether the code above in register allocated form is correct, by looking at the registers stored in the types. We did not reach a consensus on the best register allocation approach, but various people where interested in exploring good register allocation for ISA-level IRs further.

My impression of the discussion was that people generally found ISA abstractions in MLIR weird but interesting, which I’m pretty happy with for this still relatively early stage. Nicolas and I are planning to do more experimentation on the different representations over the next couple of months, and hopefully can come to an agreement and a better understanding of the tradeoffs soon. We’re also planning experiment some more with Renato’s team on the tradeoffs between the existing x86 dialect ops, which operate on vectors, and the ones I’ve been working on, which operate on vector registers. I look forward to writing these up and getting more feedback from the community!

2

Nice that there was so much interest in the roundtable!

The reason we (ASTER) destroy the SSA chain is because !risc.reg<a0> should be treated like a reference to memory, as it doesn’t holds any real value. That is:

%a = riscv.add %b, %c : (!riscv.reg<a0>, !riscv.reg<a1>) -> !riscv.reg<a1>
// ...
%b = riscv.add ... -> !riscv.reg<a1>

If someone uses %a after %b is defined, %a doesn’t really represents the value of the first addition. So better to destroy the chain to avoid any potential weird issues that are hard to debug (at some point we didn’t destroy the chain and encountered some of those issues).

3

One thing we discussed is to create a copy of the register into a new ssa value if the value has more than one user and the earlier one writes to it.

It’s not clear the impact on IR bloat or difficulty in analyses, but the feel was that it would not be much worse than carrying it on DPS.

I guess we need more data on the trade offs, but for CPU, Sasha has been successful in his approach, which may be different for GPU.

4

I think it’s not a purely CPU/GPU reason, but rather going from purely high-level into low-level ASM (Sasha’s approach), vs a mixed approach like we have in ASTER (both high-level ops and ASM ops coexisting).

When you go from high-level and lower to ASM, it’s almost impossible that you get issues like the ones I mentioned, either the original SSA value chain or the allocator will take care of making such issues impossible.

But if you allow mixing things, and allow writing kernels in this mixed mode, then it’s not impossible to introduce such issues, specially if you have pre-allocated registers mixed with non-allocated regs. For example, in AMD GPUs when the kernel gets launched register v0 gets init with the thread ID, so we always get the possibility of this issue.

5

Some years ago (before I started paying attention) there was a project that attempted to hoist MIR to MLIR which was actually presented at dev conf:

Moving LLVM’s code generator to MLIR framework

There’s some discussion there about dealing with representing registers. FWIW I tried getting into contact with those folks (in order to maybe resurrect the project) but wasn’t successful. Maybe someone here knows someone somewhere :person_shrugging:.

6

Yep, I think this point will be very interesting to discuss further. I’m not 100% convinced either way, as I’ve had to debug my share of such issues, but my current feeling is that having the SSA relationships around would help debug such issues, whereas destroying the SSA relationships will mean that similar issues of clobbering registers being hard to debug. Time permitting, I’d like to write such a clobber detection pass soon, as a debug step in our lowering.

7

Oh yeah I vaguely remember this! I think that, once we have some level of agreement about how instructions should be represented, it would be great to automate the definitions from existing LLVM infrastructure. I’ll try to get in touch with those talk authors.

8

The issue is, if you allow users writing low-level allocated code (this is the main issue), then how do you determine intent?

For example:

%a = riscv.add %b, %c : (!riscv.reg<a0>, !riscv.reg<a1>) -> !riscv.reg<a1>
// ...
%b = riscv.add %d, %e : (!riscv.reg<a3>, !riscv.reg<a4>) -> !riscv.reg<a1>
// Point x

What does it mean using %a or %b at point x, are they referring to the same register or to a value? The type seems to suggest that they are referring to the same concrete register, and if that’s the case if some one uses %a is it really a clobber? Shouldn’t those be considered almost like ptr aliases?

What we found is that from a programming model point of view things become too messy if we kept the SSA chain, so the cleanest most robust solution was to make it clear that there’s no values flowing. Instead, one is writing or reading from concrete registers with effects.

I’ll also say, that in our case destroying the SSA chain is amongst the last steps (right before register coloring), and all remaining passes seem to work just fine. Further, using reaching definitions solve most of the gaps.

9

I’d love if that were to be possible (using the defs from LLVM and translate them into MLIR). I’ve taken a look at this multiple times, and the conclusion has always been, the current structure of LLVM TD instructions are too skewed towards the LLVM backed Impl to have any real utility in MLIR. Disclaimer, I’ve only looked at AMDGPU td defs.

With this I mean, one gets too many defs for the same semantic instruction but with small variations that we would typically abstract away in MLIR (the issue here are multiclass defs).
For example, in AMDGPUs you have VGPRs and AGPRs registers, both can be used for MFMA instructions, and TD will produce 2 inst variants one for VGPRs and one for AGPRs (they have slightly different encodings).
In MLIR we would say, those are the same op, and have the verifier and type constraints deal with any of the nuances.

10

To me it feels like at point x reading %a is a bug, but I agree that keeping things straight here is a pain, and the IR definitely gets quite dense. It’ll be good to discuss the tradeoffs with more examples soon, I’ll have much more to show in a month or two. There are probably other options also worth considering.

11

Yep, here I think we’ve opted for designs that are quite different between the xDSL x86 dialect and Aster, we have a bunch of variants and I’m not yet sure that it’s really a good idea for the long term, but for the small number of ops one needs for matmul it’s not too bad. Another important aspect to discuss soon!

12

This is an interesting point. I think there’s a nuance that we need to consider.

When writing assembly by hand, one doesn’t always select input/output/clobber registers for a particular reason, just because they’re available. There are cases, though, where that’s not true:

  1. When the register has a particular purpose (PC, SP, ACC, etc).
  2. When the register is an ABI register (args, rets, unwind).
  3. When the programmer “knows a trick” that the compiler doesn’t.

I’d argue that for the first two cases, compilers should “do the right thing” regardless of user intention. But I can see how, once lowered to asm, this can be a problem (e.g. mov r9, pc). This could also be solved at raising, i.e. convert to an ABI/detail operation stream that does what the programmer intended. However, I’d argue this is not “interesting”.

The third case is the interesting one. In some cases, programmer tricks are inaccessible to the compiler, in which case, you must preserve the semantics. In other cases, the trick is old and unnecessary, and compilers have become smarter since then, and can find better code than that.

When coming from high-leve languages, we assume “the compiler knows best, unless the user writes intrinsics/inline-asm”. Here, there’s only asm. How do you know?

This is the key point: