SPIR-V to LLVM dialect conversion: Decorations and other attributes semantics

Hi,

I am working on SPIR-V to LLVM conversion for GSoC. There are cases when SPIR-V cannot be converted into LLVM directly, or doesn’t have a semantic equivalent in LLVM. There already was a discussion on spv.ControlBarrier, so I think that it may be worth dedicating a separate thread for discussions of such cases. This post is about SPIR-V BuiltIns, Decorations, Loop/Selection controls and other additional information passed to ops.

In SPIR-V we can pass additional information to ops, specifying loop unrolling or branch flattening, for example:
spv.loop "Unroll" { ... }, spv.selection "Flatten" { ... }
or specifying additional semantics and constraints, for example:
%3 = spv.Variable built_in("GlobalInvocationID") : !spv.ptr<vector<3xi32>, Uniform>

For such cases I think of the following options at the moment:

  1. Implement using proper LLVM
    This options require scaling of LLVM Dialect to support this additional metadata. As an example, this is the case of dealing with some of the Loop Controls:
    Use LLVM’s loop metadata to ge from "Unroll" to !0 = !{!"llvm.loop.unroll.full"} to enforce loop unrolling. However, there are cases that do not exist in LLVM. For instance, "IterationMultiple" that is an unchecked assertion that the loop will execute a multiple of a given number of iterations.

  2. This leads no next option - simply ignore those. This is a possible solution for GSoC, but I am wondering if anyone has some thoughts on actually implementing those?

  3. If implementing, we may extend LLVM dialect to support these. For me this is the least preferable option since it creates unnecessary complexity within LLVM dialect. It is not an op, and I suppose it is not as crucial to add it to LLVM dialect as some specific SPIR-V metadata.

Looking forward to hearing any opinions on this!

Thanks,

George

1 Like

Thanks George for starting this! I suspect we might want to spin out more discussions to their own threads if it is likely to be involved. But at the moment having one dedicated thread for decorations/enums is fine.

Also most of these issues should be discussed on a per-case basis as we might have direct mapping or solution for one but not another. Covering all the aspects of the SPIR-V spec is a huge amount of work and I won’t expect to happen within the scope of a GSOC project. But I guess a nice delivery is an analysis of which decoration/enums kind falls into which category. Largely, I see a few buckets for decorations here:

  • Decorations specifying the layout. Examples like ArrayStride, MatrixStride. For these we just need to get the ABI correct and potentially inserting paddings inside LLVM structs.
  • Decorations affecting numeric calculation. Examples like NoSignedWrap, RelaxedPrecision, FPRoundingMode. For such cases, we might be able to map to their LLVM counterparts. (For example, just quickly skimmed it seems we can use nsw for NoSignedWrap and Constrained Floating-Point Intrinsics for FPRoundingMode.) There are lots of details here though. If there is no direct mapping, we might be able to emit additional LLVM code to emulate the semantics.
  • Decorations regarding graphics intrinsics. Examples like Patch and Centroid. Let’s focus on compute side and not worry about these for now.
  • Decorations regarding runtime and kernel ABIs. Again just need to have a convention and decide on an ABI.

Builtins are a special kind of decoration. By builtin, it means basically they are something special to GPU. :stuck_out_tongue_winking_eye: So we might want to introduce intrinsics in LLVM and then plumbing through the stack to properly support. But again case by case.

For enum operands to ops, there are also a few categories I think:

  • Enums affecting op semantics. This is pretty much case by case. For example, pointers will have storage class enum operand. I think that largely maps to memory spaces into LLVM.
  • Enums that are just hints to GPU driver compilers. the Unroll, Flatten and other Selection/Loop/Function control enums are here. What we can do is to map to LLVM counterparts if exists, otherwise we can actually invoke an LLVM pass for performing the task like inlining, etc. That’s what’s those enum operands are intended for originally anyway. We can think of the SPIR-V to LLVM conversion as the compilation happening inside GPU driver compiler.

I think by following this, we can put decorations/enums into their buckets and then we can think of proper solutions for each bucket or discuss on a per-case bases. George, how do you think about this?

Disclaimer: I have not followed the original GSoC proposal discussion due to the lack of time.

A couple of quick comments.

Ultimately, the LLVM dialect needs to support everything that LLVM IR does. This is a manual process (except for intrinsics) that is currently need-driven. You are very welcome to reflect any part of LLVM IR in the dialect if there will be code that is exercising it. Big chunks of IR may require some design work that deserves an RFC, in particular MLIR offers significantly more structure in attributes than LLVM IR does in metadata.

For anything that pertains to loops or other control flow, I would encourage you to look into the SCF dialect instead and use the existing lowerings to the LLVM dialect, instead of re-inventing loop-to-cfg lowering from scratch. At that level, it is also possible to actually perform the transformations like unrolling directly. If you choose to go to the LLVM directly, I would expect to see some code reuse between the existing control flow lowering and the new code.

I will be strongly opposed to adding anything to the LLVM dialect that does not have an LLVM IR counterpart, or is otherwise necessary to model LLVM IR (like we do for globals). If one really needs that, it can live in a separate dialect that uses LLVM’s type system.

For that, you’d need LLVM IR not LLVM dialect.

I like the bucket approach. I think it’s a good idea to tackle each category separately, or convert decorations case-by-case.

I am wondering how it’s possible to do that given @ftynse’s comment that the pass is invoked on LLVM, not LLVM dialect? My guess we can lower to actual LLVM, invoke pass on those ops, and then translate back to the dialect? This seems a bit cumbersome for me. I suppose actual LLVM is targeted anyway, so the pass can be invoked at the final stage of translation?

Thank you! I will have a look at this one!

Good point. Having the conversion from SPIR-V’s structural control flow ops to scf actually can be a good reusable component that might be suitable for other tasks too. With the existing scf to llvm path, it can connect the dots. But one potential mismatch I can see here is that structural control flow ops in spv dialect follows SPIR-V spec, which mainly serves as shading language compilation target. So for example, spv.loop can be generated from either a C-like for or while or do-while statement. To properly convert a spv.loop, it means there are cases we need to have while or equivalent ops in scf given there is not always a clear induction variable which can let us map to scf.for. Going from spv.loop to these ops may also mean some non-trivial analysis. Not impossible (“just” more ops :stuck_out_tongue_winking_eye: ) as demonstrated by various projects that reverse SPIR-V back to source code. This is a path worth exploring. We aren’t aiming to support all (corner) cases in one batch anyway; supporting some common cases can already carry us a long way for real-world usage I guess. But OTOH, I’d imagine directly going to llvm will be more direct and easier. I hope that can be a fallback path we can always rely on. Massaging code to share more sounds good to me!

Ah yes. Sorry the boundary between LLVM IR and LLVM dialect got blurred in a sec for me. Best would be to find LLVM IR equivalent construct and add support in LLVM dialect if missing. Otherwise I think Alex’s scf suggestion in the above is something worth exploring. One thing also worth to keep in mind is that these decorations are just requests/hints to driver compiler. They are not enforced requirements. So technically if we choose to ignore them, the validity should still hold.

Suggestion: I think, it would be nice to have some sort of document or a table describing mapping of constructs from SPIR-V dialect to LLVM dialect. It could be just examples of SPIR-V dialect code mapped to its representation in LLVM dialect. IMO it would serve two purposes:

  1. Summarize what is already done (in scope of the GSoC project)
  2. Facilitate discussions for non-trivial cases
2 Likes

It is a good point, thanks! Actually I started working on one just recently :slightly_smiling_face:. I think this will land in a couple of days. It describes type and ops conversion, what is supported so far and future development (this can be linked to non-trivial cases).

Yes indeed. @ftynse also pointed out something similar in code reviews. It would be nice to have such a doc for sure!