[RFC] Add GEN dialect for Intel GPUs


The MLIR ecosystem contains the NVVM dialect which extends the LLVM dialect to provide operations useful for programming NVIDIA GPUs. Similarly the ROCDL dialect extends the LLVM dialect and provides operations for programming AMD GPUs.

In order to provide similar functionality for programing Intel GPUs, we propose the addition of a new LLVM target dialect (GEN) to act as a counterpart to the NVVM and ROCDL dialects. The GEN dialect will provide operations for exposing to the MLIR ecosystem selected Xe ISA (codename GEN) assembly instructions. Hierarchically, the GEN dialect sits below the XeGPU dialect and it is our intention to supports lowering from the latter to the former where it makes sense.

Initially the GEN dialect will contain operations to:

  • query GPU properties such as threads ID, block ID, block and grid dimensions, etc…
  • emit barrier and group shuffle operations
  • emit instructions useful to access systolic array HW for matrix operations


The GEN dialect has been implemented in a branch in the Intel LLVM monorepo.
Below is an example of how an operation in the GEN dialect looks like:

def GENX_BarrierOp : GENX_Op<"barrier"> {
let summary = "Workgroup barrier";

string baseDescription = [{
  The `genx.barrier` operation performs a workgroup barrier and ensures all outstanding memory
  transaction using local or global memory are complete.

string llvmBuilder = [{
  llvm::Type *retType = builder.getVoidTy();
  llvm::Type *argType = builder.getInt32Ty();
  llvm::Value *arg = llvm::ConstantInt::get(argType, 3 /*memfence*/);
  createDeviceFunctionCall(builder, "_Z7barrierj", retType, {argType}, {arg});

let assemblyFormat = "attr-dict";

This new dialect fits well with ROCDL and NVVM dialects. It’s good to see MLIR expanding its capabilities to Intel GPU direction.

I’m not familiar with the compilation flow. Could you explain what this dialect lowers down to? For example, the NVVM dialect compiles to LLVM’s NVPTX backend, allowing us to test the generated LLVM, (even PTX), and run end-to-end tests. Adding this info into RFC would be helpful.

1 Like

Do you mind amending the proposal in the same way as the recent dialect proposals? See [RFC] Add XeGPU dialect for Intel GPUs - #18 by mehdi_amini



It’s great to see this proposal. It would be important to describe though what the path to execution is from this dialect.

Note that when the LLVM/NVVM dialects were added to MLIR, there was a path to lower those ops to LLVM IR (since the latter already had the NVVM intrinsics to map to) and compile to executable code.

1 Like

Currently, the Intel GPU backend accepts SPIRV as input. The GEN dialect is a LLVM target dialect therefore we can use a translator (https://github.com/KhronosGroup/SPIRV-LLVM-Trans) to convert the generated LLVM IR to SPIRV. Another potential option is to use the SPIR-V backend in LLVM.

1 Like

Hi, I think this reply [RFC] Add GEN dialect for Intel GPUs - #5 by etiotto addresses your question.

OK I will take a look at the new format and update the RFC.

Are both options functionally equivalent? If not, are there differences w.r.t coverage or performance? When you refer to SPIR-V above, is it the IR or the binary format? (The link is broken - missing trailing text ‘lator’.)

We have experimented with the Khronos LLVM-SPIRV translator and it works well. We haven’t experimented with SPIR-V backend, although in principle it should work. When I refer to SPIR-V in this context I meant the binary format.

Sorry about the broken link, the correct link is to the Khronos public LLVM-SPIRV translator: KhronosGroup/SPIRV-LLVM-Translator: A tool and a library for bi-directional translation between SPIR-V and LLVM IR (github.com)

Question] What is the overall goal of the dialect?

[Answer] The GEN dialect is aimed at extending the LLVM dialect with operations for exposing to the MLIR ecosystem selected Intel Xe ISA (codename GEN) assembly instructions such as the instruction used to do matrix multiplication with accumulation (i.e. DPAS).

[Question] What is the first implementation milestone?

[Answer] The first implementation milestone aims at providing operations to:

  • query GPU properties such as threads ID, block ID, block and grid dimensions, etc…
  • emit barrier and group shuffle operations
  • emit instructions useful to access systolic array HW for matrix operations

[Question] How does it fit into the MLIR dialect ecosystem?

[Answer] The MLIR ecosystem contains the NVVM dialect, which extends the LLVM dialect to provide operations useful for programming NVIDIA GPUs, and the ROCDL dialect which extends the LLVM dialect in a similar way and provides operations for programming AMD GPUs. The GEN dialect is positioned at the same hierarchical level as the NVVM and ROCDL dialects and provide similar functionality for programing Intel GPUs.

[Question] Connection: how does it connect to the existing dialects in a compilation pipeline(s)?

[Answer] The GEN dialect complements the LLVM dialect to provide the ability to generate instructions for accessing Intel GPUs (just like NVVM/ROCDL dialects extend the LLVM dialect to provide access to instructions for NVIDIA/AMD GPUs).

Higher level dialects, such as the GPU dialect, can lower certain operations to the GEN dialect. For example, the GPU dialect can convert gpu::ThreadIdOp to GEN::TheadIdOp.

Also, we envision that the XeGPU dialect(link) will implement a lowering path to the GEN dialect.

[Question] Consolidation: is there already a dialect with a similar goal or matching abstractions; if so, can it be improved instead of adding a new one?

[Answer] There is no other dialect in MLIR fitting this goal.

[Question] Reuse: how does it generalize to similar but slightly different use cases?

[Answer] Over time the GEN dialect can be expanded to include more operations exposing Xe ISA instructions.

[Question] What is the community of users that it is serving?

[Answer] The GEN dialect serves open source communities who aspire to build high-performance applications using the MLIR infrastructure on Intel GPUs.

[Question] Who are the future contributors/maintainers beyond those who propose the dialect?

[Answer] Intel engineers would mainly maintain it. For example whitneywhtsang (Whitney Tsang) (github.com), pengtu (Peng Tu) (github.com) and others.

1 Like

Thanks. So, it looks like if these dialects are in the MLIR repo, it would be feasible to set up execution tests for the Intel GPU if the system has the SPIRV-LLVM-Translator. AFAICS, JIT execution (via mlir-cpu-runner) may not be possible (if the translator is used to go from LLVM to SPIRV), but an AOT test could be set up and execution verified. Is that correct?

Are there similar tests for NV/AMD dialects to mimic from?

There are numerous NVGPU and AMD GPU execution tests – they are JITted and executed via the ORC JIT seamlessly. However, that approach won’t work if using the external SPIRV-LLVM translator, as I mentioned above.

I meant AOT ones.

Quick PSA, because I think the change was introduced without much publicity:

We already have some form of GPU compilation through SPIR-V available on mlir-cpu-runner. See this test:

However, as with NVIDIA and AMD, external tooling is required to fully execute them. I can confirm they work on ALCF Aurora’s Intel GPUs.

They work by serializing the GPU SPIR-V module, and then at runtime compiling it.

CCing the author: @silee2

1 Like

Is this using the LLVM SPIRV backend? That’s not the path we were discussing - see [RFC] Add GEN dialect for Intel GPUs - #9 by etiotto If a binary is created using an LLVM backend, it would fit the mlir-cpu-runner ORC JIT approach.

No. It’s going directly to SPIR-V.

However, from what I can tell from the RFC (maybe @etiotto can correct me), the idea is always ending up in a serialized SPIR-V binary.

Hence, in all cases we would be able to execute tests with mlir-cpu-runner. Because we already have the capacity of executing SPIR-V binaries.

In the same way ORC-JIT doesn’t generate the final CUDA binary, ORC-JIT doesn’t have to generate executable code for Intel GPUs, that’s the job of Intel’s SPIR-V backend.

1 Like

I’m in general pro this as it seems to form a useful component at the same level of backend specialization as existing ones.

Re testing: currently it’s true that folks have to download separate libraries to actually run on HW. As long as the testing is sufficient to ensure working and would not impede other upstream development (e.g., action at distance effects causing rollbacks). I’d be happy to see this and have spot for further growth. Mindful of course about duplicated functionality/aligned goal to ensure reduction of those.


That is correct.

1 Like

We plan to test that the GEN dialect operations are lowered correctly by using lit tests.