Does anyone know of any existing work towards an MPI or any SPMD dialect for MLIR? I’m interested in using MLIR for non-ML use cases in the HPC / data analytics realm. Functionality that I would be interested in: Automatic sharding of large memrefs onto a cluster of known rank, and loop splitting …

We have a rough version of an MPI dialect with some things implemented internally in our lab, it’s implemented in xDSL though: https://github.com/xdslproject/xdsl/blob/main/xdsl/dialects/mpi.py We lower it to llvm using some dark magic with constant values and such, let me know if you have any ques…

This looks very interesting. Can you explain the relation between XDSL and MLIR?

See the xDSL presentation at EuroLLVM a few months ago: 2023 EuroLLVM - Prototyping MLIR in Python - YouTube

I would be very interested in seeing this materialize in MLIR. @AntonLydike any plans to upstream some of your work?

Upstreaming this is currently in the discussion, but not immediately planned. We are working on a prototype to show that this is indeed something useful, which should hopefully be done soon. I’m sure I will have updates by the time the LLVM dev meeting comes around!

@mehdi_amini Maybe we can have an ODM on that in the near future? We have a deadline coming up in two weeks, but after that maybe?

Are you looking for something closely adhering to the MPI standard or something more high-level that works on tensors?

[image] AntonLydike: @mehdi_amini Maybe we can have an ODM on that in the near future? We have a deadline coming up in two weeks, but after that maybe? Ping me whenever you feel like you’re ready!

I’m actually not using MLIR for machine learning so I’d prefer something a bit lower-level.

We are also cooking up a dialect to represent exchanges of rectangular regions of memrefs, let me know if you are interested in that!

MPI / SPMD dialect?

MLIR

AntonLydike November 6, 2023, 8:23pm 21

The RFC for the MPI dialect is now up: [RFC] `MPI` Dialect

Topic		Replies	Views
[RFC] `MPI` Dialect MLIR rfc	45	3265	February 13, 2025
Existing spmd support in MLIR dialect MLIR	1	291	February 3, 2023
[RFC] Sharding Framework Design for Device Mesh MLIR	157	7997	January 8, 2026
[RFC] An MLIR Dialect for Distributed Heterogeneous Computing MLIR mlir	11	1553	July 7, 2025
Sparse Representation MLIR	46	5201	December 3, 2020

MPI / SPMD dialect?

Related topics