Let’s take a look at the problem from this side: What is the purpose of external models? Why do we have them?
As far as I know, it’s about splitting code into small BUILD targets and keeping dependencies between BUILD targets small. Every op could just as well implement the interface directly instead of the external model. But that would introduce complex BUILD dependencies and may even result in cycles, meaning that we would sometimes have to merge multiple targets (e.g., two different dialects) into one big target.
To make external models as unsurprising for the user as possible, they should behave just as if the ops would implement their interfaces directly. Meaning: No external models registrations in
getDependentDialects or any other mentioning of external models in a pass (or dialect).
Unfortunately, external models must be registered somewhere. Why not get done with it during startup and then we can forget about them. Like a BUILD file.
We have always had two independent units of registrations: “dialect” and “pass”.
The natural extension to me would be to have 3 independent units of registrations: “dialect”, “pass” and “external model”. No changes would be necessary to dialects or passes.
You need to know about the dialects you will emit directly, and you need to know how to build a pass pipeline, nothing more.
You will also have to know about external models now. In particular that they have to be registered. But this is much simpler than passes or dialects. It’s like having to know that you must add
MLIRTensorTransforms to the
LINK_LIBS section of your
I still think during the setup of the MLIRContext would be a good place for registering external models. Along with the delayed registration mechanism that @ftynse described. As an idiom, every BUILD target that contains external models could have a public function that registers all the external models that it provides. Just like all those
register(DialectName)Passes functions, except that they’re not generated through
.td files but written manually.
Happy to do it in a different way, but it seems like nobody knows an alternative after a week of discussions.
Our current system has a high fidelity in terms of reproducer: given an an IR and a pass pipeline we should get consistent results.
I think this should not be affected.
mlir-opt should have all external models registered during startup. If you’re talking about a user of MLIR that is not using
mlir-opt, there is no guaranteed reproducibility anyway: When helping an external MLIR user, you may tell them to run pass
-my-pass, but they may not even have
-my-pass linked into their binary.