I did have a bit deeper look at your revision and I think I got a first impression. Is there a recursive type wrapper around every type? At the first glance it looked like it but I probably missed something…
I think compared to the mutable attribute solution this is definitely a better fit with respect to the MLIR core infra and the code complexity is also on the low end (except for the recursion detection / handling).
The downside is probably that it is somewhat unstructured/“lazy” and certain things cannot be verified. I.e., we probably cannot verify there is a rec_struct for a rec_self? A mutable attribute would be a more concise representation that provides some more guarantees. As most transformations will not touch this metadata, it maybe ok we cannot verify everything though. The import/export and the front-end that have to create/consume this metadata probably can verify the correctness during the traversal of the metadata.
I suppose this is to be expected with this kind of “lazy” representation of recursion, since mutually recursive types would naturally need two representations of the same cycle anyway: A → B → A, and B → A → B.
The helper function on recursive typesgetUnfoldedBaseTypeshould help deal with this in code. It takes the inner type of a recursive type and unfolds it one level, so that we get back a self-contained “outside” version of the body of the cycle.
Yeah I think helper functions such as getUnfoldedBaseType are a helpful tool here. As written above, I would hope this metadata is mainly routed through and rarely transformed. Hence, the fact that manipulating it can go wrong is less of a concern.
I think it makes sense to continue and try to implement at least the export to understand if there are other hurdles/concerns.