How are folks thinking about creating C bindings for specific operation types that in C++ would be separate types?
The current C bindings have separate functions for dealing with modules, the benefit of which seems to be that the resulting Operation ends up being a ModuleOp. Is it the intention that we create similar functions for every Operation in a dialect we want to expose via the bindings (which would likely require a TableGen pass for creating both C bindings and host language bindings for those operations)? Or is the intention that we keep things stringly-typed in the host bindings (which would be equivalent to a module being created as a generic operation with the operation name “module”)?
In the former world, it feels to me like mlirOperationCreate shouldn’t even be public API (or at least be explicitly reserved for corner cases) and I the latter world milrModuleCreateEmpty should be marked as explicitly a convenience for mlirOperationCreate where the operation name is “module”.
This post seems to touch on this issue for attributes, though I’m not sure what the implication for operations is.
The tricky part is the “stability” aspect: right now the C API has been presented as something with what we believe has a reasonable amount of stability moving forward.
Another angle, is that the C bindings are really meant to support FFI to bind other languages to MLIR, so while it is possible to emit C APIs for individual dialect, who is the intended end-user for these entry points?
In the case that the bindings are being used to output a dialect declared in C++ (or even another language via bindings), stability doesn’t really mean much because in the event that that the dialect semantics change in an unexpected way, you will eventually get an error. The question is whether that error will happen at compile time (because the generated bindings code has changed) or at parse time (because the stringly-typed MLIR you are trying to produce is wrong).
In the case that you are both declaring a dialect and emitting MLIR using that dialect in your host language, I can see how having stringly-typed primitives might be enough.
In my case, I’m hoping to emit MLIR from Swift using a dialect declared in C++, so I’m leaning in the direction of using TableGen to generate just enough Swift to basic type checking and autocompletion to function as expected.
Op-specific builders generalize poorly at the C level: one essentially needs a function per every builder for every op, as well as a type translation (e.g. what’s the C equivalent of llvm::DenseMap?) and a name mangling scheme (no overloading in C). In addition, the bindings will need to be updated every time an op or a builder signature of an op is modified. So I don’t see the generic version going away. It’s an equivalent of Operation::create in C++, and that is not going away either.
ModuleOp is a privileged class: it is known top-level op that parser produces and it has ownership-related wrappers other ops do not, hence special the APIs for it.
Stricter compile-time “validity” checking is mostly an illusion. Yes, you cannot pass a StringAttr where an IntegerAttr is expected. But there are further IR validity checks that are not represented at the C++ type level, e.g. the integer attribute having a positive value, that will still cause verification failure at runtime. Also, since builder bodies can be arbitrary, they can do arbitrarily bad things to the IR. While I am not discarding the value of types in builder signatures, they are not the main validity mechanism for ops.
With all that in mind, I think the simplest way forward is to autogenerate the default builders for ODS-defined ops in the host language that target the generic C API. Custom builders will likely require the translation and mangling schemes, so I would tend to keep them private to specific bindings.
Ironically, that post was attempting to extend the thinking we had for operations to attributes and types, which are presently not generic to the same extent. However, the discussion about thinking on operations was spread over many threads and is likely hard to reconstruct if coming in after.
In short, the C APIs aim to be a stable, dialect agnostic mechanism for manipulating the IR. This level of API exists in the c++ side too but it’s less explicitly differentiated.
The thinking is that “user APIs” would include tablegen integration that would generate language specific op view and builder code for dialects, calling whatever low level API was reflected for the c API. This approach is furthest along on the python side, where we generate per-dialect python wrapper code.
It could be argued that in this respect, C could be a viable target for a user API and we could use tablegen to produce builders and accessors for it; however, I think this would be different from the core MLIR C API that exists now (and would be built on top of it). It could even be of the form of generating private headers/impls for your project if you need them. I don’t believe anyone has asked for this yet, and I am dubious about whether it is a good entryoint for generating a type safe higher level language (like swift) binding: you’re probably going to get a more usable high level, generated API by just generating it for your language vs trying to squeeze such concepts into a C API and then trying to reflect them further up in a usable, type safe way.
I wouldn’t be opposed to exploring high level C wrappers if someone thinks it useful, but I would like that to be a separate thing from the core C API.
I think the right way to go here is to have TableGen produce dialect-specific Swift wrappers… that call into the low-level C API. I don’t think there is a need to have dialect-specific Swift wrappers calling into Dialect-specific C wrappers.
Like I mentioned above, Module is special in C++ because it has additional ownership properties materialized as OwningModuleRef class and co. These are required to properly interface with the parser and so on.
Sorry, I missed that bit in your last answer. Could you elaborate on what these additional ownership properties are? I’m having trouble wrapping my head around the hierarchy since it seems what your saying is that there are primitives (Operation, Region, Block), Modules (which are not primitives but special?), and everything else. Is there something that an operation in the “everything else” bucket cannot do, but modules can?