Generating dialect specific builder from tablegen

MLIR
1

Hi, i have a DSL written on top of mlir. When refactoring the operations definitions i have then to fix the various points where they get built.

Since clangd can’t get figure out the correct overload available for IRBuilder::create this means that I have to try to compile the project, get hit by errors that look like

home/massimo/rlc-infrastructure/llvm-install-debug/include/mlir/IR/Builders.h:512:5: error: no matching function for call to 'build'
  512 |     OpTy::build(*this, state, std::forward<Args>(args)...);
      |     ^~~~~~~~~~~
/home/massimo/rlc-infrastructure/llvm-install-debug/include/mlir/IR/PatternMatch.h:537:18: note: in instantiation of function template specialization 'mlir::OpBuilder::create<mlir::rlc::ClassDeclaration, mlir::rlc::ClassType &, llvm::StringRef, mlir::ArrayAttr, mlir::ArrayAttr>' requested here
  537 |     auto newOp = create<OpTy>(op->getLoc(), std::forward<Args>(args)...);
      |                  ^
/home/massimo/rlc-infrastructure/rlc/lib/dialect/src/TypeCheck.cpp:306:18: note: in instantiation of function template specialization 'mlir::RewriterBase::replaceOpWithNewOp<mlir::rlc::ClassDeclaration, mlir::rlc::ClassType &, llvm::StringRef, mlir::ArrayAttr, mlir::ArrayAttr>' requested here
  306 |         decl = rewriter.replaceOpWithNewOp<mlir::rlc::ClassDeclaration>(
      |                         ^
/home/massimo/rlc-infrastructure/rlc-debug/lib/dialect/include/rlc/dialect/Operations.inc:5181:15: note: candidate function not viable: no known conversion from 'mlir::rlc::ClassType' to 'llvm::StringRef' for 3rd argument
 5181 |   static void build(::mlir::OpBuilder &odsBuilder, ::mlir::OperationState &odsState, llvm::StringRef name, mlir::rlc::ClassFieldAttr members, mlir::TypeRange template_parameters, mlir::rlc::SourceRangeAttr type_location);
      |               ^                                                                      ~~~~~~~~~~~~~~~~~~~~
/home/massimo/rlc-infrastructure/rlc-debug/lib/dialect/include/rlc/dialect/Operations.inc:5186:15: note: candidate function not viable: requires at most 5 arguments, but 6 were provided
 5186 |   static void build(::mlir::OpBuilder &, ::mlir::OperationState &odsState, ::mlir::TypeRange resultTypes, ::mlir::ValueRange operands, ::llvm::ArrayRef<::mlir::NamedAttribute> attributes = {});
      |               ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/massimo/rlc-infrastructure/rlc-debug/lib/dialect/include/rlc/dialect/Operations.inc:5182:15: note: candidate function not viable: requires 7 arguments, but 6 were provided
 5182 |   static void build(::mlir::OpBuilder &odsBuilder, ::mlir::OperationState &odsState, ::mlir::Type result, ::mlir::StringAttr name, ::mlir::ArrayAttr members, ::mlir::ArrayAttr template_parameters, /*optional*/::mlir::rlc::SourceRangeAttr type_location);
      |               ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/massimo/rlc-infrastructure/rlc-debug/lib/dialect/include/rlc/dialect/Operations.inc:5183:15: note: candidate function not viable: requires 7 arguments, but 6 were provided
 5183 |   static void build(::mlir::OpBuilder &odsBuilder, ::mlir::OperationState &odsState, ::mlir::TypeRange resultTypes, ::mlir::StringAttr name, ::mlir::ArrayAttr members, ::mlir::ArrayAttr template_parameters, /*optional*/::mlir::rlc::SourceRangeAttr type_location);
      |               ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/massimo/rlc-infrastructure/rlc-debug/lib/dialect/include/rlc/dialect/Operations.inc:5184:15: note: candidate function not viable: requires 7 arguments, but 6 were provided
 5184 |   static void build(::mlir::OpBuilder &odsBuilder, ::mlir::OperationState &odsState, ::mlir::Type result, ::llvm::StringRef name, ::mlir::ArrayAttr members, ::mlir::ArrayAttr template_parameters, /*optional*/::mlir::rlc::SourceRangeAttr type_location);
      |               ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/massimo/rlc-infrastructure/rlc-debug/lib/dialect/include/rlc/dialect/Operations.inc:5185:15: note: candidate function not viable: requires 7 arguments, but 6 were provided
 5185 |   static void build(::mlir::OpBuilder &odsBuilder, ::mlir::OperationState &odsState, ::mlir::TypeRange resultTypes, ::llvm::StringRef name, ::mlir::ArrayAttr members, ::mlir::ArrayAttr template_parameters, /*optional*/::mlir::rlc::SourceRangeAttr type_location);
      |               ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

then i have to understand what is the location of the real error, go there, figure out why a parameter is not the right one, and keep doing this for the entire project.

This issue would be entirely solved if it would be possible to generate a IRBuilder specific to my dialect that knows about the operations and their build overload and hides the templated invocation. This would probably cut down at least 50% of the time it takes me to change the operation definitions.

i understand there was a proposal to do something in this realm [RFC] DSL-style / dialect-specific builders , but i don’t think it went anywhere.

Is there any plan to address this issue?

2

There was quite strong opposition to that idea being contributed, so I haven’t pursued it any further. It isn’t that difficult to implement locally though.

The “workaround” to use with upstream MLIR is to type OpName::build( to trigger autocomplete, fill in all the arguments, and then rewrite it to builder.create.

3

There was some work to enable seeing through these methods as one would through make_unique clangd side, but I think that got delayed (this was something adjacent to MLIR, but we were very eager to be test subject of such extension).

In this case, the 3rd location reported is where a change is needed it would seem. Now, recalling which builders are available does require either inspection or defining some explicitly. The latter part does allow you to define one that is more stable (if feasible).

The other option is via clang matchers, it wouldn’t be that difficult to craft one to do a rewrite, or even just dump all the locations where the build is invoked. One can then integrate these with you cmake build and you end up with a list of locations/files to change. Now that requires knowing matchers :wink:

4

i was aware of all the suggested tricks, which i sometime use, but none of those really cuts down significantly the “mean time to change a operation field”, unless it is a very sprawling operation in the codebase, which would then warrant to use a clang rewriter.

In principle i could make a standard rewriter that has methods to add, remove and change fields of create and use that, but since it is a academic project and we have some students working on it here and there, i don’t really wish to have to explain to them what matchers are. As ftynse suggests, In that case i think i would just roll out a a tablegen backend and keep it inside the project.

is there a know link to where the clangd feature is being developed?

5

It’s been a few years since it was discussed so I don’t have it off hand. Will try to track down/ask folks that were involved.

Alex’s OpBuilder trick is nice, I do the more direct (look at the TD file, right click on class being created and look at build methods or look at generated docs). The builder route makes sense in a closed domain (where there is only one thing of each) and if one can avoid the C++ DSL foot guns