Hi all, my question is why we need a PDL pattern instead of a c++ match-and-rewrites? What are the main benefits of using a PDL pattern?
A PDL pattern can be loaded at runtime and can be separated from your main project. Meaning you don’t need to recompile your code if you change your pattern. This might be useful if you often change your patterns for example. If would also recommend PDLL which is a DSL to write those patterns.
Thank you for your reply! I cannot find an example of how to use pdl pattern at runtime. Does this mean it can be compiled into a shared library and then dlopen it at runtime?
I don’t think there is one. But in https://github.com/llvm/llvm-project/blob/718af8837639b6a47ae9bf911f668437f0ce0e3c/mlir/tools/mlir-pdll/mlir-pdll.cpp#L52-L85 you can see how to get the pdl module from PDLL and in https://github.com/llvm/llvm-project/blob/main/mlir/test/lib/Rewrite/TestPDLByteCode.cpp you can see how to apply the patterns. You would need to write a pass by yourself if you want to have all of that functionality at once. So no need to compile the patterns into a shared library, they get converted into PDL IR and that can be parsed the same way as any other IR. Inside MLIR the pipeline is pdl → pdl_interp → bytecode. The bytecode will then be applied to your input IR.
I know that there is also the plugin mechanism to pass a path to a shared library to mlir-opt. So in theory you could still decouple a pass and patterns from the main build. I have never used that one though.
1 Like
A C++ pattern can do that as well. PDL is a bit more complex in that it would compile into a “bytecode” format where are runtime when multiple patterns are loaded, their bytecode can be merged and the matching optimized to eliminate redundancies.
See the talk: 2021-04-15: Pattern Descriptor Language ; slides - recording
Another aspect is to be able to decouple the pattern abstraction and application from the “authoring”, see the PDL dialect doc for info, as well as the PDLL DSL documentation (and the presentation from 2021-11-04: PDLL: a Frontend for PDL ; slides - recording 
Thank you, very clear! Please correct me if I am wrong with the PDL pipeline.
Firstly, I use pdl lang to write a pdl pattern xxx.pdll for a specific usage.
Then, I maybe include the xxx.pdll into my main code. At runtime, the pdll will be parsed into pdlmodule IR, then the pdlmodule can be used to match and rewrite the target IR’s Operations?
So my another question is does xxx.pdll will be parsed into c++ match-and-rewrite code and then build with main project?
Thank you mehdi
it would compile into a “bytecode” format where are runtime when multiple patterns are loaded, their bytecode can be merged and the matching optimized to eliminate redundancies.
I have problem to understand how the bytecode is merged. Does the bytecode link with the main project code at link time or something? Sorry, I try to understand it.
Not exactly: the PDLL code is used to produce PDL bytecode, this is what you embed in your main code.
Alternatively you can embed the PDLL parser and general PDL on the fly, but that’s not necessary if you have all the PDLL files at build time (it is useful to expose PDLL to external users at runtime).
No: PDLL → PDL Bytecode → PDL bytecode interpreter.
It does not matter how the bytecode gets in the binary: maybe it’s there at build time and embedded in your binary, alongside other C++ patterns (for example your write PDLL patterns and they get part of the build process), or maybe it is generated on the fly (from PDLL or another mechanism).
1 Like
I don’t think there is one. But in https://github.com/llvm/llvm-project/blob/718af8837639b6a47ae9bf911f668437f0ce0e3c/mlir/tools/mlir-pdll/mlir-pdll.cpp#L52-L85 you can see how to get the pdl module from PDLL and in https://github.com/llvm/llvm-project/blob/main/mlir/test/lib/Rewrite/TestPDLByteCode.cpp you can see how to apply the patterns. You would need to write a pass by yourself if you want to have all of that functionality at once. So no need to compile the patterns into a shared library, they get converted into PDL IR and that can be parsed the same way as any other IR. Inside MLIR the pipeline is pdl → pdl_interp → bytecode. The bytecode will then be applied to your input IR.
I know that there is also the plugin mechanism to pass a path to a shared library to mlir-opt. So in theory you could still decouple a pass and patterns from the main build. I have never used that one though.
sorry, I still cannot figure out how the pdll file is used. It seems that in TestPDLByteCode, the pdl module and target module have already been in the same mlir file. but when and how are they merged?
https://github.com/openxla/stablehlo/tree/main/stablehlo/conversions/tosa/transforms may help here shows a rather self contained usage too.
@jpienaar Thank you. I have a question about when and how is “stablehlo/conversions/tosa/transforms/StablehloLegalizeToTosa.pdll.h.inc” generated?
it seems that it is generated by this cmake function add_mlir_pdll_library(StablehloTOSAPDLLPatternsIncGen
StablehloLegalizeToTosa.pdll
StablehloLegalizeToTosa.pdll.h.inc
)?
But what about the bytecode usage? the above way of pdll usage seems not decoupled from the main code, even if it can be used as a shared lib. is there any example that pdll is parsed and merged at run-time? or could pdll be used in a JIT way in current PDLL framework?
PDL is in the generated file yes, generated from PDLL at compile time (I don’t recall if it is PDL or PDL bytecode in the generated file, easy to check and should also be easy to change the generator if shown useful).
As to going from PDLL to bytecode at runtime, it’s feasible, one could call parser lib and do it. Don’t think anyone has though. The generated file is rather simple in structure, one could copy that + add a parse method before hand + pass string in as constructor arg.