This proposal is about implementing functionality similar to LLVM opt’s -load/-load-pass-plugin, i.e. runtime loading and linking of rewrite passes. Such functionality enables downstream users to load and link against precompiled mlir-opt binaries, thus decreasing maintenance, particularly with respect to compile times. In addition, this functionality reduces the barrier to entry for developers that aren’t familiar with LLVM/MLIR in general. Finally, albeit of marginal value, it removes that disparity between MLIR and LLVM (which might come as a surprise to novices).
Metaphorically, the goal is to enable construction of an mlir-tutor analog to llvm-tutor.
Proof of Concept
I have a very hacky, minimal working example here. With the (egregious) hacking there I’m able to write this pass
struct BbqPass
: public PassWrapper<BbqPass, OperationPass<ModuleOp>> {
MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID(BbqPass)
StringRef getArgument() const final { return "bbq"; }
StringRef getDescription() const final {
return "my bbq pass the description";
}
void runOnOperation() override {
auto f = getOperation();
std::cerr << "-- dump first op in module\n";
f.getBodyRegion().getOps().begin()->dump();
}
};
compile it out-of-tree on Ubuntu 22 (against mlir-opt compiled prior) using this CMake
cmake_minimum_required(VERSION 3.13.4)
project(HelloWorldMLIR LANGUAGES CXX C)
set(CMAKE_C_STANDARD 11)
set(CMAKE_CXX_STANDARD 17)
set(LT_LLVM_INSTALL_DIR "" CACHE PATH "LLVM installation directory")
set(LLVM_DIR "${LT_LLVM_INSTALL_DIR}/lib/cmake/llvm/")
set(MLIR_DIR "${LT_LLVM_INSTALL_DIR}/lib/cmake/mlir/")
find_package(LLVM REQUIRED CONFIG)
find_package(MLIR REQUIRED CONFIG)
include_directories(${LLVM_INCLUDE_DIRS})
include_directories(${MLIR_INCLUDE_DIRS})
list(APPEND CMAKE_MODULE_PATH "${MLIR_CMAKE_DIR}")
list(APPEND CMAKE_MODULE_PATH "${LLVM_CMAKE_DIR}")
include(AddLLVM)
include(AddMLIR)
add_llvm_pass_plugin(HelloWorldMLIR HelloWorldMLIR.cpp)
and run it like this
mlir-opt -load-pass-plugin $(pwd)/HelloWorldMLIR.so --pass-pipeline=bbq linear.mlir
Easy things
mlir/include/mlir/Pass/PassPlugin.h and mlir/lib/Pass/PassPlugin.cpp are basically a simple matter of s/LLVM/MLIR/g (module case). The only questions here (I think) would be
- Whether to adopt the full
void (*RegisterPassBuilderCallbacks)(PassBuilder &)(substitutingPassManagerforPassBuilder). Currently I do not pass thePassManagerto the plugin because I couldn’t figure out how to do that and add the pass argument to the pass registry beforecl::ParseCommandLineOptionsis called (and without that ordering--pass-pipeline=bbqfails). - Whether to try to unify with
llvm/include/llvm/Passes/PassPlugin.hin order to avoid the blatant code duplication. One way this can be accomplished is actually by just obliging plugin devs to ignore thePassBuilder&(i.e. no updates have to be made tollvm/Passes/PassPlugin.hat all), but this route is probably a bad idea.
mlir/lib/Tools/mlir-opt/MlirOptMain.cpp is also just a matter of adding the cl::list<std::string> passPlugins("load-pass-plugin").
Hard things
The hardest part of getting this working was solving the runtime symbol resolution on the plugin side. Partially this might be because I am not a LLVM CMake conventions connoisseur so I wasn’t able to immediately discern the correct LLVM CMake macros to use; although I got things to work by using add_llvm_pass_plugin(HelloWorldMLIR HelloWorldMLIR.cpp) in the plugin’s CMake and
DEPENDS
${LIBS}
+ SUPPORT_PLUGINS
+ )
+ target_link_libraries(mlir-opt PUBLIC ${LIBS})
+ export_executable_symbols_for_plugins(mlir-opt)
llvm_update_compile_flags(mlir-opt)
in mlir/tools/mlir-opt/CMakeLists.txt, I am not sure if this is the best way. Consequently, I am not sure if the travesty of manually splicing in mangled symbol names into llvm/utils/extract_symbols.py could have been avoided altogether. I suspect that that’s not the case and that some changes do need to be made to extract_symbols.py. But as I’m also not a DWARF/ELF connoisseur, as of right now, I do not know where/why/how those changes should be made. For example, one particularly confusing thing for me was the erroring out due to the plugin being unable to locate _ZTVN4mlir4PassE (i.e. the vtable for mlir::Pass). Investigating by
$ readelf -a --wide mlir-opt | grep _ZTVN4mlir4PassE
599262: 0000000005bc56b8 112 OBJECT LOCAL DEFAULT 24 _ZTVN4mlir4PassE
where the LOCAL binding seems to the issue (after patching extract_symbols.py the binding becomes GLOBAL). Possibly there is some connection to this comment in Pass.h.
I’m happy to go read up on the necessary things (DWARF/ELF) in order to do the right things if people think there’s overall value.
Unknown things (to me)
Everything around all the other things that one might want to be “pluggable”, such as dialects and conversions.
Whether this somehow breaks the one definition rule (or any other C++/LLVM rules for that matter).
Probaby lots of other things.
Happy to hear all comments/criticisms/concerns.
