We’d like to propose a progressive move of the TOSA dialect construction utility functions into the MLIR core.
These are currently sitting as identical copies in two frontend places:
legalize_common.* and legalize_utils.*
TosaLegalizeCommon and TosaLegalizeUtils
The code is intended to remain framework invariant. The few places where framework-implemented support functions are used will be replaced with standalone code. The Torch side only has the reduction operator support functions right now, but the rest will progressively be used as new legalizations are added on the Torch side.
There’s the obvious question of why it wasn’t in core to begin with. When originally contributed (~Nov 2020) it was easier to develop and stabilize legalizations from TensorFlow dialects to TOSA while sitting in the framework side close to the legalization pass.
They’ve now reached a point of stability and generality - dynamic shape inference support added by @stellaraccident @rsuderman and team for example - that it’s a good time to consider moving them core side to enable better backend codegen expression, particularly now that there are multiple independent framework paths to TOSA in development.
With these sitting in core, it enables a common level of service for TOSA legalization paths from multiple frameworks, carrying shape inference and similar capabilities. The related TosaInferShapes pass is already in the core.
E2E paths will likely have frontend exporters emitting TOSA, being consumed by a backend codegen stack. Typically both these will be some form of (separate) out of tree LLVM based construct, but all of them will benefit from common constructors for TOSA callable after their framework-side rewriter has parsed the input op parameters to extract this information.
It is expected that if a framework has a very unique form of high level op construct, it will maintain its own specialized variant of the standard convert*() function in this library.
Since these utility functions get called from the framework side and there is no framework-like dialect in the core that builds TOSA, testability would continue to be framework-driven as it is now.
A similar utility function set is lib/Dialect/Tosa/Utils/QuantUtils.cpp. It does have a simple test pass, which converts between two quantized domains for the same TOSA IR. However, for this RFC, it’s unclear how to implement a test pass in-core, since there’s no dialect to start from whose TOSA legalization construction could invoke these utility functions.