[RFC] Proposal for a high-level ML dialect in MLIR

Responding to many disconnected posts all at once, I’m sorry if I missed a key concern:

I disagree with the opinion that this is just a new dialect and support code and therefore the LLVM “new project” policy doesn’t apply. This is a massive expansion of the charter of MLIR and LLVM, it carries significant design risk, and there are no adopters of this. I agree there is a big community that wants this to exist, but LLVM doesn’t take new projects into the monorepo based on desire. This was true for major projects like clang and mlir as well as backend targets, and many other smaller things that are major expansions of the project.

The LLVM policies are published and exist for very good reasons that have been rehashed and discussed many times before on this forum and elsewhere. LLVM does not take early “in progress” projects into the monorepo for a wide variety of reasons (including exposing people to thrash, concerns contributors will walk away, and concerns that the project never develops into something “actually good”). The policies are built for the “long view” of maintaining a healthy and vibrant ecosystem that can scale for decades, and this puts additional burden on contributors of major new components by design (even if it causes “friction” during development).

We want innovation in the LLVM community, which is exactly why we created the LLVM Incubator process, to foster innovation and provide a place within the LLVM project to host this sort of work. Creating a new incubator for this, or expanding torch-mlir should both be fine because it provides a “graduation” process for code to go into the monorepo when it matures and is understood.

More broadly, there are many things from the early days of MLIR that were not done by proper process - that is a mistake, but that “precedent” should not be used to justify future mistakes. Making analogies to existing code in the MLIR tree with similar character and similar levels of immaturity isn’t justification.

MLIR is part of the LLVM project - if its policies for accepting changes are more broad than the LLVM policy, then we should fix the MLIR policy or revise the LLVM one.

-Chris