I’d like to propose a GitHub repo under LLVM project and governance/contribution model where we could move the current work to. I believe it means I’m proposing it as an incubator project. In the previous discussions there were multiple options proposed and the suggested route of developing on a branch was followed. Skip ahead, and most development is happening in a different repo with direct company affiliation due to various practical reasons.
As in the original discussion, I believe a good step here is as an incubator. I’ll state my current reasoning
I believe all involved from initiation want this to be in a vendor neutral spot that allows for contributions independent of company/neutral collaboration space.
It is currently in a standalone repository, so moving into another is trivial. We would be able to immediately get to more standard contribution model with little pain.
Discussions wrt charters and governance is ongoing at large. I don’t want to gate progress here on that, move of this form would not represent any technical decision that would be made in the new structure. E.g., the discussions etc should still happen, the cost of changing is low (we’d effectively be “spending” a repo in a Github organization and its O(10) mouse clicks to move in and remove).
Including not dictating any new project organizational, it also does not constitute a new technical decision. It was proposed before as element here, the possible incubation steps were discussed etc. I’m not revisiting any of those or fit in this. Those can come later, and may probably require new structure/charter fit discussions.
Thanks for initiating this, @jpienaar. This is actually a very good time to move the TCP repo to an incubator.
We have been developing it in a separate repo under direct company affiliation as you noted. It has gotten to a stage where we have contributed substantial work and made it work with other open source projects as well.
Also, we got some early interests from folks in other companies regarding this, after the MLIR workshop talk we gave last year. So, I do strongly believe making this an incubator and moving to a vendor neutral governance would benefit others too.
@mehdi_amini, you had some thoughts regarding this in the past. Let us know if you have any concerns.
I have not followed the development of TCP. How is this project this project different compared to TOSA, StableHLO, Torch-MLIR, ONNX, Linalg, etc.?
Not trying to derail this thread or oppose the incubation, but I wanted to understand that this is actually a new distinct thing instead of another parallel ML compiler ecosystem which overlaps with what we already have.
In the near term we plan to upstream the TensorRT-MLIR conversions, which would serve as a backend to TCP (alongside the CPU codegen backend we already use for AOT compilations/validations). This would co-exist with the usual GPU codegen backend (allowing configurability to turn on/off a given backend as needed). As the stack is not opinionated (e.g. kernels vs codegen), it allows for multiple approaches to co-exist.
We hope this serves as a demonstration of a heterogeneous / end-to-end flow that folks can pick up and extend (say by plugging in multiple backends) to serve different needs in the industry. A few people from the MLIR community (including @sjarus and @jpienaar) expressed interest in this, and I think having it developed as an LLVM incubator would help foster collaborations and reuse across such projects.
In case folks are wondering, here’s the TCP repo (as it stands today) and is being proposed to be developed as an LLVM incubator going forward:
The repo started off as a branch of torch-mlir (another LLVM incubator) and as such has preserved the licensing and/or code guidelines from LLVM/MLIR so this should be a smooth transition provided there is community consensus.
I don’t have a technical opinion on the content of the project itself – and I don’t think I need to in order to support the RFC. It seems like one of those situations where folks very connected to the community need a home for the code and would like to make sure it remains license/governance adjacent to the rest of the project. That seems like a reasonable investment in the future of the commons and was a key motivator in establishing the incubator repos to begin with.
So +1 from me (and +1 to Jacques’ characterization of “spending” a repo being basically free).
+1 from me. There’s substantial value here - both within and beyond the dialect itself. There’s a lot of robust pass infrastructure around the dialect constructs that are high value and worth retaining.
Just an attempt to address this question (which warrants a separate follow-up) at the high level without going into too many details in this RFC:
TCP is at the tensor abstraction, so shares many similarities to TOSA. However, unlike TOSA / StableHLO, it is not tied to a spec and doesn’t provide stability guarantees (i.e. if the IR were to be serialized then de-serialized at a later point it most likely wouldn’t work). This allows for it to be used as a compiler IR (that is flexible and can adapt quickly to evolving needs).
The important thing to note here is that TCP is expected to complement TOSA and/or StableHLO rather than replace them. At Cruise we use a combination of TCP + TOSA for our middle end and have found this combination rather useful - TOSA is quite comprehensive in its op/converter coverage, and TCP provides strong dynamism constructs (symbolic shape prop, shape dynamism etc) for the ops that needed it.