[RFC] Upstreaming a proper SPIR-V backend

I would prefer the non-MLIR route first, as proposed by the original
RFC. MLIR is great and all but it certainly opens up the possibility
for unrelated problems to slow down and derail the effort. My 2c.

~ Johannes

Hi,

A very good question. I was actually expecting it :blush:

So, at the moment, it does not integrate into MLIR SPIRV backend and we have not thought about it. I guess You are referring to having a SPV dialect in MLIR and using a ‘serialize’ option to produce a SPIR-V binary?

I agree that developing two backends in parallel is a bit redundant. If SPIR-V LLVM backend becomes a production quality it means actually it could consume any LLVM IR (provided it does conform to some SPIR-V restrictions).
By any LLVM IR input I mean: it should be irrelevant whether it is produced by a clang, MLIR to LLVM IR lowering or just some other front-end that produces LLVM IR.

The biggest ‘impedance mismatch’ that I currently see is that SPV MLIR dialect is now targeted mostly at Vulkan, while LLVM SPIR-V backend targets compute. Besides instruction set, the fundamental difference is a memory model.
So if we want to unify those, we should actually make SPIR-V LLVM backend able to produce Vulkan dialect of SPIR-V as well.

My answer is a bit elusive, but I totally agree with Your proposal: we should work towards having a one solution, and, LLVM SPIR-V backend seems like a more universal one (since it sits lower in the compiler stack).
My proposal would be to include some MLIR → LLVM-IR translated code in the testing so to have this final goal in mind.

Something you’re missing here, and maybe Lei clarified but I’ll reiterate: the SPIRV dialect in MLIR is equivalent to what your GlobalISel pass will produce. It can actually round-trip to/from the SPIRV binary format. So it is sitting lower than your backend in my view.
I can’t figure out a situation where it would make sense to go from MLIR SPIRV dialect to LLVM to use this new backend, but I may miss something here…

It would be really great to find a common path here before duplicating a lot of the same thing in the lllvm-project monorepo, for example being able to target the MLIR dialect from GlobalISel, or alternatively converting the MIR to it right after would be an interesting thing to explore.
I haven’t seen it, but there was a talk last Sunday on this topic: https://llvm.org/devmtg/2021-02-28/#vm1

This sort of problem seems like just one of those unfortunate consequences of MLIR being effectively an “LLVM IR 2.0 – Generic Edition”, but not yet actually layered underneath LLVM where it really wants to be.

I don’t understand what you mean here with “layered underneath LLVM”? Can you elaborate on this?

That ultimately the goal should be for LLVM IR to be a dialect of MLIR, and for much of the optimization and codegen processes in LLVM to be implemented as MLIR dialect lowering. Then, MLIR is foundational – “layered” underneath LLVM’s core – LLVM would have a hard dependency on MLIR.

OK I see what you mean now, I didn’t connect to this because I think it is an open question whether we see this happen in this decade :wink:

So my assumption coming here is that:

  1. LLVM IR as it is now is “granted” (at least in the context of this thread).
  2. A SPIRV backend that takes LLVMIR and use GlobalISel is desirable.

Considering this, my angle is mainly one of library, software engineering, and reuse / avoiding duplication.
So adding intrinsics to LLVM IR and improving the GPU support in LLVM IR is something I see as “obviously good” and necessary for this project. The only opportunity for sharing and avoiding duplication appears to me right after GlobalISel for the rest of the pipeline.

At that point, SPIR-V as an MLIR dialect, and the SPIR-V backend doing MLIR dialect lowering would be effectively no different from how every target works – just with a different output dialect.

I think it doesn’t really make sense to tie this project to those long-term goals of layering MLIR under LLVM-IR, given the extremely long timescale that is likely to occur in. The “proper” solution probably won’t be possible any time soon.

I’m not sure if we’re talking about the same thing here: there is nothing that I suggest that would operate at the level of LLVM IR. And nothing that requires a “long timescale”, it seems quite easily in scope to me here.

So, in the meantime, we could implement a special-case hack just for SPIRV, to enable lowering it to MLIR-SPIRV dialect. But, what’s the purpose? It wouldn’t really help move towards the longer term goal, I don’t think? And if someone does need that at the moment, they can just feed the SPIRV binary format back into the existing MLIR SPIRV dialect, right?

Do we want to maintain, in the LLVM monorepo, two different implementations of a SPIRV IR and associated serialization (and potential deserialization)? All the tools associated to manipulate it? I assume the backend may even want to implement optimization passes, are we gonna duplicate these as well?
(note that this isn’t at the LLVM IR level, but post-instruction selection, so very ad-hoc to the backend anyway).0

Quite possibly yes. It’s unfortunate to have duplication, but given the current state of things, I think it should not be ruled out.

My inclination is that the following factors are likely to be true:

  • The amount of code for SPIRV binary format serialization is not particularly large or tricky.

  • The work to emit SPIR-V MLIR dialect from the LLVM SPIR-V backend will not be simpler than serializing to SPIR-V directly.

  • Writing this custom code to emit SPIR-V MLIR dialect from the SPIR-V backend will not noticably further the longer-term goals of having LLVM core be implemented as MLIR dialect lowering.

These are great considerations, I subscribe entirely :slight_smile:

It seems to me that the choice here is either writing new code in LLVM to emit the SPIR-V MLIR dialect in the GlobalISel SPIR-V backend, or new code in LLVM to emit SPIR-V directly. And while I find the long-term prospects of MLIR integration into LLVM extremely promising, using MLIR just as step-stone to MLIR SPIR-V serialization does not seem particularly interesting.

So, to me the interesting question is whether we’d expect to be doing something interesting after converting to the SPIR-V MLIR dialect form besides simply serializing to SPIR-V binary format. Something that would make the added complexity of serializing through MLIR seem more worthwhile. I guess I’m not immediately seeing this as likely to be the case, but it seems well worth further discussion.

A possibility you’ve mentioned is post-instruction-selection optimizations. Do you have something in particular in mind there?

I suspect that post-Global ISel there is a bit more “than taking the MIR as-is and emit the SPIRV serialization in a single traversal”. So converting MIR to MLIR means that everything that we want to happen at this point will be shared.
Note that this is different from other backends, because I don’t expect SPIRV to share passes (RA, Scheduling, …) on MIR or to take advantage of the MC layer in the same way (if I’m wrong here then my point is less strong though).

Cheers,

Answering to Renato's points:

I think there are two points here:

1. How many SPIRV end-points we have

I would rather call this 'two entry points', as to having two entry points for accessing SPIR-V: either through LLVM-IR with augmentation (metadata/intrinsics), or, a proper MLIR 'SPV' dialect.

This is mostly about software engineering concerns of duplication, maintenance, etc. But it's also about IR support, with MLIR having an upper hand here because of the existing implementation and its inherent flexibility with dialects.

It's perfectly fine to have two back-ends for a while, but since we moved MLIR to the monorepo, we need to treat it as part of the LLVM family, not a side project.

Agreed. We are not treating MLIR as a side project :blush:

LLVM IR has some "flexibility" through intrinsics, which we could use to translate MLIR concepts that can't be represented in LLVM IR for the purpose of lowering only. Optimisations on these intrinsics would bring the usual problems.

2. Where do the optimisations happen in code lowering to SPIRV

I think Ronan's points are a good basis for keeping that in MLIR, at least for the current code. Now, if that precludes optimising in LLVM IR, than this could be a conflict with this proposal.

I think You are referring to points made by Lei, not by Ronan :wink:
I believe that the idea of MLIR path is to completely skip LLVM-IR optimization passes, so having just the MLIR 'entry point' would preclude that possibility. (though, it would be possible to do FE -> LLVM-IR (optimize here) -> LLVM-IR to MLIR -> MLIR to 'spv' MLIR dialect -> SPIR-V, but that seems like an overkill...)

If the code passes through MLIR or not will be a decision of the toolchain, that will pick the best path for each workload. This allows us to have concurrent approaches in tree, but also makes it hard to test and creates corner cases that are hard to test.

If we provide two 'entry points' for accessing SPIR-V, it is up to the toolchain to decide the most convenient way. I'm not sure whether this would be a runtime decision though. I believe that all future front-ends would like to target MLIR directly (and skip LLVM-IR altogether).

So, while I appreciate this is a large proposal, that will likely take a year or more to get into shape, I think the ultimate goal (after the current proposal) should be that we end up with one back-end.

Agree. Though I would say : one back-end, but two 'entry points'. As I wrote in a reply to Mehdi, it seems that an option of having 'LLVM IR backend' produce 'spv' MLIR dialect would be a good way to ultimately unify the implementation. Though, that seems like a longer distance and needs some research, since, at the moment I'm not sure how to tackle this (to have GlobalISel produce MLIR as an output).

I'm a big fan of MLIR, and I think we should keep developing the SPIRV dialect and possibly this could be the entry point of all SPIRV toolchains.

MLIR should be an entry point for all future SPIRV toolchains - that is the future. There will be still toolchains that are legacy and cannot be rewritten to use MLIR.

While Clang will take a long time (if ever) to generate MLIR for C/C++, it could very well generate MLIR for non-C++ (OpenCL, OpenMP, SYCL, etc) which is then optimised, compiled into LLVM IR and linked to the main module (or not, for multi-targets) after high-level optimisations.

I'm not sure about Clang OpenCL support. I believe that OpenCL C/C++ cannot produce MLIR directly. For OpenMP, I know that flang (Fortran) does have a MLIR based 'codegen'. Not sure about SYCL either. Someone from Intel should know? Does clang based SYCL compiler have a path to produce MLIR?

This would answer both questions above and create a pipeline that is consistent, easier to test and with lower overall maintenance costs.

We should definitely aim at SPIR-V support to be less fragmented if possible (at the moment, we also have SPIR-V <-> LLVM bidirectional translator which is an external project).

So, at the moment, it does not integrate into MLIR SPIRV backend and we have not thought about it. I guess You are referring to having a SPV dialect in MLIR and using a ‘serialize’ option to produce a SPIR-V binary?

I agree that developing two backends in parallel is a bit redundant. If SPIR-V LLVM backend becomes a production quality it means actually it could consume any LLVM IR (provided it does conform to some SPIR-V restrictions).
By any LLVM IR input I mean: it should be irrelevant whether it is produced by a clang, MLIR to LLVM IR lowering or just some other front-end that produces LLVM IR.
The biggest ‘impedance mismatch’ that I currently see is that SPV MLIR dialect is now targeted mostly at Vulkan, while LLVM SPIR-V backend targets compute. Besides instruction set, the fundamental difference is a memory model.
So if we want to unify those, we should actually make SPIR-V LLVM backend able to produce Vulkan dialect of SPIR-V as well.

My answer is a bit elusive, but I totally agree with Your proposal: we should work towards having a one solution, and, LLVM SPIR-V backend seems like a more universal one (since it sits lower in the compiler stack).
My proposal would be to include some MLIR → LLVM-IR translated code in the testing so to have this final goal in mind.

Something you’re missing here, and maybe Lei clarified but I’ll reiterate: the SPIRV dialect in MLIR is equivalent to what your GlobalISel pass will produce. It can actually round-trip to/from the SPIRV binary format. So it is sitting lower than your backend in my view.
I can’t figure out a situation where it would make sense to go from MLIR SPIRV dialect to LLVM to use this new backend, but I may miss something here…

By ‘lower’ I was referring to the place of backend in a typical compiler flow that I could imagine: MLIR → LLVM-IR (opt) → Bakcend (llc).
And yes, I agree, if we treat MLIR SPV dialect as a final result of what this backend would produce, then MLIR SPV could be the lowest-level representation (before streaming into SPIR-V binary).

It would be really great to find a common path here before duplicating a lot of the same thing in the lllvm-project monorepo, for example being able to target the MLIR dialect from GlobalISel, or alternatively converting the MIR to it right after would be an interesting thing to explore.
I haven’t seen it, but there was a talk last Sunday on this topic: https://llvm.org/devmtg/2021-02-28/#vm1

We should investigate that. I believe though that GlobalISel is not really that flexible to produce MLIR (or dialects) - but that is something we might want to change :blush: That path would open us a door to have a great deal of unification:

+1. This sounds quite interesting and worth exploration. Starting with some initial feasibility/blockers investigation would be awesome. It’s also not scoped to SPIR-V per se; done right it actually can have the ability to connect any MLIR dialect I guess? (So later we can design other dialects for LLVM CodeGen probably.) Connecting LLVM and MLIR is a huge effort. And it needs to start somewhere. This might be a nice first attempt? (Please pardon me if this is obviously not; I’m not super familiar with GlobalISel infrastructure.)

Hi,

A very good question. I was actually expecting it :blush:

So, at the moment, it does not integrate into MLIR SPIRV backend and we have not thought about it. I guess You are referring to having a SPV dialect in MLIR and using a ‘serialize’ option to produce a SPIR-V binary?

I agree that developing two backends in parallel is a bit redundant. If SPIR-V LLVM backend becomes a production quality it means actually it could consume any LLVM IR (provided it does conform to some SPIR-V restrictions).
By any LLVM IR input I mean: it should be irrelevant whether it is produced by a clang, MLIR to LLVM IR lowering or just some other front-end that produces LLVM IR.

The biggest ‘impedance mismatch’ that I currently see is that SPV MLIR dialect is now targeted mostly at Vulkan, while LLVM SPIR-V backend targets compute. Besides instruction set, the fundamental difference is a memory model.
So if we want to unify those, we should actually make SPIR-V LLVM backend able to produce Vulkan dialect of SPIR-V as well.

My answer is a bit elusive, but I totally agree with Your proposal: we should work towards having a one solution, and, LLVM SPIR-V backend seems like a more universal one (since it sits lower in the compiler stack).
My proposal would be to include some MLIR → LLVM-IR translated code in the testing so to have this final goal in mind.

Something you’re missing here, and maybe Lei clarified but I’ll reiterate: the SPIRV dialect in MLIR is equivalent to what your GlobalISel pass will produce. It can actually round-trip to/from the SPIRV binary format. So it is sitting lower than your backend in my view.
I can’t figure out a situation where it would make sense to go from MLIR SPIRV dialect to LLVM to use this new backend, but I may miss something here…

It would be really great to find a common path here before duplicating a lot of the same thing in the lllvm-project monorepo, for example being able to target the MLIR dialect from GlobalISel, or alternatively converting the MIR to it right after would be an interesting thing to explore.
I haven’t seen it, but there was a talk last Sunday on this topic: https://llvm.org/devmtg/2021-02-28/#vm1

This sort of problem seems like just one of those unfortunate consequences of MLIR being effectively an “LLVM IR 2.0 – Generic Edition”, but not yet actually layered underneath LLVM where it really wants to be.

I don’t understand what you mean here with “layered underneath LLVM”? Can you elaborate on this?

That ultimately the goal should be for LLVM IR to be a dialect of MLIR, and for much of the optimization and codegen processes in LLVM to be implemented as MLIR dialect lowering. Then, MLIR is foundational – “layered” underneath LLVM’s core – LLVM would have a hard dependency on MLIR.

At that point, SPIR-V as an MLIR dialect, and the SPIR-V backend doing MLIR dialect lowering would be effectively no different from how every target works – just with a different output dialect.

I think it doesn’t really make sense to tie this project to those long-term goals of layering MLIR under LLVM-IR, given the extremely long timescale that is likely to occur in. The “proper” solution probably won’t be possible any time soon.

I’m not sure if we’re talking about the same thing here: there is nothing that I suggest that would operate at the level of LLVM IR. And nothing that requires a “long timescale”, it seems quite easily in scope to me here.

So, in the meantime, we could implement a special-case hack just for SPIRV, to enable lowering it to MLIR-SPIRV dialect. But, what’s the purpose? It wouldn’t really help move towards the longer term goal, I don’t think? And if someone does need that at the moment, they can just feed the SPIRV binary format back into the existing MLIR SPIRV dialect, right?

Do we want to maintain, in the LLVM monorepo, two different implementations of a SPIRV IR and associated serialization (and potential deserialization)? All the tools associated to manipulate it? I assume the backend may even want to implement optimization passes, are we gonna duplicate these as well?
(note that this isn’t at the LLVM IR level, but post-instruction selection, so very ad-hoc to the backend anyway).0

Quite possibly yes. It’s unfortunate to have duplication, but given the current state of things, I think it should not be ruled out.

My inclination is that the following factors are likely to be true:

  • The amount of code for SPIRV binary format serialization is not particularly large or tricky.

  • The work to emit SPIR-V MLIR dialect from the LLVM SPIR-V backend will not be simpler than serializing to SPIR-V directly.

Serialization itself is not a large amount of work; But I think there are also other considerations worth calling out. In the development of the SPIR-V MLIR dialect, we have been following the spec verbatim and thoroughly to add validation rules when introducing new operations. Going through SPIR-V MLIR dialect we can have all the validation on the operations and against the target environment to guarantee the generated binary is valid. I’d assume you’ll need that validation too before serialization in the LLVM SPIR-V backend?

Actually SYCL is pure C++, just with a few special C++ classes similar
to some other special things like std::thread or setjump()/longjump().

OpenMP, when associated to C++, is also pure C++.

In your list OpenCL is a language based on C/C++ to program accelerators
while SYCL & OpenMP are single-source frameworks to program full
applications using a host and some accelerators, with both part in the
same source program in a seamless and type-safe way.

So the MLIR approach is quite compelling with its "Multi-Level"
representation of both the host and the device code to enable
multi-level inter-procedural or inter-module optimizations which cannot
be done today when compiling single-source OpenMP/SYCL/CUDA/HIP/OpenACC
because most implementations use early outlining of the device code,
thus it is super hard to do inter-module optimization later without a
multi-level view.

As you and most other people said, it looks we are stuck with plain LLVM
for a while.

But perhaps you were considering in your sentence the case where with
OpenMP/SYCL/CUDA/HIP you generate LLVM for the host code part and MLIR
just for the hardware accelerator parts?

While it would obviously allow to recycle more easily the MLIR SPIR-V
generator, it would still require somehow to generate MLIR from the C++
accelerator parts. At least the C++ code allowed in accelerator parts is
often restricted, so it is easier to do than with full-fledged host part
C++ and actually there are a few hacks trying to do this (for example
leveraging Polly, PPCG...). But it seems we are far from a
production-quality infrastructure yet.

So it looks like, while we do not have a robust C++ to MLIR path, we need
an LLVM IR to SPIR-V path somehow.

At least, as others like Mehdi said, let's do good software
engineering and factorize out as much as we can between the LLVM IR and
MLIR paths.

So, at the moment, it does not integrate into MLIR SPIRV backend and we have not thought about it. I guess You are referring to having a SPV dialect in MLIR and using a ‘serialize’ option to produce a SPIR-V binary?

I agree that developing two backends in parallel is a bit redundant. If SPIR-V LLVM backend becomes a production quality it means actually it could consume any LLVM IR (provided it does conform to some SPIR-V restrictions).
By any LLVM IR input I mean: it should be irrelevant whether it is produced by a clang, MLIR to LLVM IR lowering or just some other front-end that produces LLVM IR.
The biggest ‘impedance mismatch’ that I currently see is that SPV MLIR dialect is now targeted mostly at Vulkan, while LLVM SPIR-V backend targets compute. Besides instruction set, the fundamental difference is a memory model.
So if we want to unify those, we should actually make SPIR-V LLVM backend able to produce Vulkan dialect of SPIR-V as well.

My answer is a bit elusive, but I totally agree with Your proposal: we should work towards having a one solution, and, LLVM SPIR-V backend seems like a more universal one (since it sits lower in the compiler stack).
My proposal would be to include some MLIR → LLVM-IR translated code in the testing so to have this final goal in mind.

Something you’re missing here, and maybe Lei clarified but I’ll reiterate: the SPIRV dialect in MLIR is equivalent to what your GlobalISel pass will produce. It can actually round-trip to/from the SPIRV binary format. So it is sitting lower than your backend in my view.
I can’t figure out a situation where it would make sense to go from MLIR SPIRV dialect to LLVM to use this new backend, but I may miss something here…

By ‘lower’ I was referring to the place of backend in a typical compiler flow that I could imagine: MLIR → LLVM-IR (opt) → Bakcend (llc).
And yes, I agree, if we treat MLIR SPV dialect as a final result of what this backend would produce, then MLIR SPV could be the lowest-level representation (before streaming into SPIR-V binary).

It would be really great to find a common path here before duplicating a lot of the same thing in the lllvm-project monorepo, for example being able to target the MLIR dialect from GlobalISel, or alternatively converting the MIR to it right after would be an interesting thing to explore.
I haven’t seen it, but there was a talk last Sunday on this topic: https://llvm.org/devmtg/2021-02-28/#vm1

We should investigate that. I believe though that GlobalISel is not really that flexible to produce MLIR (or dialects) - but that is something we might want to change :blush: That path would open us a door to have a great deal of unification:

+1. This sounds quite interesting and worth exploration. Starting with some initial feasibility/blockers investigation would be awesome. It’s also not scoped to SPIR-V per se; done right it actually can have the ability to connect any MLIR dialect I guess? (So later we can design other dialects for LLVM CodeGen probably.) Connecting LLVM and MLIR is a huge effort. And it needs to start somewhere. This might be a nice first attempt? (Please pardon me if this is obviously not; I’m not super familiar with GlobalISel infrastructure.)

It would be great to hear backend experts’ opinions on that, but it seems that it’s more of a problem of producing MLIR with standard LLVM codegen infra, no matter whether it’s SelectionDAG or GlobalISel.

We can support two ‘entry points’ :

  1. Directly through MLIR. It gets translated to SPV dialect, and then streamed to SPIR-V binary. (without even going into LLVM-IR)
  2. Start with LLVM-IR (with some augmented metadata and intrinsics). Feed that into proposed SPIR-V backend. Backend will produce MLIR SPV dialect and make use of whatever legalization/binary emission/etc. it provides.
    This way, SPIR-V LLVM backend will be (a probably tiny) wrapper around MLIR SPV. Then, the majority of work would focus on MLIR SPV (e.g. adding support for OpenCL environment in addition to existing Vulkan compute).

I’m actually wondering what’s the point of having a backend for ‘spv’ dialect of MLIR rather than some generic ‘mir’ dialect? The latter may help in retargeting other backends eventually, which may be perceived as a huge plus.

From the implementation point of view, that would bring us huge re-use. Still, from the design point of view, we need to maintain two ‘GPU centric’ representations’: LLVM-IR with SPIR-V intrinsics/metadata/attributes + MLIR SPV dialect.
Still that would be a much better situation from the community point of view.


konrad


LLVM Developers mailing list
llvm-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


LLVM Developers mailing list
llvm-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Thanks,Alex

Just thinking out loud if clang couldn’t be a hybrid front-end, emitting LLVM IR and MLIR for different parts of the program (for example, accelerators), and either use SPIRV (for supported accelerators) or lower to LLVM IR (for the rest). This would allow us to use MLIR directly in hybrid programming models (like OpenMP, OpenCL) and make real use of the high-level optimisations in MLIR. Perhaps SYCL wouldn’t fit here.

I’m just going back to the overall goal of MLIR and trying to see on our current approach, what do we lower too soon to LLVM IR, so it could lower to MLIR instead, piece wise, then lower to LLVM IR (or not) later.

     > While Clang will take a long time (if ever) to generate MLIR
     > for C/C++, it could very well generate MLIR for non-C++
     > (OpenCL, OpenMP, SYCL, etc) which is then optimised,
     > compiled into LLVM IR and linked to the main module (or not,
     > for multi-targets) after high-level optimisations.

Actually SYCL is pure C++, just with a few special C++ classes similar
to some other special things like std::thread or setjump()/longjump().

OpenMP, when associated to C++, is also pure C++.

In your list OpenCL is a language based on C/C++ to program accelerators
while SYCL & OpenMP are single-source frameworks to program full
applications using a host and some accelerators, with both part in the
same source program in a seamless and type-safe way.

So the MLIR approach is quite compelling with its "Multi-Level"
representation of both the host and the device code to enable
multi-level inter-procedural or inter-module optimizations which cannot
be done today when compiling single-source OpenMP/SYCL/CUDA/HIP/OpenACC
because most implementations use early outlining of the device code,
thus it is super hard to do inter-module optimization later without a
multi-level view.

FWIW, the indirection "caused by" early outlining is already covered
with callbacks in LLVM-IR. The separation of host and device code
will be covered with heterogeneous LLVM-IR modules.

"Super hard" is definitively not how I would describe it given our
current state, we mainly need to put the things together.

~ Johannes

But perhaps you were considering in your sentence the case where with
OpenMP/SYCL/CUDA/HIP you generate LLVM for the host code part and MLIR
just for the hardware accelerator parts?

Just thinking out loud if clang couldn't be a hybrid front-end, emitting
LLVM IR and MLIR for different parts of the program (for example,
accelerators), and either use SPIRV (for supported accelerators) or lower
to LLVM IR (for the rest). This would allow us to use MLIR directly in
hybrid programming models (like OpenMP, OpenCL) and make real use of the
high-level optimisations in MLIR. Perhaps SYCL wouldn't fit here.

As Ronan said, OpenMP (and others) is C/C++/Fortran + some additions.
Without having a full fledged C/C++ MLIR route we won't be able to build
a toolchain I would trust, not to mention the lack of evidence we need
to go to MLIR in the first place :wink:

~ Johannes

Just thinking out loud if clang couldn't be a hybrid

    > front-end, emitting LLVM IR and MLIR for different parts of
    > the program (for example, accelerators), and either use
    > SPIRV (for supported accelerators) or lower to LLVM IR (for
    > the rest).

At some point everything is possible.

    > This would allow us to use MLIR directly in hybrid
    > programming models (like OpenMP, OpenCL) and make real use
    > of the high-level optimisations in MLIR. Perhaps SYCL
    > wouldn't fit here.

I guess you have exchanged the words OpenCL and SYCL in this sentence.
OpenCL is like graphics shader languages: a foreign language to the
host. From the host point-of-view it is just a host foreign API managing
memory allocation on the device and controlling kernel execution on some
devices (think RPC). The advantage is that you can use OpenCL from a
COBOL host program if you want but this is another story... :slight_smile:

Hi,

Thank you all for a very in-depth discussion so far.
Besides the major topic of MLIR and LLVM-IR coexistence, are there any other comments, especially regarding 'Open questions' section that we proposed?

My recap so far is:
  * There is a good reception from the community that is interested in LLVM-IR path (a classical FE/opt/code-generation path, clang community)
  * There is a concern on maintenance cost if we have two solutions in parallel: MLIR based, and LLVM-IR based. We will look for the ways to address this, one investigation point would be generating MLIR 'spv' dialect from the target backend infrastructure (GlobalISel)
  * We also need to iron out the details of the semantics and capabilities of SPIR-V that we would like to expose: 1) which exact subset of LLVM-IR is acceptable by the backend, 2) how do we expose the extensions and core builtins, 3) how do we map a memory model to LLVM-IR (especially if we think about adding Vulkan compute memory model) etc.

Based on the feedback so far, that would be roughly my plan:
  * Go ahead with a SPIR-V backend in LLVM-IR, as planned. Look for clang integration.
  * Midterm: investigate MLIR 'spv' dialect generation from GlobalISel (or other means) as an unifying solution
  * Long-term: come up with a single, MLIR based backend (which is going to take care of serialization, deserialization, automatic specification updates and all the rest of the infrastructure work). This still means we will have two entry points: directly through MLIR 'spv' dialect or through LLVM-IR with intrinsics/metadata. Will support both Vulkan compute and OpenCL compute models.
  * When/if clang eventually switches to MLIR code generation, that would be eventually the end of LLVM-IR path :slight_smile:

For the first point, we need a close collaboration with GlobalISel and clang communities. For the last three points, we will work closely with MLIR community on engineering and IR specification.

What do You think? Does this sound like a reasonable plan?

thanks,
konrad

The first point sounds good, let's start there and revisit the rest as it becomes
interesting, no need to finalize anything as the context is arguably fluid.

Hi,

Thank you all for a very in-depth discussion so far.
Besides the major topic of MLIR and LLVM-IR coexistence, are there any other comments, especially regarding ‘Open questions’ section that we proposed?

My recap so far is:

  • There is a good reception from the community that is interested in LLVM-IR path (a classical FE/opt/code-generation path, clang community)
  • There is a concern on maintenance cost if we have two solutions in parallel: MLIR based, and LLVM-IR based. We will look for the ways to address this, one investigation point would be generating MLIR ‘spv’ dialect from the target backend infrastructure (GlobalISel)
  • We also need to iron out the details of the semantics and capabilities of SPIR-V that we would like to expose: 1) which exact subset of LLVM-IR is acceptable by the backend, 2) how do we expose the extensions and core builtins, 3) how do we map a memory model to LLVM-IR (especially if we think about adding Vulkan compute memory model) etc.

Based on the feedback so far, that would be roughly my plan:

  • Go ahead with a SPIR-V backend in LLVM-IR, as planned. Look for clang integration.
  • Midterm: investigate MLIR ‘spv’ dialect generation from GlobalISel (or other means) as an unifying solution

To me this is something to look at in the very short term: I’m wary of investment in duplicated infrastructure for SPIRV binary serialization/deserialization in-tree. I.e. this covers the path from MIR (post-GlobalISel)->SPIRV (and potentially any MIR-level transformations).
To be clear: this does not block landing parts of the backend (all the GlobalISel specification for instance), but I suspect you’ll want to get something end-to-end.

The two points below are indeed more long-term / hypothetical in my opinion: good to keep in mind but clearly off the table right now.