I agree that having the conversion live as part of LLVM is the best option going forward.
libclc is an library implementation of the OpenCL builtins and I think its fine for that aspect of the converter,
i.e. consumption of SPIR-V into LLVM IR in preparation for code generation by LLVM by its various backends as a library
to link the generated code to, to live in an external repo.
This is fine because SPIR<->LLVM IR is trivial, SPIR-V<->LLVM is not.
Having the SPIR-V <-> LLVM-IR conversion part of the tool as an external repo as opposed to the llvm would
make it very difficult to use as a fontend:
First the changes to the build pipeline are a hinderance. In my fork of SPIRV-LLVM the backend is a target
because I couldn’t get CMake to link the libraries to LDC as a support library. This would be made even more
complicated with at external project. I would probably resort to invoking an external program despite integrated
conversion being the best solution.
Secondly (unless you've changed things) all of the builtins are not intrinsics, but C++ Itanium with extensions mangled.
This makes it so that anyone not implementing a C++ frontend has to deal with it, presumably by hardcoding the
names since C++ is very non-trivial. Even for projects like LDC (the LLVM based D compiler) which have a C++ mangler,
we still have to hardcode the names because Windows does not use Itanium and the compiler is not set up to do two
different forms of C++ mangling, and even if that were possible the non-standard extensions make it infeasible.
Whereas any LLVM frontend worth is salt has a way of exposing intrinsics.
The backend as it currently stands is not based on tablegen and that is required AFAIU to support intrinsics.
I have tablegen descriptions of the SPIR-V format mostly complete (all core instruction relevant to OpenCL +
most of the ones for Vulkan, + all the OpenCL extension instructions) though not the intrinsics.
However I didn’t get around to using them because I have more important things to work on in the front end.
Then there is the question of what to do about Vulkan? I would love to have support for Vulkan in LLVM.
That would be relatively simple to set up for with SPIRV as a target, even more so with a tablegen based
infrastructure. I have no idea how that would work as an external project.
I also think that having it in an external project will greatly reduce the visibility in comparison to a target.
How many of you have cloned libclc for example? Not many I suspect. Probably because you are not
working on implementing OpenCL for a backend. And while libclc is useful to those implementing OpenCL
they are vastly outnumbered by those using OpenCL i.e. producing SPIR/SPIR-V. My usage of SPIR-V is
I generate it from my source files though LDC/LLVM then I hand it off to the OpenCL implementation.
Nowhere in that pipeline do I need something like libclc. Now libclc may be used in the implementation
but that’s a different copy of LLVM to the one that produced it.
This really needs to be a target so that we can use intrinsics, have it as part of regular LLVM releases
so that it can be used by many frontends. Having an external project for assisting backends consuming
SPIR-V (a la libclc) is fine but that shouldn’t mean that the processes used for generating SPIR-V need to go
into a separate project.
Yes, for compilers it would be much more convenient to have a backend, but what OpenCL implementations need (for example) is only a bidirectional converter.
I don't think anybody will oppose to the converter becoming a backend at some point (as long as we keep exposing the conversion passes as today), but at the current pace it will take years and I don't think it's fair for everybody that we keep this codebase in the limbo for much longer.
Currently there's people shipping this code in products, and have to carry heavily modified forks because there's no community upstream to contribute to.
So, I think we really need at this point to find some place where the community can gather and work together on stability, intrinsics, have a proper backend, etc.
This proposal is only about where the collaboration would happen in the near future. If at some point we have also a backend, and people want to have it along the other backends in the main LLVM repo, that will be great. But in the meantime we need a place to work together.