COAT: an EDSL making just-in-time code generation easier

Hi all,

I open sourced a small project of mine recently. It is an EDSL for C++
which makes just-in-time compilation much easier to write and maintain.
One of its backends is LLVM's OrcJIT (LLVM 7, still).

Here is a blog post of mine introducing the concept:
https://tetzank.github.io/posts/coat-edsl-for-codegen/

You can see it as an abstraction layer on top of OrcJIT, streamlining
the process of JIT compiling a single function one after the other as
much as possible. One use case is ad hoc code generation in databases.

Feedback is welcome. I also like to hear from other users of the JIT
compiler what abstractions they came up with to make it easier to
generate code. I'm probably not the first person to come up with such
things.

Best regards,
Frank

+Lang Hames, father of Orcs - who may indeed benefit from COATs in the coming winter. :wink:

(Lang created ORC - figure he might be interested in this thread)

  • Dave

Hi Frank

Cool project. Thanks for sharing! It reminds me of
https://github.com/BitFunnel/NativeJIT

DSLs appear to be the way to go for now and it totally makes sense to
me. My concern is the "cardinality vs. barrier of entry" balance: the
more powerful a DSL, the harder it is to learn vs. the more specialized
the DSL, the less use-cases it has. Maybe a narrow domain is the key? :slight_smile:

The alternative approach is embedded bitcode from static C++ like:
https://github.com/jmmartinez/easy-just-in-time It appears very
powerful, but it's also much more complicated. Also, it can't deal with
arbitrary C++ either, so I wonder how much benefit it has in practice.

Looking forward to where these experiments will lead us one day!

Cheers,
Stefan

Hi Stefan,

Cool project. Thanks for sharing! It reminds me of
https://github.com/BitFunnel/NativeJIT

Looks interesting and indeed a bit similar. Thanks for the link.

DSLs appear to be the way to go for now and it totally makes sense to
me. My concern is the "cardinality vs. barrier of entry" balance: the
more powerful a DSL, the harder it is to learn vs. the more
specialized the DSL, the less use-cases it has. Maybe a narrow domain
is the key? :slight_smile:

My main concern is readability. If one mixes C++ with EDSL code, it is
sometimes hard to distinguish which variable is what. Is it part of the
function I want to generate or some pre-calculation? I was sometimes
falling back to naming conventions which is less then ideal.

Otherwise, I fully agree. It feels much nicer to nested expressions
with normal operators than just a sequence of API calls.

The alternative approach is embedded bitcode from static C++ like:
https://github.com/jmmartinez/easy-just-in-time It appears very
powerful, but it's also much more complicated. Also, it can't deal
with arbitrary C++ either, so I wonder how much benefit it has in
practice.

Right, how could I forget about Easy::jit? It's a nice approach with a
simple interface. ClangJIT is arguably even a bit more elegant by
defering template instantiations to the runtime and therefore has a
better language integration.

Both approaches have the problem of "large" compilation latency. For
example, modern databases create LLVM IR themselves to avoid the
overhead of Clang. And they are still facing latency problems for short
running queries. COAT just tries to make the IR generation easier. With
the second backend AsmJit, compilation latency is very low.

Best regards,
Frank

ClangJIT is arguably even a bit more elegant by
defering template instantiations to the runtime and therefore has a
better language integration.
[1904.08555] ClangJIT: Enhancing C++ with Just-in-Time Compilation

Ah I had heard of it, but didn't know the paper. Thanks for sharing!

Both approaches have the problem of "large" compilation latency.

IIUC, ClangJIT does C++ -> IR at runtime while EasyJIT does it at
compile-time and embeds the IR into the binary. I think there's pros and
cons with both. I still agree, the LLVM backends are not made for
low-latency.

With the second backend AsmJit, compilation latency is very low.

Good to hear your opinion on AsmJit. I haven't tried it, but now I am
interested!