Another way to JIT: "dlopen from memory"

Project Infrastructure LLVM Dev List Archives
#1

This idea has been kicked around a few times on IRC and elsewhere, and
I think it's pretty clever. The idea is to JIT by making a valid
dylib/dll/so (heceforth, dylib) with the JITed code in memory, and
then convincing the loader to load it. This is nice for the reasons
listed above.

I think it would be neat if LLVM could package up this functionality,
since it fits most use cases for the current LLVM JIT. LLVM
compilation is already heavyweight, so the overhead of linking and
loading the library shouldn't be too bad.

For now, this can be achieved by rolling your own compilation pipeline of:
1. run llc or the equivalent to make a .s in a temp dir
2. assemble it
3. link it to make a dylib in the temp dir
4. dlopen it
5. use dladdr to get the function pointer for your code

1 and 2 can already be done in memory with LLVM.

3 cannot, and doesn't look like it is on anyone's roadmap.

I think 4 may become possible on OS X, and there's this feature
request for glibc that essentially implements "dlopen from memory":
http://sourceware.org/bugzilla/show_bug.cgi?id=11767

5 is trivial.

Even a prototype of this initial JIT model that uses temporaries on
the filesystem would be useful for JIT users doing debugging or
profiling.

I don't really have time to implement this. I'm just kicking the
proverbial can down the road. I may file a feature request PR or
write up something in the style of sabre's LLVMNotes:
http://nondot.org/sabre/LLVMNotes/

Reid

#2

Wouldn't it mean each time a function is compiled it would need to be bundled to its own dylib? How well would that scale?

Félix

#3

Probably not well. But it depends on the application, and probably the loader.

Some applications (OpenCL comes to mind) have the bitcode ready to go
as soon as they start. The only reason they use the JIT is because
they want to compile the code to run fast on the available hardware.

It really depends on how often the application wants to generate code,
which for some people may not be that often.

This is a new model that doesn't really fit with the current
interface, but it could be made to work. I just wanted to throw the
idea out for discussion and to gauge interest.

Reid

#4

Some applications (OpenCL comes to mind) have the bitcode ready to go
as soon as they start. The only reason they use the JIT is because
they want to compile the code to run fast on the available hardware.

If your code never changes and you just want to use target specific
optimizations, can you just statically compile as an installation
step?
I think most applications using JIT (excluding .NET implementations --
which explains popularity of ngen) are at least building the code from
basic bitcode blocks based on some dynamic configuration, if not
generating the whole thing dynamically.

Eugene