Jit singleton

Hi everyone !

If I call ExecutionEngine::createJIT (or EngineBuilder::create) more than one time, the second time fails on a assertion “Multiple JIT resolvers?”.
It seems that the JIT is designed to be a singleton in the process, and I was wondering if it was something mandatory.
How hard will it be to make it a non-singleton object ? Is this a JIT-only problem (work needed on JIT classes only) ? Is it a much wider design constraint (on codegen ?) ?

In case, if you ask why I want to create more than one JIT, here is a reason :
I have a JIT application which works fine, a thread is responsible to prepare code execution (work on module, add functions, optimize, call JIT…) and a another thread is responsible for executing the code produced in the first thread.
It works great, but the preparation thread is heavily loaded. I decide to look at what happens, if I spawns 2 threads for preparation of the code.
I have a working prototype, which have one module and one context by thread, and it works. But the JIT is still shared between 2 threads and the time needed to emit a function is not negligible. It will be a great boost of performances, if I can create one execution engine by thread.

What do you think ? Do you have any comments on it ?
Thanks for reading !

Olivier.

In eager compilation mode, I don't know of anything that would go
wrong with having multiple JITs in the process. However, in lazy
compilation mode, we need to map stub addresses to the JIT that knows
how to compile them. Right now, that's done by looking up the static
"TheJITResolver" variable and assuming it's the only JIT, but we could
1) use a static DenseMap<stub_address, JITResolver*> instead, or 2)
include the JITResolver* inside the stub as an argument to the
compilation callback. Nobody's needed this enough to get it working
though.

I think it'd make some sense to fix it for eager compilation even
before getting lazy compilation working. Would you like to write a
patch? You'll have to remove all uses of TheJIT and TheJITResolver
except for the one in JITResolver::JITCompilerFn, and change the "only
one JIT" check to say something about "only one JIT compiling lazily".

I don't think this change will require passing an LLVMContext to the
JIT--it should just use the Context of the function it's jitting.

The code freeze for llvm-2.7 is on Feb 21 (this probably isn't a
"major" change), so if you want it in the 2.7 release, please try to
mail the patch well ahead of that.

Hi Jeffrey,

Thanks for pointing me in the right direction !
I’m not using the JIT in lazy mode, but it was fun to understand the lazy-stub code.

Attached you will find a patch which follow your 1st option : a map Stub_address → JITResolver instance, except that the used map is a “std::map” to apply the same upper_bound trick as in the map CallSiteToFunctionMap of the ResolverState. (If it is necessary for call_site → function, this should be necessary for call_site → resolver… Event if I’m not sure to master all subtle reasons - code alignment/prologue ?)
A mutex on the StubToResolverMap should prevent any possible race conditions.

Every object (Emitter, Resolver, ResolverState) have now a private member instance of JIT (JIT *Jit).

The bugpoint tool is using a extern ‘C’ function : “getPointerToNamedFunction” to get native pointer on function. This function used the TheJit static pointer. To keep alive this functionality, I have added a static SmallVector<JIT*> JitPool (with its mutex JitPoolLock), which keep pointers of available JIT instance. A new static method on JIT object “getPointerToNamedFunctionOnJitPool” fills the need (iterate over available JIT to find the function).

Attached you will find a test, based on “HowToUseJit” example, which instantiate 2 JIT, in eager/lazy mode and retrieve native pointer via getPointerToNamedFunction function. Where does it belong ? Should I add something to llvm test suite ? To the test directory ?

Let me know what you think !
Thanks.

Olivier.

jit_not_a_singleton.diff (28.6 KB)

HowToUseMultipleJIT.cpp (5.92 KB)

Hi!

Olivier Meurant wrote:

Attached you will find a patch which follow your 1st option...

Is there any chance this patch (or a similar solution) will be in 2.7? This
would be a very useful for my application. I assumed one could have multiple
execution engines, however, from search this mailing list I get the sense
that a single execution engine will remain the preferred way of JITting
multiple modules.

My application is a server that uses LLVM to apply shaders to graphics. The
shaders are passed to the server along with the graphical material. The
shading language uses global values, and I assumed the way to achieve
separation (both in terms of concurrency as well as address space) between
the concurrent sessions was to instantiate multiple execution engines. The
singleton nature of JITResolver prevents this.

If this patch makes it in 2.7 (or any future release), I can for now use a
patched trunk until it makes it in an official release. However, if the
intend is not to support this, it is more productive to use the
ExecutionEngine as intended. It would be nice if the documentation provides
some information on the subject...

Thanks.

Maarten

Thanks for the patch! I'll clean this up, convert your sample to a
unit test, and commit it for 2.7.

Thanks Jeffrey !

If possible, keep me inform (on revision number), I’m interested to see how you will do the unit test. (For my future patch… :slight_smile: ).

Thanks again.
Olivier.

I've committed this as
http://llvm.org/viewvc/llvm-project?view=rev&revision=95837. Thanks
again!