How would I go about enabling the ORC JIT to compile code in multiple threads?
I’ve switched Clasp (an implementation of Common Lisp that uses llvm as the backend - github.com/drmeister/clasp) over to using ORC as its JIT. I did this by following the Kaleidoscope tutorial. I have a ClaspJIT_O class that copies the KaleidoscopeJIT class.
Clasp is multithreaded and it needs to compile code in multiple threads because it uses the JIT to generate dispatch functions for multiple dispatch/generic functions. To make this possible, every thread gets its own LLVMContext and every type and llvm::Module that is linked into JITted code in each thread is initialized lazily and thread-locally. Despite this - I experience frequent, random crashes when I try to use the ORC JIT from multiple threads.
Here’s what I’ve tried:
(1) This works: wrap a lock/mutex around access to one ClaspJIT_O object, the calls to ClaspJIT_O::addModule and a call to jitFinalizeReplFunction are protected by the lock:
The Common Lisp code that does the lock and calls these functions:
This throttles the system and limits one thread at a time to add modules to the JIT and lookup symbols in the JIT. It’s not bad - I can live with it.
(2) This fails: Keep a thread local copy of a ClaspJIT_O object that is lazily initialized as soon as any compilation happens in a thread.
(3) This fails: Keep a thread local copy of a ClaspJIT_O object that is initialized as in #2 AND wrap a lock/mutex around ClaspJIT_O::addModule and a call to jitFinalizeReplFunction. What I thought I was testing here was if there was some global resource that ORC uses and despite having multiple thread-local ClaspJIT_O objects the different threads were trampling that common global resource.
I can provide many more details on request.