LLVM JIT in multi-threaded environments

Hi all,

I have some questions on some strange behavior I’m seeing with LLVM JIT in a multithreaded application. If this is the wrong forum for questions like this, I apologize, and please redirect me.

The setup of the application is a singleton instance of a static Compiler object, which maintains local copies of an LLVM Module, JIT ExecutionEngine, and FunctionPassManager with registered optimization passes to be applied to all functions.

There are a handful of worker threads that pass state pointers to IR building routines within the Compiler object. These Compiler routines return a list of llvm::Function pointers to all functions generated for the given state. These functions can get pretty big, so in the interest of performance during the optimization passes, I break them up into several smaller “chunks” that are called by a parent function. Each of these chunks are setup with InternalLinkage, and the parent is setup with ExternalLinkage. The parent and all resulting children chunks are passed back to the caller in the list of llvm::Function pointers. The llvm::Function pointers are stored in the caller threads.

This list is then run through JIT by calling getPointerToFunction() for each element. All llvm::Function pointers are kept around for deallocation later. Each thread client uses these generated functions for a while, then frees them all by calling freeMachineCodeForFunction() and eraseFromParent() on each llvm::Function* it allocated. Each worker thread then allocates another set of functions and JITs them, continuing indefinitely for the life of the application.

All of the thread interaction around LLVM is protected by locks. Only 1 thread can be constructing a function and calling the JIT at any given time. The same is true for freeing the function.

Now, the problem. The application runs indefinitely with a single client thread constructing and destructing LLVM functions. Since this is true, I’m sure there’s no memory leaking in the executable 16MB machine code buffer from forgetting to deallocate functions. As soon as a 2nd thread gets involved, however, the application runs for a few hundred iterations and then I get a JIT: ran out of space for machine code error from LLVM.

At first I would suspect some bad thread interaction in my client code, however I’ve expanded the scope of locks so that only 1 thread can talk to the LLVM singleton at a time. I have spent considerable effort ensuring this is true.

So my question is, is there anything inside the LLVM JITEmitter that could be thread aware? What if multiple threads are JIT’ing functions to the same module with the same basic block names (presumably these would get renamed automatically?).

At this point I am at a loss to understand how the LLVM JIT could be acting any differently with multiple threads calling JIT routines, than in a single-threaded environment, unless the identical IR build routines executing for multiple threads are creating some sort of conflicting state within the list of internal FunctionBlocks or something of that nature. Any and all ideas would be appreciated.

Thanks much,
Eric Yew