Implement a shared-memory based JITLinkMemoryManager for out-of-process JITting

Community GSoC
#1

Write a shared-memory based JITLinkMemoryManager.

LLVM’s JIT uses the JITLinkMemoryManager interface to allocate both working memory (where the JIT fixes up the relocatable objects produced by the compiler) and target memory (where the JIT’d code will reside in the target). JITLinkMemoryManager instances are also responsible for transporting fixed-up code from working memory to target memory. LLVM has an existing cross-process allocator that uses remote procedure calls (RPC) to allocate and copy bytes to the target process, however a more attractive solution (when the JIT and target process share the same physical memory) would be to use shared memory pages to avoid copies between processes.

Expected results

Implement a shared-memory based JITLinkMemoryManager:

  • Write generic LLVM APIs for shared memory allocation.
  • Write a JITLinkMemoryManager that uses these generic APIs to allocate shared working-and-target memory.
  • Make an extensive performance study of the approach.

Desirable skills

Intermediate C++; Understanding of LLVM and the LLVM JIT in particular; Understanding of virtual memory management APIs.

Project type

Large

Mentors:

@vvassilev, @lhames

2 Likes
#2

Hello, I want to participate in GSoC with this idea.

I have been going through JITLinkMemoryManager class and related code. I want to confirm my understanding and also have a few questions.

LLVM has an existing cross-process allocator that uses remote procedure calls (RPC) to allocate and copy bytes to the target process

Where is the code for this? Does this refer to EPCGenericJITLinkMemoryManager?

Write generic LLVM APIs for shared memory allocation.

Does it mean functions similar to sys::Memory::allocateMappedMemory etc. for shared memory?

Write a JITLinkMemoryManager that uses these generic APIs to allocate shared working-and-target memory.

Does it mean implementing something similar to InProcessMemoryManager but with shared memory using this new API?

Finally what other resources, code etc. should I look at and what can I do to get more familiar with the relevant things?

Thank you.

#3

Hi @argentite!

Where is the code for this? Does this refer to EPCGenericJITLinkMemoryManager ?

Yes. :slight_smile:
You’ll find the code for the client side in llvm/lib/ExecutionEngine/Orc/EPCGenericJITLinkMemoryManager.cpp (and associated header), and the server code in llvm/lib/ExecutionEngine/Orc/TargetProcess/SimpleExecutorMemoryManager.cpp (and associated header).

Does it mean functions similar to sys::Memory::allocateMappedMemory etc. for shared memory?

That’s not entirely clear yet. It may end up looking like sys::Memory, but the design should be driven by the needs of the JITLinkMemoryManager API, and the solution might end up looking like a more elaborate version of llvm::SectionMemoryManager::MemoryMapper.

There is an orthogonal project (one that might end up being folded into this one if you’re interested) to create a slab-based allocator – an allocator that reserves chunks of memory up-front to enable use of the default code models (which assume that all code/data for the library is located within a narrow address range). I think an ideal off-the-shelf JITLink memory manager would look something like this:

MemoryMapper (reserve slabs, apply protections, release slabs) +
SlabManager (allocates and tracks memory within slabs) +
SideChannel (sends commands to run finalization actions);

Where the MemoryMapper could be implemented via shared memory (where available) or EPC (where shared memory is not an option).

Finally what other resources, code etc. should I look at and what can I do to get more familiar with the relevant things?

Do you already know the shared memory APIs for your OS? If not, I think I’d start there. If you’re able to play around with them on multiple OSes that’s even better.

Then I would prototype the ideas using the llvm-jitlink and llvm-jitlink-executor tools. You could do this by hacking SimpleEPCJITLinkMemoryManager and SimpleExecutorMemoryManager (or making copies and hacking those, if you wanted this to be something that you could develop in-tree).

I hope that gives you some ideas to get started, but I know it’s also quite high level. If you’re looking for more specifics I’m happy to answer any follow-up questions. :slight_smile:

#4

Thank you. :slight_smile:

Do you already know the shared memory APIs for your OS? If not, I think I’d start there. If you’re able to play around with them on multiple OSes that’s even better.

I have previously used POSIX shared memory API under Linux before. I assume it almost works identically on macOS. I just tried the Windows shared memory API yesterday as well.

Currently I am playing around with llvm-jitlink and llvm-jitlink-executor under a debugger. I will try to test some kind of prototype shared memory based system next.

I think including the slab allocator will be a good idea as the project is categorized as large.

#5

Hi @argentite, can you also send us your cv (apologies if I missed that email) privately. I would like to encourage you to start thinking about writing a proposal.

#6

I have made a very dirty proof of concept POSIX shared memory based allocator for testing by directly modifying the SimpleExecutorMemoryManager and EPCGenericJITLinkMemoryManager as I don’t know how the final structure should be. It seems to be working with c-ray as example code on Linux.

I have sent the email.

#7

Hi Argentite,

I just checked this out locally – very cool!

I don’t know how the final structure should be.

I think that is an open question in this project, but we know the general outline:

  1. We want to encapsulate the shared memory mapping in a new class – a sort of SectionMemoryManager::MemoryMapper analogue for shared memory.

  2. We want a generic memory manager written against this mapper type.

  3. We want the generic memory manager to reserve larger slabs of address space up front (e.g. on the first call to allocate for a given JITLinkDylib), and then hand out sub-ranges from these slabs. This has two advantages: (1) because the slabs are contiguous we can safely use more efficient code-models, and (2) allocation may be cheaper (depending on the specifics of the mapper used), since we don’t need to communicate with the executor on every allocate call (only every finalize call).

We’ll want to leave EPCGenericJITLinkMemoryManager as-is in-tree, so we can create a copy if you decide to take on the project and work with that. :slight_smile:

#8

Hello,

I have started drafting a proposal based on the template posted in another thread. Please provide feedback/suggestions. I am unsure about whether it makes any sense at all, especially the timeline. Is the level of detail adequate?

#9

Can you change the access to leave comments?

#10

Sorry. I have done that.