Dynamic Compilation

Hi Manoel,

CC’ing the dev list again.

When you say that you are using LLVMOrcJit, what do you mean? Are you constructing your own stack along the lines of the Kaleidoscope tutorials? Or are you using OrcMCJITReplacement? You should be able to do what you want with your own stack, but the OrcMCJITReplacement class has to replicate MCJIT’s behavior, which includes taking ownership of Modules.

Assuming you’re using your own stack, I have some observations:

For simple expressions, hopefully you can recognize forms that you have already compiled and reuse the cached code. Then there’s no additional work to do.

For expressions that haven’t been compiled yet I would expect the cost of compiling the expression to dwarf the cost of building the Module. Have you profiled your program? Is Module construction actually showing up as a source of high overhead?

If constructing Modules really is proving to be a big time-sink, some of the Orc layers* allow you to re-use a Module as many times as you like. The ModuleSet type for the addModuleSet methods is a template parameter, so whether or not you pass in ownership is up to you. E.g.

(1) Pass vector of unique_ptrs. Ownership of the Modules is transferred to the JIT, and Modules cannot be modified.
std::vector<std::unique_ptrllvm::Module> Modules;
// …
addModuleSet(std::move(Modules), …);

(2) Pass a vector of raw pointers. Ownership is not transferred to the JIT, and Modules can be modified and re-added:
std::vectorllvm::Module* Modules;
// …
addModuleSet(std::move(Modules), …);
// … Modify modules…
addModuleSet(std::move(Modules), …);

*The ownership models for each of the layers are as follows:

  1. ObjectLinkingLayer - Totally ownership agnostic. Object pointers discarded after use. If the pointer type is unique_ptr this will destroy the object in the process.

  2. IRCompilingLayer - Totally ownership agnostic. Module pointers are discarded after use. If the pointer type is unique_ptr the will destroy the module in the process.

  3. LazyEmittingLayer - Ownership agnostic API with contract: Modules can only be modified and re-added after they’ve been either (a) compiled, or (b) erased. Adding a module then modifying it before it is compiled or erased is undefined.

  4. CompileOnDemandLayer - Owning. Modules cannot be modified after being added.

From what I understand of your use case I think you want the simplest stack - just the ObjectLinking and IRCompiling layers. These should be totally compatible with retaining ownership and modify existing modules.