I’m using MCJIT to run some loops on my ARM processor.
I was trying to perform some runtime optimizations on some function, and this requires recompiling the function at runtime.
I know that this feature is not available yet in MCJIT , and to modify a function I have to create a new module with the newly optimized code.
My questions are:
The newly created module can be added to the same execution engine or I have to create a new one?
Is it possible to swap the execution from the running module to the new optimized one at runtime? If yes, how? If no, are there any further solutions?
I don’t believe this is supported in any way other than to load everything up anew and start again.
I know lang has some ideas about how to add spur for this and may have some more precise description of the current state of affairs.
When you add a new definition of an existing symbol to an EE instance it replaces the old definition’s address in the EE’s symbol table. All uses of a symbol ‘foo’ that are finalized after foo is updated will refer to the new version of foo. All uses that are finalized before that will refer to the original definition of foo.
So the answer to your first question is yes: You can add the newly created module to the same execution engine.
The answer to your second question is “sort of”: New code will get to use the newly optimized function, old code will use the original version.
If you want all of your code to refer to the optimized version of foo you have two options at the moment:
(1) Recompile all functions that transitively use foo. If you take this approach you’ll want to pull all users of foo in to one module to ensure that any references between these functions resolve to the newly compiled versions. This approach will allow direct calls to foo, but will consume compile-time (you have to recompile everything) and memory (since the old definitions can never be released).
(2) Make sure that all references to ‘foo’ are made indirectly through a global function pointer. That way when you recompile ‘foo’ you can just update the global function pointer to make all calls to foo point to the newly compiled version. This will minimize your compile time and memory consumption at the cost of requiring indirect calls to foo. If your language contains function pointers (and allows them to be tested for equality) then you’ll also need to be careful here: Wherever you would have taken the address of ‘foo’ as a function pointer, you’ll need to use your ‘foo_addr’ global instead, otherwise function pointer equality will break when you recompile foo.
If you want to be able to release memory for old versions of foo it would be worth taking a look at Andy Kaylor’s Kaleidoscope/MCJIT blog posts. He devised a scheme that allowed symbol resolution between execution-engine instances, which allowed (among other things) memory for compiled functions to be released without tearing down your entire JIT’d program.
As Dave said - I’m working on some new JIT APIs to better support use cases like yours, but I’m afraid I don’t have a clear ETA for them yet.