[RFC] MCJIT usage models

Another usage case, slightly different than your #1:

6. Dynamic code generation (not interactive with respect to the source code)
   - app generates code IR which is compiled as needed for execution
  (important difference: no waiting for a typing human in the loop,
   so very different expectations about responsiveness and bottlenecks)
   - compilation speed IS critical (because many such compilations may be
     necessary and the app itself may be interactive, even if the source code
     entry is not)
   - retained memory use is critical -- ok to use much mem while JITing, but once
     we've reduced it to callable machine code function, we want to reclaim
     as much as possible of the memory needed by LLVM.
   - app is multithreaded and does not want to lock on a single MCJIT --
     ok to have multiple MCJIT's, one per app thread, needing to run
     simultaneously.

I'm currently using the old JIT, but would happily switch to MCJIT if its speed (especially ability to lazily compile functions as the old one can) were to improve and memory was as good.

(Here's the GitHub page for the project I'm thinking of: https://github.com/imageworks/OpenShadingLanguage/)

Greetings,

Larry Gritz wrote:

Another usage case, slightly different than your #1:

6. Dynamic code generation (not interactive with respect to the source code)
   - app generates code IR which is compiled as needed for execution
  (important difference: no waiting for a typing human in the loop,
   so very different expectations about responsiveness and bottlenecks)
   - compilation speed IS critical (because many such compilations may be
     necessary and the app itself may be interactive, even if the source code
     entry is not)
   - retained memory use is critical -- ok to use much mem while JITing, but once
     we've reduced it to callable machine code function, we want to reclaim
     as much as possible of the memory needed by LLVM.
   - app is multithreaded and does not want to lock on a single MCJIT --
     ok to have multiple MCJIT's, one per app thread, needing to run
     simultaneously.

For what it's worth, the above describes my usage case as well. (With the
exception that at present the app uses only a single JIT thread.)

Regards,

Bill

Same here, in fact I’d imagine #6 is the case for many DSLs.

v/r,
Josh

Hi Larry,

Thanks for the detailed description.

Have you done any speed and memory tests with MCJIT? I'm curious how far it is from meeting your needs.

-Andy