MCJIT and Lazy Function Creators

Out of curiosity, I'm replacing the JIT with MCJIT on my compiler. As
all "external" functions are provided by the language's FFI mechanism,
it does

MyExecutionEngine->DisableSymbolSearching();
MyExecutionEngine->InstallLazyFunctionCreator(&MyLazyFunctionCreator);

which works fine with the JIT. However, MCJIT insists on resolving
unknown symbols by searching them outdoors and seems oblivious of the
existence of MyLazyFunctionCreator.

Is this a bug?

Is there a workaround, other than subclassing JITMemoryManager and
implementing my own getPointerToNamedFunction? (which would make the
existence of ExecutionEngine::InstallLazyFunctionCreator pointless.)

Óscar Fuentes <ofv@wanadoo.es> writes:

Out of curiosity, I'm replacing the JIT with MCJIT on my compiler. As
all "external" functions are provided by the language's FFI mechanism,
it does

MyExecutionEngine->DisableSymbolSearching();
MyExecutionEngine->InstallLazyFunctionCreator(&MyLazyFunctionCreator);

which works fine with the JIT. However, MCJIT insists on resolving
unknown symbols by searching them outdoors and seems oblivious of the
existence of MyLazyFunctionCreator.

Is this a bug?

Is there a workaround, other than subclassing JITMemoryManager and
implementing my own getPointerToNamedFunction? (which would make the
existence of ExecutionEngine::InstallLazyFunctionCreator pointless.)

After working around the issue mentioned above by subclassing
JITMemoryManager, I hit another roadblock:

Full-featured argument passing not supported yet!
UNREACHABLE executed at /home/oscar/dev/llvm/llvm/lib/ExecutionEngine/MCJIT/MCJIT.cpp:272!

That's from MCJIT::runFunction. The argument has type [i x i8]*, but
there are llvm_unreachable calls for other cases, including varargs,
integer types larger than 64 bits and long doubles.

I guess we'll have to add that to the list of things that needs to be done to replace the old JIT.

In the projects I've worked on that use MCJIT, we've been using getPointerToFunction and then calling it from outside the MCJIT engine, but I suppose that the generic handling in runFunction would be useful in some cases.

-Andy

"Kaylor, Andrew" <andrew.kaylor@intel.com> writes:

I guess we'll have to add that to the list of things that needs to be
done to replace the old JIT.

In the projects I've worked on that use MCJIT, we've been using
getPointerToFunction and then calling it from outside the MCJIT
engine, but I suppose that the generic handling in runFunction would
be useful in some cases.

FWIW, in my language `main' and other externally-callable function does
not have a fixed signature: it can take and return any type(s) from a
set of well-known types. Most test cases here are failing because, for
returning a string, `main' is called with a pointer parameter of type
[i8 x 8]* pointing to the space reserved for the returned value. Even
that simple case is not supported by MCJIT::runFunction. Looking at how
JIT::runFunction implements the generic case I can see why MCJIT can't
do the same: JIT::runFunction creates and calls a stub function on the
fly, which is then used for calling the real function. This is not
doable with MCJIT, at least not easily.

I could do what you describe: get the pointer to the function and
implement all the handling for my language's supported types and a
reasonably large number of parameters. That sort of exercise on
combinatorics could work for me, but others would be forced to replicate
the work for they own projects and that's precisely what a library
should avoid. And, possibly, on some cases a user might need a truly
generic method: think on a paremeter or return value with user-defined
type (`user' here means the guy who wrote the code being compiled, not
the compiler's writer.)

Yes, I agree that it's better to have it done correctly in the library.