Trying to use patchpoint in MCJIT

Hi all,

I'm trying to play with patchoint (with MCJIT and VMKit) and I don't
understand something. I generate this call for my first patch point.
Basically, I want to call f(0).

%5 = call i64 (i64, i32, i8*, i32, ...)* @llvm.experimental.patchpoint.i64(
  i64 42, ;; patch point id is 42
  i32 0, ;; 0 bytes for the padding
  i8* bitcast (i32 (i32)* @f to i8*), ;; my function f
  i32 1, ;; 1 argument
  i32 0) ;; the argument

I think that the types are correct (at least, the compiler seems happy
with that:)). But when I'm trying to compile the code (that I generate
from C++, it's not a bitcode file), I have this message:

Assertion failed: (isa<X>(Val) && "cast<Ty>() argument of incompatible
type!"), function cast, file
/Users/gthomas/research/vmkit4/llvm-src/include/llvm/Support/Casting.h,
line 232.

With this stack (I only give the relevant part):

    frame #5: 0x000000010092ed44
libjvm.dylib`llvm::SelectionDAGBuilder::visitPatchpoint() + 3780 at
SelectionDAGNodes.h:610
    frame #6: 0x000000010091f511
libjvm.dylib`llvm::SelectionDAGBuilder::visitIntrinsicCall(this=0x0000000106ff4d20,
I=0x0000000106fea460) + 10961 at SelectionDAGBuilder.cpp:5349
    frame #7: 0x0000000100904c4a
libjvm.dylib`llvm::SelectionDAGBuilder::visitCall(this=0x0000000106ff4d20,
I=0x0000000106fea460) + 234 at SelectionDAGBuilder.cpp:5870
    frame #8: 0x00000001008fc9ea
libjvm.dylib`llvm::SelectionDAGBuilder::visit(this=0x0000000106ff4d20,
I=0x0000000106fea460) + 74 at SelectionDAGBuilder.cpp:971
    frame #9: 0x0000000100947e98
libjvm.dylib`llvm::SelectionDAGISel::SelectBasicBlock(this=0x0000000106ff40d0,
HadTailCall=0x00007fff5fbfe31e) + 40 at SelectionDAGISel.cpp:579
    frame #10: 0x0000000100947aee
libjvm.dylib`llvm::SelectionDAGISel::SelectAllBasicBlocks(this=0x0000000106ff40d0,
Fn=0x0000000106fe7020) + 8174 at SelectionDAGISel.cpp:1192
    frame #11: 0x0000000100944b94
libjvm.dylib`llvm::SelectionDAGISel::runOnMachineFunction(this=0x0000000106ff40d0)
+ 1588 at SelectionDAGISel.cpp:434
    frame #12: 0x0000000100afefc0
libjvm.dylib`llvm::MachineFunctionPass::runOnFunction() + 128 at
MachineFunctionPass.cpp:33
    frame #13: 0x00000001010837fb
libjvm.dylib`llvm::FPPassManager::runOnFunction(this=0x0000000106fedf20,
F=0x0000000106fe7020) + 331 at LegacyPassManager.cpp:1538
    frame #14: 0x0000000101083abb
libjvm.dylib`llvm::FPPassManager::runOnModule(this=0x0000000106fedf20)
+ 59 at LegacyPassManager.cpp:1558
    frame #15: 0x000000010108402d
libjvm.dylib`llvm::legacy::PassManagerImpl::run(llvm::Module&)
[inlined] llvm::legacy::PassManagerImpl::getContainedManager(N=0,
this=0x0000000106feb9f0) + 429 at LegacyPassManager.cpp:1616
    frame #16: 0x0000000101083e80
libjvm.dylib`llvm::legacy::PassManagerImpl::run(this=0x0000000106feb9f0,
M=0x0000000106fe6f30) + 544 at LegacyPassManager.cpp:1711
    frame #17: 0x000000010108466d
libjvm.dylib`llvm::legacy::PassManager::run() + 13 at
LegacyPassManager.cpp:1746
    frame #18: 0x0000000100eeaf8c
libjvm.dylib`llvm::MCJIT::emitObject(this=0x0000000106fa69a0,
M=0x0000000106fe6f30) + 236 at MCJIT.cpp:131
    frame #19: 0x0000000100eeb2d3
libjvm.dylib`llvm::MCJIT::generateCodeForModule(this=0x0000000106fa69a0,
M=0x0000000106fe6f30) + 323 at MCJIT.cpp:169
    frame #20: 0x0000000100eec022
libjvm.dylib`llvm::MCJIT::getFunctionAddress(std::string const&)
[inlined] llvm::MCJIT::getSymbolAddress(this=0x0000000106fa69a0,
CheckFunctionsOnly=true) + 58 at MCJIT.cpp:279
    frame #21: 0x0000000100eebfe8
libjvm.dylib`llvm::MCJIT::getFunctionAddress(this=0x0000000106fa69a0,
Name=0x00007fff5fbfe7b0) + 40 at MCJIT.cpp:295
...

So, I don't understand where could be my mistake?

Thank you in advance,
Gaël

It appears patchpoint doesn't yet support function symbols as the target:

SelectionDAGBuilder.cpp:6949:

test.ll (458 Bytes)

patchpoint is intended to be used in VMs that do their own linking, and so you wouldn’t expect the function parameter to be resolved by LLVM.

Presumably, in VMKit, you could just plant a pointer constant to the function you wish to call initially?

-Filip

Ok I see. Of course, at runtime, it’s enough for dynamic linking or for deoptimization. However, wmkit acts both as a jit and as an aot. For the aot, it means that I can not use patchpoint and that I should have two different compilation strategy. It’s not so difficult, but in this case, I can not use patchpoints to generate gc’s stackmap for the aot (basically, I think that I could use a patchpoint to generate a safepoint where I know the locations of my roots by ysing them as arguments of the patchpoint?)?

Do you think that using a symbol for my targets is easy to implement? Or maybe that there is a conceptual problem for symbols in this case?

By the way, were can I find the metadata at the end of the compilation? They are generated in a special symbol/section?

Anyway, thank you for your answer,
Gaël

Ok I see. Of course, at runtime, it’s enough for dynamic linking or for deoptimization. However, wmkit acts both as a jit and as an aot. For the aot, it means that I can not use patchpoint and that I should have two different compilation strategy. It’s not so difficult, but in this case, I can not use patchpoints to generate gc’s stackmap for the aot (basically, I think that I could use a patchpoint to generate a safepoint where I know the locations of my roots by ysing them as arguments of the patchpoint?)?

Yes that makes sense.

Note that you can pass a NULL for the callee argument to emit an “inactive” safepoint and have the runtime patch a call in its place later. I suspect that you’re not actually interested in patching the code though. You probably just want to find the location of roots at an existing call site. To do this, your runtime needs to be able to reliable unwind the stack restoring callee save registers.

Do you think that using a symbol for my targets is easy to implement? Or maybe that there is a conceptual problem for symbols in this case?

It would be great if patchpoint supported symbols. I’m not aware of any fundamental problem with doing it. There are a few tricky pieces of code that need to be reworked.

e.g.
SelectionDAGBuilder::visitPatchpoint should allow a Function type operand.

LowerPATCHPOINT should recognize the symbolic operand and emit the right kind of call (maybe CALL64pcrel32).

I’ll be happy to review patches.

By the way, were can I find the metadata at the end of the compilation? They are generated in a special symbol/section?

Please see the docs here.
http://llvm-reviews.chandlerc.com/D1981

I know I should check this in now. There are just a few offline review comments I should to address. In particular:

For a patchpoint with a non-null callee, the runtime must reserve enough space to encode the call under all conditions (on x86 this could result in an extra nop because we can’t predict the REX prefix ahead of time).

When the runtime patches the code at a patchpoint, it must patch all reserved bytes. It cannot assume that LLVM emits a particular call sequence or flavor of nops.

Someone recently suggested emitting the call at the end of the reserved bytes (nops before the call). I think that’s a good idea. If we do that we may allow the runtime to assume the return address is one byte past the end of the reserved bytes.

-Andy

Ok I see. Of course, at runtime, it's enough for dynamic linking or for
deoptimization. However, wmkit acts both as a jit and as an aot. For the
aot, it means that I can not use patchpoint and that I should have two
different compilation strategy. It's not so difficult, but in this case, I
can not use patchpoints to generate gc's stackmap for the aot (basically, I
think that I could use a patchpoint to generate a safepoint where I know
the locations of my roots by ysing them as arguments of the patchpoint?)?

Yes that makes sense.

Note that you can pass a NULL for the callee argument to emit an
"inactive" safepoint and have the runtime patch a call in its place later.
I suspect that you're not actually interested in patching the code though.
You probably just want to find the location of roots at an existing call
site. To do this, your runtime needs to be able to reliable unwind the
stack restoring callee save registers.

Do you think that using a symbol for my targets is easy to implement? Or
maybe that there is a conceptual problem for symbols in this case?

It would be great if patchpoint supported symbols. I'm not aware of any
fundamental problem with doing it. There are a few tricky pieces of code
that need to be reworked.

e.g.
SelectionDAGBuilder::visitPatchpoint should allow a Function type operand.

LowerPATCHPOINT should recognize the symbolic operand and emit the right
kind of call (maybe CALL64pcrel32).

I’ll be happy to review patches.

By the way, were can I find the metadata at the end of the compilation?
They are generated in a special symbol/section?

Please see the docs here.
http://llvm-reviews.chandlerc.com/D1981

Andy, commit this already! It has LGTM from all three reviewers (although
for some reason Juergen's indicator at the top still shows it unreviewed,
but going through the comments he LGTM'd it).

-- Sean Silva