Quick question: How to BuildMI mov64mi32 arbitrary MMB address to memory

Project Infrastructure LLVM Dev List Archives
1

Dear All,

I am working on a x86 backend machineFunction pass, where I am trying to save the hard coded address of an arbitrary machine basic block to memory in ASM. I know the assembly needed for this but am lost at how to construct the BuildMI().

Note that these machine basic blocks are not entry’s to a function. but in the middle of the function. so using addGlobalAddress or addExternalSymbol are no use to me and am trying to avoid having to make a separate jump table section. Is a jump table section with symbols associated with these new basic blocks the only option?

In x86 assembly this would look something like:

MOVQ 0x40044540, 0x8(%rsp) # Store address of trampoline basic block to stack

The BuildMI looks like:

BuildMI(MBB, MBIt, DL, TII->get(X86::MOV64mi32))
.addImm(0x1) // Scale
.addReg(X86::RSP) // Base
.addImm(0x8) // Disp
.addMBB(my_target_mbb); // Source

So far I have looked into the BuildMI API of LLVM and the only one that looks relevant is addMBB. While my LLVM pass compiles, my linker complains (and note that I am compiling with -fPIC):

/usr/bin/ld: /tmp/foo-d523b6.o: Unknown temporary symbol

or 2)

/usr/bin/ld: /tmp/foo-d523b6.o: relocation R_X86_64_32S against `.text’ can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output

Any suggestions? Much appreciated for taking a look!

Sincerely,

K Jelesnianski

2

Hi,

/usr/bin/ld: /tmp/foo-d523b6.o: relocation R_X86_64_32S against `.text' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output

The issue is that you're presumably building in position-independent
mode (PIC). Shared libraries are almost always built like this, and a
lot of executables these days to aid ASLR.

Anyway, because of that you can't directly reference the absolute
address of a basic block from the .text section because that's
supposed to be read-only and so couldn't be fixed up by ld-linux or
dyld.

The solution is to instead calculate the address of the block relative
to %rip, which can't be stored in the same instruction (it'd have two
addressing-mode operands, which no x86 instruction does). So instead
it'd be a lea/mov pair (and you'll need a scratch register).

To get the exact form, it's useful to compile a simple .ll file:

    define void @foo(i8** %addr) {
      br label %next
    next:
      store i8* blockaddress(@foo, %next), i8** %addr
      ret void
    }

$ bin/llc -relocation-model=pic simple.ll -o -
[...]
        leaq .Ltmp0(%rip), %rax
        movq %rax, (%rdi)

You seem to be reasonably OK with BuildMI so I'm not going to try and
translate that to C++. And yes, addMBB is the one you want for that
operand.

The BuildMI looks like:

BuildMI(MBB, MBIt, DL, TII->get(X86::MOV64mi32))
.addImm(0x1) // Scale
.addReg(X86::RSP) // Base
.addImm(0x8) // Disp
.addMBB(my_target_mbb); // Source

This looks like some operands might be wrong to me. Aren't the x86
addressing-mode operands [base, scale, index,disp, segment]?

Cheers.

Tim.

3

Dear Mr. Northover,

Thank you for the quick reply. You are correct about the address-mode operands :slight_smile: . I guess an important detail left out was that the basic block (call it A) that wants to calculate the address of the target stationary trampoline basic block (call it B) will be moved around in memory during run-time. Our earlier solution, before the feature was implemented to move around (A) is exactly as you explained using the following with a scratch reg:

$ bin/llc -relocation-model=pic simple.ll -o -
[…]
leaq .Ltmp0(%rip), %rax
movq %rax, (%rdi)

We now run into the problem that with this feature enabled, if we try to perform LEA backwards to the trampoline, after A has been moved the %rip relative offset that was put in by LLVM is no longer valid. Also if we perform LEA forwards to the target address in A that trampoline B is supposed to trampoline us too, that address will also be invalidated once A has been moved. Thus calculating forwards is most likely impossible. This leaves calculating LEA backwards to the trampoline BB (B) since we know that the trampoline BB will remain stationary throughout execution. That is why I would “like” to somehow store (B)'s address.

I am looking for a work around to accommodate this feature. I have never attempted to make my own section/symbols using LLVM, but I assume this is the route I should take? Pairing each trampoline BB to a symbol should make it visible and this MOVQ instruction I want possible?

With that approach, my questions are:
Do I need to make these symbols for the trampoline BBs as an IR opt pass, can I get away with it using a MachineModule Pass to add the trampolines per module (file) (so far I have only created BasicBlock, MachineBasicBlock, and MachineFunction passes)??
Do I need to make a separate custom section for these trampolines symbols, or can I just add them to the .text section?

Thanks again for your reply.

Sincerely,

K Jelesnianski

4

I also confirm I am building my executables and libraries with “-fPIC -pie” CFLAGS to be able to take advantage of ASLR.

5

Hi,

I am looking for a work around to accommodate this feature. I have never attempted to make my own section/symbols using LLVM, but I assume this is the route I should take? Pairing each trampoline BB to a symbol should make it visible and this MOVQ instruction I want possible?

You've still got to access that symbol, and it's not obvious how a
block that's moving around in memory could do that. The same arguments
that it can't use %rip relative addressing for a local BB would seem
to apply to any other entity.

If you have a solution for that problem, an alternative to creating
entirely new symbols would be to reference the stationary BB relative
to the function's entry-point. In assembly something like:

    movq func@GOTPCREL(%rip), %rax
    addq (.LBB0_0-func), %rax

Do I need to make these symbols for the trampoline BBs as an IR opt pass, can I get away with it using a MachineModule Pass to add the trampolines per module (file) (so far I have only created BasicBlock, MachineBasicBlock, and MachineFunction passes)??

If you go that route you can probably add entries to the
MachineConstantPool with a MachineModule pass. The same addressing
concerns seem to apply though.

Cheers.

Tim.

6

Dear Dr. Northover,

You’ve still got to access that symbol, and it’s not obvious how a
block that’s moving around in memory could do that. The same arguments
that it can’t use %rip relative addressing for a local BB would seem
to apply to any other entity.

Agreed, for now I will probably perform need to perform load-time analysis, grab some info, and patch the binary to get around this. I have gone ahead and begun implementing your proposed work around.

My question now is how would you create the BuildMI for the second assembly instr (addq (LBB0_0 - func), %rax) you proposed? I know I can get the global address of the MF we are currently in with .addGlobalAddress(M->getNamedValue(MF.getName())) but how do we take that and make an expression out of it. I am not sure the MBB object gives us any way to get its address and perform the given subtraction expression of LBB0_0 - func.

So far I have the first instruction working:

movq func@GOTPCREL(%rip), %rax

const Module M = MF.getMMI().getModule();
/ movq func@GOTPCREL(%rip), %rax */
BuildMI(MBB, MBIt, DL, TII->get(X86::MOV64rm))
.addReg(X86::RAX) //dest
.addReg(X86::RIP) //base
.addImm(0x1) //scale
.addReg(0x0) //index
.addGlobalAddress(M->getNamedValue(MF.getName())) //Disp
.addReg(0x0); //seg

addq (.LBB0_0-func), %rax

/* addq (.LBB0_0-func), %rax ??? */
BuildMI(MBB, MBIt, DL, TII->get(X86::ADD64ri32))
.addReg(X86:RAX) //destination
.addReg(X86::RAX) //base
.addImm(0x1) //scale
.addReg(0x0) //index

.addImm(<<<< I assume expression is related to displacement and goes here >>>>>>>)

.addReg(0x0); //segment

If I try to put a simple asm.s into llvm-mc -show-inst, it tells me to use MCExpr, but I am not sure that is correct (shown below). There does exist .addExpr but it only valid for MCInstBuilder, not MachineInstrBuilder::BuildMI.

$ llvm-mc -show-inst asm.s
foo:
.LBB0_0:
movq 2099957(%rip), %rax # <MCInst #1810 MOV64rm

>

addq (.LBB0_0 - foo) , %rax # <MCInst #202 ADD64rm

<MCOperand Expr:(.LBB0_0-foo)>

>

retq # <MCInst #2601 RETQ>

Do I need to make these symbols for the trampoline BBs as an IR opt pass, can I get away with it using a MachineModule Pass to add the trampolines per module (file) (so far I have only created BasicBlock, MachineBasicBlock, and MachineFunction passes)??
If you go that route you can probably add entries to the
MachineConstantPool with a MachineModule pass. The same addressing
concerns seem to apply though.

I actually forgot about this constraint, we already have experienced some unintended side-effects when attempting to reference .rodata information (e.g. printf printing garbage). So that is something to look fix in the near future once this part is done. :slight_smile:
Thanks again for your reply!

Sincerely,

K Jelesnianski

7

In the mean time I thought I could do the inverse and “sum” the size of each instruction between the entry MBB and the target .LBB0_0 by doing a double for loop
for each MBB
for each MInstr
currInstrBytes = getInstSizeInBytes(MInstr);
totalAsmbytes += currInstrBytes;

Unfortunately X86-64 is missing a very convenient function called "getInstSizeInBytes( MachineInstr* ) that I found in AAarch64InstrInfo. ;/ So I’m still unsure how to proceed.

8

Yes, I don't think you can put an MCExpr on a MachineInstr so you'll
probably need a pseudo-instruction that's expanded in
X86MCInstLower::Lower (when the MachineInstr gets converted to an
MCInst). You can use MachineBasicBlock::getSymbol and probably
something like GetSymbolFromOperand for the function (i.e. global)

Cheers.

Tim.