Question about SimplifyXorInst

Hi all,

I am master student in Edinburgh, UK. I am doing my MSc project with LLVM compiler and I have to modify LLVM to implement the StackGuard with a XOR random Canary. However, I am not familiar with LLVM.

My problem is that I want to XOR the random canary word with the return address which are both 32 bits. I found a method called SimplifyXorInst(Value *, Value *, const TargetData *,const DominatorTree *, unsigned) but I do not know whether it is right method and how to use it. Could you give me some advice or sample code about it?

Thank you
Best wishes,
Ying

Hi all,

I am master student in Edinburgh, UK. I am doing my MSc project with
LLVM compiler and I have to modify LLVM to implement the StackGuard
with a XOR random Canary. However, I am not familiar with LLVM.

This sounds similar to the StackProtector feature that LLVM already has.

My problem is that I want to XOR the random canary word with the
return address which are both 32 bits. I found a method called
SimplifyXorInst(Value *, Value *, const TargetData *,const
DominatorTree *, unsigned) but I do not know whether it is right
method and how to use it. Could you give me some advice or sample code
about it?

I have no idea what SimplifyXorInst() does, but since the address of the return address is not visible at the LLVM IR level (to the best of my knowledge), it is probably not what you want.

The most straightforward thing to do, I think, is to write a MachineFunctionPass that adds prologue/epilogue code during code generation to do the xor'ing of the return address.

I haven't written a MachineFunctionPass, so I can't really help with any details. My best suggestion is to find the StackProtector code in LLVM and see how it works. The doxygen docs are also valuable (http://llvm.org/doxygen/hierarchy.html).

-- John T.

Hi Ying,

Like John said, it sounds like you want to modify the stack protector code. Look at this file:

  lib/CodeGen/StackProtector.cpp

and see if it could help you out. The SimplifyXorInst function is part of the "Instruction Combine" pass, which is an optimization pass. I doubt that it's the correct place to do what you're suggesting doing.

-bw