MallocInst/CallInst bitcast,

Hello,

I’m writing a virtual machine that functions as a sandbox based on llvm. In order to prevent programs from accessing memory that has not been allocated to them, I want to replace calls to malloc and free with calls to a logged functions that will record the memory that is being allocated to the program. Is it possible to cast/convert a MallocInst or FreeInst to a CallInst?

Thanks,

Daniel

Never mind, I used ExecutionEngine’s InstallLazyFunctionCreator and DisableSymbolSearching to cause malloc and free calls to be handled by my logging functions. Sorry for the unnecessary list mail.

Is it possible to find out the size and beginning pointer of the current stack frame, from a function operating outside of the virtual machine, but called by a function within it?

Thanks,

Daniel

2009/10/16 Daniel Waterworth <da.waterworth@googlemail.com>

Never mind, I used ExecutionEngine’s InstallLazyFunctionCreator and DisableSymbolSearching to cause malloc and free calls to be handled by my logging functions. Sorry for the unnecessary list mail.

No problem, this is a better way to go. The MallocInst and FreeInst instructions are about to be removed from LLVM IR. Malloc and free will be represented normal ‘call’ instructions.

Is it possible to find out the size and beginning pointer of the current stack frame, from a function operating outside of the virtual machine, but called by a function within it?

I don’t think so. Typically, applications that need this sort of thing get the address of the function and then do a fuzzy disassembly of the prolog.

-Chris

Thanks very much. I only have one more question, (hopefully), which is, is there a better way of finding the direction of stack growth other than:

static bool StackCmp(void *ptr) {
volatile int a;
return (void *)&a > ptr;
}

bool FindStackDirection() {
volatile int a;
return StackCmp((void *)&a);
}

Preferably one which isn’t destroyed by optimization.

Thanks again,

Daniel

2009/10/16 Chris Lattner <clattner@apple.com>

Thanks very much. I only have one more question, (hopefully), which is, is there a better way of finding the direction of stack growth other than:

That’s a somewhat reasonable, but fragile way to go. Make sure to mark StackCmp with attribute(noinline). Don’t be too surprised if it doesn’t work on some compiler. I don’t know a better way to do it though.

-Chris

Daniel Waterworth skrev:

Thanks very much. I only have one more question, (hopefully), which is, is there a better way of finding the direction of stack growth other than:

static bool StackCmp(void *ptr) {
  volatile int a;
  return (void *)&a > ptr;
}

bool FindStackDirection() {
  volatile int a;
  return StackCmp((void *)&a);
}

Preferably one which isn't destroyed by optimization.

I suggest you turn the scalars into arrays and make the ptr argument volatile as well.

Other ways: If you are careful with tail recursion eliminiation, you can compare local var addresses from different recursive calls. I believe there are va_arg based approaches as well.

That said, there is no truly portable way.

/Stein Roger

2009/10/16 srs <skaflotten@gmail.com>

Daniel Waterworth skrev:

Thanks very much. I only have one more question, (hopefully), which is, is there a better way of finding the direction of stack growth other than:

static bool StackCmp(void *ptr) {
volatile int a;
return (void *)&a > ptr;
}

bool FindStackDirection() {
volatile int a;
return StackCmp((void *)&a);
}

Preferably one which isn’t destroyed by optimization.

I suggest you turn the scalars into arrays and make the ptr argument volatile as well.

Other ways: If you are careful with tail recursion eliminiation, you can compare local var addresses from different recursive calls. I believe there are va_arg based approaches as well.

That said, there is no truly portable way.

/Stein Roger

I’ve changed the ptr argument to volatile and the StackCmp function to noinline. It’s now providing the correct answer in gcc and llvm-gcc with optimization on -O3. This will suite my puposes, at least for the time being.

Thanks alot,

Daniel