JIT and security

Hello,

I'm writing JIT compiler that will run a third party code. My goal is to build it with security layer that will allow me to prevent some basic operation that possibly can harm JIT application host computer.

Maybe some of you can guide me on how to do following operations:
  1) prevent system calls
  2) memory allocation management (set some limits that can't be used by JIT)
  3) CPU power limiting

I have a guess on how to prevent system calls or some known function calls by parsing IR and picking up functions calls that maybe unwanted. Another way is to check list of functions in object model. Maybe there is already implemented mechanism like that. So, anyone could give me a clue.

About RAM and CPU management I have no idea how to do that.

Ideally my JIT compiler have to become a BlackBox, that hosting party can configure to his needs.

I hope I can find some help here. That would be great!

Thank you for your time,
Oleg.

Hello,

I'm writing JIT compiler that will run a third party code. My goal is
to build it with security layer that will allow me to prevent some
basic operation that possibly can harm JIT application host computer.

Maybe some of you can guide me on how to do following operations:
1) prevent system calls
2) memory allocation management (set some limits that can't be used
by JIT)
3) CPU power limiting

I have a guess on how to prevent system calls or some known function
calls by parsing IR and picking up functions calls that maybe
unwanted. Another way is to check list of functions in object model.
Maybe there is already implemented mechanism like that. So, anyone
could give me a clue.

Hi Oleg,

I don't think that a JIT has anything to do with this. These are all properties of the code running in the JIT (e.g. C, Java, etc). If you can solve these (hard!) problems in a statically compiled language, the solutions would translate over directly.

-Chris

Hi Chris,

Thank you for your quick answer!

I knew, that my question is not easy enough.

Maybe anyone know such projects or implementation examples on how to do a memory and CPU management in C++? This could be a good start point for me.

Thanks,
Oleg.

2009/10/19 Chris Lattner <clattner@apple.com>

Hi Oleg,

This is totally system dependent, it'd be *very* hard to do it
multi-platform. On Unix, a good part of it is resolved with jails and
user control, so it's more of an "installation and basic checks before
runing" issue. But for Windows and Mac, I have no idea, not even if
it's possible.

Normally, the kernel will try to hide statistics of running processes
from one another and themselves, so it's very unlikely that you'll be
able to restrict CPU usage without the help of the kernel.

Memory allocation is the easiest, STL and boost are full of allocators
that you can override. I'm just not sure how far can you go when the
JIT is running and allocating huge blocks, it might be too late until
you get back to your allocation routines.

I suggest you take a look at what Java does (http://gcc.gnu.org/java/
and Sun's, if possible), they do all sorts of security checks...

cheers,
--renato

Reclaim your digital rights, eliminate DRM, learn more at
http://www.defectivebydesign.org/what_is_drm

On part 1), there are also many papers on "system call interposition" or "system call monitoring." Google or Bing followed by tracking back references should uncover them.

--Vikram
Associate Professor, Computer Science
University of Illinois at Urbana-Champaign
http://llvm.org/~vadve

Check out professor Saman Amarasinghe’s work. It was being commercialized by a company called Determina (which has since been acquired by VMWare).

Evan