rwx pages dangerous?

I noticed that JITMemoryManager allocates its slabs as rwx. Isn’t it a security problem to have memory mapped as both writable and executable? I think JITs often avoid this by mapping their memory as rw, then switching it to rx once the data has been written. I was facing a similar problem in a JIT of my own and was curious to see how LLVM addresses this issue.

Thanks,
Josh

I don’t work on or with the LLVM JIT, but I do work on memory safety, so I think I can comment. First, I suspect that the choice to leave rwx memory lying around is for convenience and performance. Having to toggle page protections takes time as it requires a system call and some TLB flushes/changes, so I am not surprised that the JIT does not do it. There are other designs that might mitigate this (e.g., running the JIT in a separate process from the C code so that each have different MMU mappings of the same physical memory), but these designs may have other performance issues. I’ll let someone more familiar with JIT design discuss that. Second, if you’re using the JIT to run memory-unsafe programs (such as C programs), having writable and executable memory may not make matters too much worse than they would be otherwise. Return-to-libc and Return Oriented Programming attacks are effective and do not require writable memory. If you’re not using some mitigation technique (e.g., SAFECode or a version of control-flow integrity that doesn’t depend on non-wx memory), you have no real security. – John T.

Hi Josh,

You're absolutely right that this can be an issue. The LLVM JIT allows you to implement your own JITMemoryManager policy class, and this policy enables you to implement things like this.

-Chris

I'm not sure about the legacy JIT interface, but I don't think this can be done cleanly with the current MCJIT interface.

I mentioned a while ago that this is one of the improvements we'd like to make to MCJIT. I can definitely see Josh's point about performance, and so we won't want permissions to always be set, but I do think it's worth revising the interface between the RuntimeDyld component and the memory manager to facilitate setting permissions when that is desired.

-Andy

In the world of the MCJIT, memory allocation, including permissioning, is the responsibility of the client application. The MCJIT only directly touches memory in the address space of the host. Once the old JIT is removed, all of the allocations done by the included memory manager can become RW. lli's trivial memory manager will then have the RW->X transition for code sections. All of the interfaces regarding permissions and such are solely for the support of the old JIT.

-Jim

The main problem, as I see it, is that there's no way for the memory manager to know when it can safely change the protection state of sections apart from making assumptions about the implementation of MCJIT (namely that it will generate code on construction) and receiving some sort of notification outside the standard interface. I believe there's also no provision for handling read-only data sections.

These are minor problems, of course, but I think it's a place where there's room for improvement.

-Andy

Somewhat, yes. The MCJIT currently doesn't support lazy compilation in general, and things like notifications back (via the memory manager) when new sections have been produced and such needs to be part of that. Your'e right that, for now, the underlying assumption is that everything gets built up at once. The following is mostly stuff we've talked about before, but just to make sure we're on the same page and for the benefit of those who weren't involved in those earlier conversations, here's a rough outline.

The flow would be something like this:

1) Create an IR module
2) Fire up the MCJIT, which will compile it
  2a) Record the memory regions requested by the JIT in the client's memory manager.
  2b) Allocate space in the target process memory for each section (if different from the host process)
3) Tell the RuntimeDyld about the mapping from host to target section addresses as recorded in (2)
4) Call resolveRelocations() to use that data.
5) Copy each section into the target process
6) Set the permissions of each section in the target process appropriately
7) Inform the target it has new code/data and to being executing it or whatever

For lazy compilation, there will be stubs inserted (client specific since we're talking about target->host communication) that re-engage that process. In particular there will need to be hooks to set up the stubs and to replace them with calls to the real code when it get lazily filled in.

You're absolutely right there's no communication about R/W vs. RO data sections in the current interface. That's definitely something that should be added.

The key thing in all of that is that the MCJIT itself doesn't know about most of the gory bits regarding the copying, permissions, etc.. It just knows it's given a blob of bitcode to compile and to break it down into blobs of code and data for the client to do something with. We're making a much stronger API layering between the JIT itself and the memory management issues of actually doing anything with the code that's been compiled via the JIT.

-Jim

When you talk about lazy compilation, it isn't clear to me if you mean the single compilation step that produces a loadable module being delayed until a function is requested or the full-blown, legacy-JIT style lazy compilation of individual functions within a module being JITed only when needed. If the latter, it isn't clear to me how that would be done within the MC model.

That's a bit of a tangential matter though. I do agree with the basic flow as you laid it out.

I also agree with your statement that the MCJIT doesn't know about most of the gory bits regarding the copying, permissions, etc.. However, the RuntimeDyld should be involved with at least the interface that triggers these gory bits, if not the implementation details as such. That is, the main thing that I think is missing is a way for the RuntimeDyld, which knows about what the permissions should be and when they can be set, to communicate that information to the memory manager, which should be responsible for actually setting the permissions.

-Andy

When you talk about lazy compilation, it isn't clear to me if you mean the single compilation step that produces a loadable module being delayed until a function is requested or the full-blown, legacy-JIT style lazy compilation of individual functions within a module being JITed only when needed. If the latter, it isn't clear to me how that would be done within the MC model.

The former. The level of granularity would change, basically, from a single function to a single module. Clients of the MCJIT could get effectively the old behavior by putting each function into its own Module prior to supplying them to the MCJIT. Right now there's only support for a single Module, and the compilation happens as soon as the MCJIT is created. Both of those things will need to change and become more robust to support on-demand compilation.

That's a bit of a tangential matter though. I do agree with the basic flow as you laid it out.

I also agree with your statement that the MCJIT doesn't know about most of the gory bits regarding the copying, permissions, etc.. However, the RuntimeDyld should be involved with at least the interface that triggers these gory bits, if not the implementation details as such. That is, the main thing that I think is missing is a way for the RuntimeDyld, which knows about what the permissions should be and when they can be set, to communicate that
information to the memory manager, which should be responsible for actually setting the permissions.

Yes, absolutely. It's possible that'll be as simple as additional parameters to the allocate*() functions for telling the client what the permissions will end up being. Likewise, once things are in a lazy model, there will be a notification hook whenever steps 4-6 of the below need to happen for whatever new bits got compiled (steps 2 and 3 will have already happened via the existing allocate* hooks).

Regards,
   Jim