mmap and vm_protect on ARM+Apple systems

Hi,

Can anyone tell me something about mmap and vm_protect on ARM+Apple systems?

I’m working on a new memory manager implementation for MCJIT and I want to replace calls to Memory::AllocateRWX with calls to Memory::allocateMappedMemory, possibly still with the RWX flags. However, looking at the Memory::AllocateRWX implementation I see that it’s jumping through some hoops in the case where both ‘APPLE’ and ‘arm’ are defined.

I want to handle two cases:

  1. Allocate memory as RWX, copy JITed code into it and execute it without ever touching the permissions again.

  2. Allocate memory as RW, copy JITed code into it and then set the permissions to RX before executing.

This seems pretty straight-forward with the functions that use mmap and mprotect, but the AllocateRWX implementation leads me to believe that won’t work in the ARM+Apple case. I read some vm_protect documentation, but I can’t quite reconcile it with the AllocateRWX implementation.

My goal is to make sure the Memory::allocateMappedMemory and Memory::protectMappedMemory functions will work everywhere.

Any suggestions?

Thanks,

Andy

Hi Andy,

Hi,

Can anyone tell me something about mmap and vm_protect on ARM+Apple systems?

Some. :slight_smile:

I’m working on a new memory manager implementation for MCJIT and I want to replace calls to Memory::AllocateRWX with calls to Memory::allocateMappedMemory, possibly still with the RWX flags. However, looking at the Memory::AllocateRWX implementation I see that it’s jumping through some hoops in the case where both ‘__APPLE__’ and ‘__arm__’ are defined.

I want to handle two cases:

1) Allocate memory as RWX, copy JITed code into it and execute it without ever touching the permissions again.

ARM Darwin can't do this. Memory is never allowed to be both writable and executable at the same time.

2) Allocate memory as RW, copy JITed code into it and then set the permissions to RX before executing.

This is the path that can work, if the JIT process has been appropriately blessed by the system. Normal applications can't do this (the vm_protect() will fail transitioning from writable to executable).

“ARM Darwin can’t do this. Memory is never allowed to be both writable and executable at the same time.”

That seems very sensible. So do the JIT engines just not work there now?

Currently both MCJIT and JIT are using AllocateRWX which on that platform mmap’s the memory as RX, then vm_protect’s it as RX+VM_PROT_COPY, then vm_protect’s it as RW.

My best guess is that this is trying to set up something that’s actually backed by two pages where whatever is written to the RW page will be copied to the RX page, though if that’s possible it would seem to defeat the purpose of not allowing RWX.

The case I’m ultimately interested in is the second case where things get allocated as RW and later set to RX. I was trying to support the RWX case to provide an implementation that behaved like the old code, but maybe I just shouldn’t bother?

-Andy

The correct way of implementing this is to have one physical page mapped to two locations in virtual memory. One is writeable, the other is read + execute. This does not defeat the point of W^X, because you must find both pointers to be able to exploit it, rather than just one. This is really how JIT'd memory maps should be created on all platforms, as it allows you to use the memory without significant fragmentation (i.e. you don't need a whole page per function) and it makes it much harder to exploit the JIT'd code.

David

Thanks, David.

Is that what is being done in the Memory::AllocateRWX code (in lib/Support/Unix/Memory.inc)?

-Andy

“ARM Darwin can't do this. Memory is never allowed to be both writable and executable at the same time.”

That seems very sensible. So do the JIT engines just not work there now?

They work by allocating RW memory, then switching the permissions to RX before running any code stored there. That requires special process permissions.

Currently both MCJIT and JIT are using AllocateRWX which on that platform mmap’s the memory as RX, then vm_protect’s it as RX+VM_PROT_COPY, then vm_protect’s it as RW.

My best guess is that this is trying to set up something that’s actually backed by two pages where whatever is written to the RW page will be copied to the RX page, though if that’s possible it would seem to defeat the purpose of not allowing RWX.

The case I’m ultimately interested in is the second case where things get allocated as RW and later set to RX. I was trying to support the RWX case to provide an implementation that behaved like the old code, but maybe I just shouldn’t bother?

Yep. There was never a working RWX on ARM/Darwin. AFAIK it only existed on X86/Darwin because it was allowed and easier that way than twiddling the permissions. If you get a RW->RX path, it's fine for both.

-j

Is this really required? The canonical way of getting around this restriction is to mmap the same physical page as RW in one location and RX in another. I've not tested this on ARM/Darwin, but it works on every R^X platform I've tried. You can then publish the address of the RX page and keep the RW page private inside the JIT, which makes exploiting the writeable-and-executable page very hard for malicious code because it needs to find both virtual addresses.

David

They work by allocating RW memory, then switching the permissions to RX before running any code stored there. That requires special process permissions.

Is this really required? The canonical way of getting around this restriction is to mmap the same physical page as RW in one location and RX in another. I've not tested this on ARM/Darwin, but it works on every R^X platform I've tried. You can then publish the address of the RX page and keep the RW page private inside the JIT, which makes exploiting the writeable-and-executable page very hard for malicious code because it needs to find both virtual addresses.

In this design, is the JIT and the JIT'ed program in the same address space/process, or are they in different processes so that the JIT'ed code never has access to the RW mapping of the frame?

If they're in the same process/address space (i.e., the JIT'ed program does have access to the RW mapped page), then I think the argument that having to guess two addresses to make an exploit work is a very weak protection (especially since I would expect the JIT to arrange the virtual address space so that fast bit masking could be used to compute the RX address from the RW address and vice versa).

-- John T.