RFC: MCJIT enhancements

[andykaylor]

Following up and expanding on an earlier conversation, I’d like to discuss making several non-trivial changes to the MCJIT engine and related objects. There may be some interdependencies between these changes, but I think that they can be logically grouped as follows:

• Lazy module compilation

• Enhance the JIT memory manager interface to enable section-based memory protection

• Clean up object/memory ownership problems

• Introduce a mechanism for caching generated objects

• Support JIT events for MCJIT

• Support ELF generation on Windows

Now let me expand on each of these a bit.

[James_Grosbach]

Hi Andy,

This all sounds fantastic. I'm sure we'll have some spirited discussions about the details, but I completely agree with the approach and goals. Good stuff, and thanks for working on this!

-Jim

Following up and expanding on an earlier conversation, I'd like to discuss making several non-trivial changes to the MCJIT engine and related objects. There may be some interdependencies between these changes, but I think that they can be logically grouped as follows:

* Lazy module compilation

* Enhance the JIT memory manager interface to enable section-based memory protection

* Clean up object/memory ownership problems

* Introduce a mechanism for caching generated objects

* Support JIT events for MCJIT

* Support ELF generation on Windows

Now let me expand on each of these a bit.

---------------------
Lazy compilation
---------------------

Right now a Module is passed to the EngineBuilder::create call and is compiled immediately within the MCJIT constructor. I don't believe it is possible to perform lazy compilation on a per function basis as was done in the legacy JIT interface, but there's no reason that module compilation and loading can't be deferred until a function is requested. I think this is a fairly straight-forward and non-controversial change.

Yep, also effectively a pre-requisite for meaningful lazy JITing of multiple modules.

----------------------------------
Memory manager interface
----------------------------------

The memory manager changes are a bit more complicated. In the current MCJIT implementation, MCJIT clients may specify a memory manager through the EngineBuilder and this memory manager will be passed along to the MCJIT engine. The JITMemoryManager interface was originally created to suit the needs of the legacy JIT engine. As the MCJIT engine has come into existence, its memory needs have been somewhat wedged in to the old JITMemoryManager interface. The result has been that there are effectively two interfaces existing side-by-side in the same abstract base class and implementations will typically only support one or the other set of functions. This is further complicated by the RuntimeDyldMemoryManager, which doesn't inherit from the JITMemoryManager but does share a subset of functions and is implemented as a wrapper around the client-supplied memory manager in the current MCJIT code. There's also an MCJITmemoryManager class which derives from RuntimeDyld and is used by MCJIT but isn't exposed to the client at all.

I'd like to see this situation cleaned up as we move forward with a way for the client to specify an MCJITMemoryManager directly and a separation of the legacy memory manager interface from the new interface. In addition, I'd like the new memory manager interface to be extended with the functions that the RuntimeDyld needs to manage setting of section-specific permissions including non-writable code and read-only data.

The current situation is a bit ad-hoc as we try to minimize the impact of MCJIT bring up on any clients of the old JIT. Now that the MCJIT is becoming more mature, we can start being a bit more aggressive about that and making the interfaces conform to what the MCJIT really needs.

----------------------
Object ownership
----------------------

Within the current MCJIT implementation there are a number of ugly cross dependencies in object/memory life cycle, making the MCJIT/RuntimeDyld/JITMemoryManager relationship fragile and very dependent upon a particular order of object destruction.

In the current implementation, the buffer into which code is generated is owned by the MCJIT component, but when sections from the generated code are loaded, they are loaded into memory owned by the JITMemoryManager. During object loading, an ObjectFile is created within the RuntimeDyld which references both of these buffers. The situation is further compilicated in the case where the RuntimeDyldELF object attempts to register the generated object with GDB, because the GDB-interface requires a reference to both memory buffers. The GDB_required references are currently maintained in an ObjectImage instance which is held by RuntimeDyldELF.

For those who are visually oriented, I am attaching a diagram which shows the object relationships.

I would like to change this by introducing an ObjectBuffer which would be allocated by the MCJIT object at compilation time and then passed to the RuntimeDyld::loadObject. RuntimeDyld::loadObject would hand this ObjectBuffer off to the new ObjectImage instance (which it already creates today). The ObjectImage would be returned from the RuntimeDyld::loadObject to MCJIT and MCJIT would own that object.

Again, a diagram is attached.

Sounds reasonable.

[Demikhovsky_Elena]

I have some vision about Object Caching.

A client, which is interested in object caching, will need some help from compiler and LLVM side. The object file, located on disk, should contain information that allows to decide – rebuild or not rebuild.

The client (a backend engine, which translates IR to JIT) compares data from object file, found on disk and module IR.

The information that object file may contain I call “Signature”. It can be stored in “.comment” section.

Signature includes:

· Name of source file, full path. (It may be .ll or .cl or .cpp or any other extension, the compilation was started from)

· Time stamp of the source file.

· Name and version of the compiler.

· Compilation flags.

· Environment variables (all or those who affect compilation process)

The compiler can generate this data and put it in metadata.

If we’ll take Intel OpenCL product, the clang can generate this special metadata. MCJIT, while building an object file, will compose a “ .comment” with a signature, described above.

Ø I would like to introduce a mechanism whereby JITed objects could be cached and loaded from cache at a future time. It seems that the caching should logically be performed transparently within the MCJIT engine, but it also seems that the caching mechanism will need to be provided by the client (or at least under client control). I think that the best way to accomplish this is the introduce a new interface (ObjectManager?), similar to the memory manager, that can be specified by the client and used by the MCJIT engine. What I have in mind is that when MCJIT is about to compile a module, it will first check with the ObjectManager to see if a pre-compiled image is available. If so, it will simply pass that image to the RuntimeDyld for loading. If not, it will compile as usual. Likewise, after compilation (but before loading) MCJIT would offer the compiled image to the ObjectManager and the ObjectManager could save the image to a file cache (or whatever).

- Elena

[James_Grosbach]

This sounds interesting, but strikes me as significantly outside the scope of the MCJIT itself. That is, this is the sort of thing a client application or framework would build on top of the MCJIT.

-Jim

[andykaylor]

I agree. I’d like to see the interface be set up to enable clients to implement something like Elena described, but I was imagining a much simpler implementation in the LLVM trunk code.

-Andy

[Pawel_Bylica]

I’m using this patch mentioned and it works just fine.
However, I would like to see it implemented in more generic way:

1. When a container type is explicitly specified in triple it should be used by MCJIT.
2. When a container type is not specified it is deduced from OS.
3. Use ELF by default.

“Environment” part of the triple may be used to specify the container. Some options are already there (like MachO). But there may conflicts exist where both environment and container must have been specified together (e.g. “arm-apple-ios-eabi-macho”).

• Paweł

[andykaylor]

Hi Paweł,

Thanks for continuing this discussion.

I like the simplicity of your suggestion. My only concern involves the ambiguity of what is meant by “environment”. Presently there are functions in the llvm::Triple class to access the environment as an enumeration of a fixed set of values. It seems that some non-enumerated values are already in use, but introducing possible combinations of ABI and object format would seem to strain the API.

I couldn’t find an explanation anywhere of what is meant by environment in the context of the triple, and it isn’t clear from looking at the possible values whether object format properly belongs to the environment concept or whether it should be a new triple component. Looking at the uses in the LLVM code base, it seems that the environment element has been used specifically to enable generation of MachO objects on non-Darwin OSes. I see that clang recognizes “iphoneos” as a value (though it’s not in the Triple::EnvironmentType enum), and it’s not clear to me what it uses it for. Beyond that, the environment seems to be used to influence ABI selection, which would seem to have some degree of overlap with object format selection while being essentially different.

In other words, environment seems to be a bit of a catch-all dumping ground at the moment. That’s not entirely bad, I suppose, but it does make the task of cleanly extending the target triple more complicated.

It would seem that at some point the target triple has evolved into a target quadruple with the addition of environment. The question now, I think, is whether it makes sense to extend it further into a target quintuple, adding object format as a well-defined, but optional, component with full API support. The difficulty, of course, is that with two optional extensions the expected canonical form becomes ambiguous, but I don’t think that’s much of a problem. Given that object format is a relatively small enumeration, the triple parser should be able to handle it presence or absence while parsing or normalizing.

So here’s what I would suggest.

1. Define the target triple as follows (adapted from Triple.h):

/// Target triples are strings in the canonical form:

/// ARCHITECTURE-VENDOR-OPERATING_SYSTEM

/// or

/// ARCHITECTURE-VENDOR-OPERATING_SYSTEM-OBJECT_FORMAT

/// or

/// ARCHITECTURE-VENDOR-OPERATING_SYSTEM-ENVIRONMENT

/// or

/// ARCHITECTURE-VENDOR-OPERATING_SYSTEM-OBJECT_FORMAT-ENVIRONMENT

2. Add an llvm::Triple::ObjectFormatType enum with all llvm-supported object formats.

3. Remove “MachO” from the llvm::Triple::EnvironmentType enum.

4. Add the following methods to llvm::Triple:

Triple(const Twine &ArchStr, const Twine &VendorStr, const Twine &OSStr, const Twine &ObjFmtStr)

Triple(const Twine &ArchStr, const Twine &VendorStr, const Twine &OSStr, const Twine &ObjFmtStr, const Twine &EnvStr)

bool hasExplicitObjectFormat() const

ObjectFormatType getObjectFormat() const

StringRef getObjectFormatName() const

bool isOutputObjectFormatELF() const

bool isOutputObjectFormatCOFF() const

bool isOutputObjectFormatMachO() const

static const char * getObjectFormatName(ObjectFormatType Kind)

5. When an object format is specified, it will be used by MC and MCJIT if supported for the specified architecture

6. When an unsupport object format is specified, a fatal error will be thrown

7. When an object format and environment are both specified but are incompatible, a fatal error will be thrown

8. When an object format is not specified everything will behave as it currently does

How does that sound?

-Andy

[Pawel_Bylica]

Hi Andy,

That sounds great. Do we need any approval for this change? I can do the coding I you don’t mind.

I have one more question. Triples are specified in both modules and TargetMachine. In 3.1 triple from module was used by default. In trunk it has been changed, and LLVM_HOSTTRIPLE is used by default. Do we have any rules in case module triple conflicts with TargetMachine triple?

• Paweł

[James_Grosbach]

Hi Paweł,

Thanks for continuing this discussion.

I like the simplicity of your suggestion. My only concern involves the ambiguity of what is meant by “environment”. Presently there are functions in the llvm::Triple class to access the environment as an enumeration of a fixed set of values. It seems that some non-enumerated values are already in use, but introducing possible combinations of ABI and object format would seem to strain the API.

I couldn’t find an explanation anywhere of what is meant by environment in the context of the triple, and it isn’t clear from looking at the possible values whether object format properly belongs to the environment concept or whether it should be a new triple component. Looking at the uses in the LLVM code base, it seems that the environment element has been used specifically to enable generation of MachO objects on non-Darwin OSes. I see that clang recognizes “iphoneos” as a value (though it’s not in the Triple::EnvironmentType enum), and it’s not clear to me what it uses it for. Beyond that, the environment seems to be used to influence ABI selection, which would seem to have some degree of overlap with object format selection while being essentially different.

In other words, environment seems to be a bit of a catch-all dumping ground at the moment. That’s not entirely bad, I suppose, but it does make the task of cleanly extending the target triple more complicated.

It would seem that at some point the target triple has evolved into a target quadruple with the addition of environment. The question now, I think, is whether it makes sense to extend it further into a target quintuple, adding object format as a well-defined, but optional, component with full API support. The difficulty, of course, is that with two optional extensions the expected canonical form becomes ambiguous, but I don’t think that’s much of a problem. Given that object format is a relatively small enumeration, the triple parser should be able to handle it presence or absence while parsing or normalizing.

So here’s what I would suggest.

1. Define the target triple as follows (adapted from Triple.h):

/// Target triples are strings in the canonical form:

/// ARCHITECTURE-VENDOR-OPERATING_SYSTEM

/// or

/// ARCHITECTURE-VENDOR-OPERATING_SYSTEM-OBJECT_FORMAT

/// or

/// ARCHITECTURE-VENDOR-OPERATING_SYSTEM-ENVIRONMENT

/// or

/// ARCHITECTURE-VENDOR-OPERATING_SYSTEM-OBJECT_FORMAT-ENVIRONMENT

2. Add an llvm::Triple::ObjectFormatType enum with all llvm-supported object formats.

3. Remove “MachO” from the llvm::Triple::EnvironmentType enum.

4. Add the following methods to llvm::Triple:

Triple(const Twine &ArchStr, const Twine &VendorStr, const Twine &OSStr, const Twine &ObjFmtStr)

Triple(const Twine &ArchStr, const Twine &VendorStr, const Twine &OSStr, const Twine &ObjFmtStr, const Twine &EnvStr)

bool hasExplicitObjectFormat() const

ObjectFormatType getObjectFormat() const

StringRef getObjectFormatName() const

bool isOutputObjectFormatELF() const

bool isOutputObjectFormatCOFF() const

bool isOutputObjectFormatMachO() const

static const char * getObjectFormatName(ObjectFormatType Kind)

5. When an object format is specified, it will be used by MC and MCJIT if supported for the specified architecture

6. When an unsupport object format is specified, a fatal error will be thrown

7. When an object format and environment are both specified but are incompatible, a fatal error will be thrown

8. When an object format is not specified everything will behave as it currently does

How does that sound?

-Andy

Hi Andy,

That sounds great. Do we need any approval for this change? I can do the coding I you don’t mind.

Changes to the Triple have pretty wide-reaching implications. Chris, what do you think?

-Jim

[andykaylor]

Personally, I’d be happy to handle the explicit ELF request the same way the explicit MachO request is currently handled (i.e. in the environment string, which is what the non-committed patch does), but when this came up before the consensus was that that wasn’t general enough. My proposal below is just an attempt to flesh out what general support for explicit object format requests might look like.

-Andy

[andykaylor]

Has anything more happened with this?

-Andy

[andykaylor]

ping

[James_Grosbach]

Chris, are you OK with the below changes to the Triple?

-Jim

[Chris_Lattner]

Chris, are you OK with the below changes to the Triple?

If at all possible, I'd like to keep the triple changes separate (separate patch series and separate discussion) from the other MCJIT changes. How dependent are the MCJIT improvements on the Triple changes?

As you've noticed, Triple is not a particularly well defined class, because it is heavily influenced by the GNU target triples, which are themselves not particularly well designed, and don't capture the information we really need. Instead of adding another component to triple, I think it would be interesting to finally tackle the big redesign of this area. For example, something along these lines might be interesting:
http://nondot.org/sabre/LLVMNotes/TargetSpec.txt

-Chris

[andykaylor]

The other MCJIT improvements aren’t dependent on the triple changes at all. The reason that this came up in the context of MCJIT is that the quickest way to get MCJIT support on Windows is to enable the generation of ELF object images on Windows.

We previously proposed a minimal approach of just adding “ELF” to the recognized triple environment strings. This is the way that MachO generation is enabled on non-Darwin platforms in the current LLVM code, and it’s the way we did this in our in-house implementation.

If the target specification is going to be redesigned, I’d like to add the minimal approach for ELF on Windows in the mean time. The bigger redesign sounds like a good idea, and I’d be open to helping out in that regard if I can be useful, but I’d like to see something in place as soon as possible even if it’s only as a stop-gap measure.

-Andy

[Chris_Lattner]

Ok, I'm not trying to sign you up to redesign the world :). If this is just going to be a relatively short-term thing, I'd recommend doing the minimally invasive approach, even if it is another terrible triple travesty.

-Chris