ORC JIT Weekly #32 -- Latest ORC runtime preview

Project Infrastructure LLVM Dev List Archives
1

Hi All,

Apologies for the lack of updates – I’ve been flat out prototyping the ORC runtime, but the details of that process weren’t particularly newsworthy. Good news though: The result that work is available in the new preview branch at https://github.com/lhames/llvm-project/pull/new/orc-runtime-preview, and it’s starting to look pretty good.

A quick recap of this project, since it’s been a while since my last update: Some features of object files (e.g. thread local variables, exceptions, static initializers, and language metadata registration) require support code in the executor process. We also want support code in the executing process for other JIT features (e.g. laziness). The ORC runtime is meant to provide a container for that support code. The runtime can be loaded into the executor process via the JIT, and the executor and JIT processes can communicate via a builtin serialization format (either serializing/deserializing directly in-process, or communicating serialized data via IPC/RPC) to coordinate on complex operations, for example discovering (on the JIT side) and running (on the executor side) all initializers in a JITDylib.

After a bit of hacking, the setup code for all this is looking very neat and tidy. For example, to turn on support for advanced MachO features:

if (auto P = MachOPlatform::Create(ES, ObjLinkingLayer, TPC, MainJD,

OrcRuntimePath))
ES.setPlatform(std::move(*P));
else
return P.takeError();

That’s it. The MachOPlatform loads the runtime archive into the ObjectLinkingLayer, then installs an ObjectLinkingLayer::Plugin to scan all loaded objects for features that it needs to react to. When it encounters such a feature it looks up the corresponding runtime functionality (loading the runtime support into the executor as required) and calls over to the runtime to react. For example, if an object contains an __eh_frame section then the plugin will discover its address range during linking and call over to the runtime to register that range with libunwind.

Having set up the platform, you can add objects compiled from C++, Objective-C, Swift, etc., using static initializers, thread locals, etc. and everything should Just Work.

Of course it doesn’t all Just Work yet: The plumbing work is mostly complete, but I haven’t written handlers for all the special sections yet. A surprising number of things do work (e.g. C++ code with static initializers/destructors, TLVs and exceptions, and simple Objective-C and Swift programs). An equally surprising number of simple things don’t work (zero-initialized thread locals fail because I haven’t gotten around to handling .tbss sections yet).

If you would like to play around with the runtime (and have access to an x86-64 Mac) you can build it by checking out the preview branch above and configuring LLVM like this:

xcrun cmake -GNinja
-DCMAKE_BUILD_TYPE=Debug
-DLLVM_ENABLE_PROJECTS=“llvm;clang”
-DLLVM_ENABLE_RUNTIMES=“compiler-rt;libcxx;libcxxabi”
/path/to/llvm

Then you can try running arbitrary MachO objects under llvm-jitlink, which has been updated to load the built runtime by default. You should be able to run the objects both in-process (the default), and out-of-process (using -oop-executor or -oop-executor-connect) and have them behave exactly the same way.

What we have so far is a pretty good proof of concept, so I’ll start a new thread on llvm-dev tomorrow to discuss how we can land this in the LLVM mainline.

– Lang.

2

Thanks for sharing the preview!

I am referring to this code state: It looks like WrapperFunctionUtils in OrcShared define the common data structures that can be passed back and forth between runtime and JIT. It contains a non-trivial amount of code, but all in all it appears to be self-contained and header-only. I guess that’s because the runtime is supposed to not depend on any LLVM libs? The duplication of the Error, Expected and ExtensibleRTTI classes is a little unfortunate. I assume we won’t need arbitrary data structures in the runtime and for communication with it, but what we need must be duplicated? Isn’t there any way to avoid it? A separate Error lib above Support? It’s probably fine as is for the moment, but maybe a mid-term perspective could be discussed when integrating into mainline. There are no platform-specific flavors of the clang_rt.orc static library. The memory footprint will likely be small, but won’t it impact JITLink performance since it will have to process symbols for all platforms (, __orc_rt_elf_tlv_get_addr, etc.)? If not, then why? Can they be dead-stripped early on? Best, Stefan

3

Hi Stefan,

It looks like WrapperFunctionUtils in OrcShared define the common data structures that can be passed back and forth between runtime and JIT. It contains a non-trivial amount of code, but all in all it appears to be self-contained and header-only. I guess that’s because the runtime is supposed to not depend on any LLVM libs?

That’s right. I’m not sure what design we’ll end up with, but it will either be careful header sharing like this or duplication of some structures between LLVM and the runtime.

The duplication of the Error, Expected and ExtensibleRTTI classes is a little unfortunate. I assume we won’t need arbitrary data structures in the runtime and for communication with it, but what we need must be duplicated? Isn’t there any way to avoid it?

For now my goal has been “Make sure compiler-rt doesn’t end up depending on libSupport”. Anything that requires libSupport (including llvm::Error as it’s currently written) has to be duplicated. The common data structures above were carefully written to avoid any dependence on libSupport.

A separate Error lib above Support? It’s probably fine as is for the moment, but maybe a mid-term perspective could be discussed when integrating into mainline.

I like the idea of separating ADT from libSupport and moving Error to ADT. If we ever did that then the ORC runtime (and other LLVM projects) could just use LLVM ADT. On the other hand there are some nice features that we can add to Error if we do depend on libSupport, like backtraces to the “throw” location when an error is dropped – if we made Error sharable with compiler-rt then we’d have to give that up.

There are no platform-specific flavors of the clang_rt.orc static library. The memory footprint will likely be small, but won’t it impact JITLink performance since it will have to process symbols for all platforms (__orc_rt_macho_tlv_get_addr, __orc_rt_elf_tlv_get_addr, etc.)? If not, then why? Can they be dead-stripped early on?

The static library is added as a definition generator, and the platform-specific runtime symbols will be looked up by the platform-specific orc::Platform instance. The archive’s symbol table will be bigger than it otherwise would have been (since it will have symbols for all platforms in it) but that only impacts the first lookups during Platform initialization, and the overhead will be negligible. All subsequent platform-symbol lookups go directly to the JITDylib.

– Lang.

4

A separate Error lib above Support? It’s probably fine as is for the moment, but maybe a mid-term perspective could be discussed when integrating into mainline.

I like the idea of separating ADT from libSupport and moving Error to ADT. If we ever did that then the ORC runtime (and other LLVM projects) could just use LLVM ADT. On the other hand there are some nice features that we can add to Error if we do depend on libSupport, like backtraces to the “throw” location when an error is dropped – if we made Error sharable with compiler-rt then we’d have to give that up.

You wouldn’t necessarily need to give up backtraces, you could have a weak function for taking the backtrace with a default no-op implementation in LLVM ADT, that function would be overridden by libSupport with a function that does the actual backtrace. This is basically how backtraces are planned on being handled in Rust’s Error trait when it is moved to the core crate, where the backtrace functionality is provided by std (which depends on core). The only difference is they are using special rust compiler hooks instead of weak functions.
https://github.com/rust-lang/project-error-handling/issues/3

5

You wouldn’t necessarily need to give up backtraces, you could have a weak function for taking the backtrace with a default no-op implementation in LLVM ADT, that function would be overridden by libSupport with a function that does the actual backtrace. This is basically how backtraces are planned on being handled in Rust’s Error trait when it is moved to the core crate, where the backtrace functionality is provided by std (which depends on core). The only difference is they are using special rust compiler hooks instead of weak functions.
https://github.com/rust-lang/project-error-handling/issues/3

Oh – That’s a really good point. Maybe decoupling Error from libSupport isn’t as difficult as I had assumed.

Thanks Jacob!

– Lang.