ORC JIT Weekly #36 -- MachO Platform support lands, ELF JITLink backends get easier, C API improvements

Hi All,

The schedule for this work slipped a little, but I’ve got good news:

First, I’m happy to announce that MachO platform support has landed in [1]. This allows ORC users to load and use MachO support from the ORC runtime with the following idiom:

if (auto P = MachOPlatform::Create(ES, ObjLayer, EPC, PlatformJD,
return P.takeError();

ES is the ExecutionSession,
ObjLayer is an ObjectLinkingLayer,
EPC is an ExecutorProcessControl instance,
PlatformJD is the JITDylib that the MachO platform symbols should be loaded into, and
OrcRuntimePath is the path on disk of the ORC runtime archive.

Once the MachOPlatform instance has been created and attached to the execution session MachO programs should Just Work under ORC*,* including features like static initializers, static destructors, exceptions, native thread-local-variables, and language runtime support (e.g. for Objective-C and Swift)*. This support should be independent of the program representation added (e.g. object file, LLVM IR, or higher-level module representation), which will enable object-caching schemes to work even in the presence of these features. The support should also work equally for both in-process and out-of-process JITs, making it easier for users to execute JIT’d code in separate processes.

As JITLink ELF support continues to grow, and with the fundamental infrastructure in place, we’re in good shape if anyone wants to start work on a corresponding ELFPlatform.

Speaking of JITLink ELF support, there’s good news on that front too: I’ve added a generic ELFLinkGraphBuilder in [2] and [3]. This should make it much easier to contribute new ELF backends for JITLink. Generic section/symbol parsing is now handled for you, and you should only need to implement relocation parsing/application for new architectures when bringing up a new ELF backend.

The first new backend that uses the generic ELFLinkGraphBuilder is already up for review at https://reviews.llvm.org/D105429 – Thanks very much to StephenFan for working on that!

Finally, thanks very much to Valentin Churavy for recent improvements to the ORC C APIs in https://reviews.llvm.org/D104672 and https://reviews.llvm.org/D104799. Valentin’s additions enable lazy compilation via the new ORC C bindings, including lazy partitioning of modules. He has provided example code in llvm/examples/OrcV2Examples/OrcV2CBindingsLazy/, and llvm/examples/OrcV2Examples/OrcV2CBindingsVeryLazy/. These APIs are experimental for now (and likely to see some churn as the ORC runtime and related features settle in) but restore some key functionality that was lost when the ORCv1 C bindings were removed.

– Lang.

[1] https://github.com/llvm/llvm-project/commit/bb5f97e3ad10a0f8a62560890e5a87b4bc2c00bd

[2] https://github.com/llvm/llvm-project/commit/aff57ff24aca5074b427d6bbc2f3246aa97910c5
[3] https://github.com/llvm/llvm-project/commit/8e66fc438463e8cf6fa05e88b51ce29b604ce307

  • This initial commit omitted TLV and language runtime support to try to keep the size down (it was already quite large), but I expect support for those features to land this week.