Hi Chris,
- Using the addIRModule method for adding new global variables, I’m having to set them to external linkage, so that they can be found during lookup. One issue with this, is I have to manually loop through all the underlying globals/functions and change their linkage to external as well. If I don’t do that, I’ll get a JIT Session Error about not being able to find the underlying globals/functions. Is there an easy way to set all of the underlying globals/functions to external linkage as well?
There is no “change all linkages” convenience function, you just have to loop over all the global values and set them individually. If you’re doing that though there’s likely a design issue somewhere else: In general languages will define which symbols get external linkage and which do not, and the front-end should set up the linkages for you: you shouldn’t need to modify linkages in order to add something to the JIT. E.g. If you have the following in C:
static void foo(void) { }
void bar(void) { foo(); }
then clang will translate that to:
define internal void @foo() { … }
define void @bar() { … }
This IR, with foo having internal linkage and bar external, can be added to a JITDylib as-is and will result in one visible definition: bar. (ORC can handle internal symbols just fine, they just don’t appear in the public interface for the JITDylib because that’s what internal linkage means: visible only from within the module).
The two main exceptions to the “you shouldn’t need to promote linkage yourself” rule are:
(1) Debuggers / REPLs, where you want to give a user the ability to call directly into what would usually be an internal function. In that case you should only need to promote the linkage on the internal symbols that you want the user to be able to access. If that’s everything, then promote everything. If that’s only certain functions/variables then just promote those functions/variables.
(2) Manually breaking up modules: If you want to perform more fine grained or parallel compilation you may choose to break up large modules. In that case you need to promote internal symbols to external-hidden symbols so that they can still be resolved across the new module boundaries. You will usually have to rename them to avoid clashes too. If you’re using LLLazyJIT though you don’t need to worry: all the symbol promoting and renaming required for lazy compilation is taken care of internally.
So: Why is your front-end producing internal definitions for symbols that you want to be able to call in the JIT? Should your front-end be producing external definitions?
- We’ve had this problem ever since we switched to OrcMCJitReplacement. When jitting code on linux, we have to comment the body of registerEHFrames(). If we don’t, then any exceptions that are thrown in jitted code will result in a segfault. Do you know what issue we might be running into here? RuntimeDyIdELF::registerEHFrames() is commented out for new LLJIT. OrcMCJitReplacement::registerEHFrames when using OrcMCJitReplacement.
registerEHFrames is supposed to register the __eh_frames section with libunwind. I know that RuntimeDyldMachO’s __eh_frame support was weak, and would often result in registering broken frames. I don’t know RuntimeDyldELF’s __eh_frame support code well, but from a glance I suspect it suffers the same problems. That is almost certainly why your code is segfaulting when exceptions are thrown: the JIT is registering broken frames.
The poor quality of RuntimeDyld’s exception handling is why I turned it off by default. The new JIT linker (JTILink) has much better eh_frame support, but unfortunately there is no ELF JITLink implementation yet.
The upshot of all this is that at the moment it’s not safe in general to throw exceptions from/through JIT’d code on ELF. I’m hoping that JITLink ELF variants will solve this eventually, but there’s no clear timeline for that work.
– Lang.