ORC API C Standard/System Libraries

Hi all,

First of all, I want to say that I really appreciate all the work that’s gone into LLVM, clang and the ORC API. I would be very unlikely to be doing my current project if they weren’t available.

I’m using the ORC API to JIT some C code (soon to be C++) on Windows.

(I was on LLVM 8.0.0 but it didn’t handle the ‘allocator’ attribute so I upgraded to LLVM 9.0.0.)

The JIT compiling works well for simple files, #included local files and object files.
As soon as I start trying to use system libraries with functions, things start to fall down.
I think this is largely just due to my ignorance of clang in general, but any guidance would be greatly appreciated.

  1. I get multiple warnings for the system headers (e.g. stdlib) of the form:
    “warning: macro expansion producing ‘defined’ has undefined behavior”

Is there a way to prevent these warnings? Preferably only for the system libraries.
I tried -Wno-expansion-to-defined, -fms-extensions, -fms-compatibility-fms, -compatibility-version=14.16.27023. I also tried to reference the system library folder by -isystem, -internal-isystem and -c-isystem

  1. Disregarding the warnings, I also get the following error message:
    “JIT session error: Symbols not found: { malloc }”

I assumed this might be because I wasn’t linking the standard library, so I tried appending “msvcrt.lib” and “-stdlib=msvcrt.lib” but to no avail. Can you use .lib files with clang on Windows?

How do I make sure that the symbols defined in the standard libraries are available to my JIT?

Thanks for any help you can give!
All the best,
Andrew

+Lang Hames, Liberator of the Orcs

Hi Andrew,

  1. I get multiple warnings for the system headers (e.g. stdlib) of the form:
    “warning: macro expansion producing ‘defined’ has undefined behavior”
    Is there a way to prevent these warnings? Preferably only for the system libraries.
    I tried -Wno-expansion-to-defined, -fms-extensions, -fms-compatibility-fms, -compatibility-version=14.16.27023. I also tried to reference the system library folder by -isystem, -internal-isystem and -c-isystem

I’m afraid I don’t have any insight on this. If nobody chimes in on this list it may be worth asking this on the clang-dev mailing list.

  1. Disregarding the warnings, I also get the following error message:
    “JIT session error: Symbols not found: { malloc }”

How is your JIT set up? Are you using LLJIT, or a custom JIT class?

Either way, the key is to make sure that process symbols are added to a JITDylib so that they can be found by JIT’d code. As an example, the code to do this in LLVM’s lli tool is:

J->getMainJITDylib().addGenerator(
ExitOnErr(orc::DynamicLibrarySearchGenerator::GetForCurrentProcess(
J->getDataLayout().getGlobalPrefix())));

Cheers,
Lang.

Hi Lang,

Thanks for the response.

I’m using a custom JIT that was copied from LLJIT and then slightly edited.
It’s jitting to memory that is shared with a child process and the child process runs the ‘code’.
I assume this generator won’t work as the symbols won’t be in the current process.

My recollection is that LLVM used to have remote process support but has since deprecated it. Is that correct?

My working model was that clang would create all the symbols that are used (both from a user code file and from system library), these would go into the IR, and then the JIT would turn these into executable code.
This doesn’t seem to be correct given that the library symbols aren’t being found. (Is this because I’m not statically linking the system libs? I’ll check this out.)

I’m not sure whether I’m misunderstanding the clang side, the LLVM side, something else, or some combination of the above!
I’ll have another look at the IR now and see if I can work anything out but any light you (or anyone else) can shed on the matter would be greatly appreciated.

All the best,
Andrew