I’ve finally managed to extract from Kaleidoscope one particular thing that it seems to me should be working and isn’t. Given the global declaration
extern “C” void foo() {}
within the same program I have
RTDyldMemoryManager::getSymbolAddressInProcess(“foo”)
And it’s returning null. (LLVM 3.8, Windows 7 x64.) What am I missing?
Have you tried to add dllexport?
Going a few more layers down the rabbit hole, it looks like this is the call chain
C:\llvm\lib\ExecutionEngine\RuntimeDyld\RTDyldMemoryManager.cpp
uint64_t
RTDyldMemoryManager::getSymbolAddressInProcess(const std::string &Name) {
…
return (uint64_t)sys::DynamicLibrary::SearchForAddressOfSymbol(NameStr);
C:\llvm\lib\Support\DynamicLibrary.cpp
#include “Windows/DynamicLibrary.inc”
C:\llvm\lib\Support\Windows\DynamicLibrary.inc
void *DynamicLibrary::getAddressOfSymbol(const char *symbolName) {
…
if (Data == &OpenedHandles)
return SearchForAddressOfSymbol(symbolName);
return (void *)(intptr_t)GetProcAddress((HMODULE)Data, symbolName);
void* DynamicLibrary::SearchForAddressOfSymbol(const char* symbolName) {
…
return 0;
but wait, getAddressOfSymbol was never called. The call skipped that function - which presumably would have done the job - and hopped directly to SearchForAddressOfSymbol, which only checked OpenedHandles, and didn’t do a straight check on the current process? Am I reading this correctly?
Tried that, still didn’t work. Then I tried making a direct API call,
GetProcAddress(GetModuleHandle(0),“foo”)
And this works if and only if __declspec(dllexport) is supplied. So it looks like we were both right.
Hi Russel, I guess you’re using ORC? Then you need a mapping layer. Have a look at this line:
Best, Stefan
Thanks! Comparing your code to Kaleidoscope, they are superficially very different but when you boil it down, you can see the common elements. I think the most significant difference is that where you have
auto compiler = std::make_unique(targetMachine);
compiler->addGlobalMapping(“myAlloc”, (void)myAlloc);
Kaleidoscope has
auto resolver = createLambdaResolver(
[&](const std::string &name) {
if (auto sym = compileLayer->findSymbol(name, false))
return RuntimeDyld::SymbolInfo(sym.getAddress(), sym.getFlags());
return RuntimeDyld::SymbolInfo(nullptr);
},
(const std::string &S) { return nullptr; });
compileLayer->addModuleSet(std::vector<Module *>(1, module),
new SectionMemoryManager(), std::move(resolver));
I’m going to guess the reason for this difference is something along the lines of:
Orc provides a way to supply your own custom code for resolving symbols - for some reason this is packaged with parameters for memory manager and a set of modules rather than a single module - maybe they figured instead of providing various custom options and complicating the API, best to just provide one big function that lets you customise everything, you can always supply basic default parameters for things you don’t want to customise - which makes sense - and that’s what Kaleidoscope uses.
Orc also provides a class that contains a basic system for resolving symbols if you don’t want to customise, and that works by just supplying it with each symbol you’re going to want to look up, together with the associated function pointer - and that’s what you used.
Is that a more or less accurate summary of the situation?
Hi Russell,
Orc provides a way to supply your own custom code for resolving symbols - for some reason this is packaged with parameters for memory manager and a set of modules rather than a single module - maybe they figured instead of providing various custom options and complicating the API, best to just provide one big function that lets you customise everything, you can always supply basic default parameters for things you don’t want to customise - which makes sense - and that’s what Kaleidoscope uses.
You pretty much nailed it. While Orc and MCJIT provide the same functionality, their design goals are different. MCJIT is a black-box that takes in modules and spits out linked code. It’s easy to use, but any customization requires new API to be exposed to enable/disable it, and because any change to the black box affects all MCJIT clients changes are considered relatively high-risk. Orc was designed as a component-library for building things like MCJIT - you take the bits you want off the shelf, wire them together, and you’ve got a custom JIT that does what you want without affecting any other JIT users - changes and customization are cheap and low-risk. In that spirit, addModuleSet provides a very flexible interface which you can then wrap in something more friendly if you don’t need all its features.
Orc also provides a class that contains a basic system for resolving symbols if you don’t want to customise, and that works by just supplying it with each symbol you’re going to want to look up, together with the associated function pointer - and that’s what you used.
MappingLayer provides a StringMap symbol table, similar to what I suggested in a previous email. Once you’ve added a symbol, it will appear in findSymbols as if you’d JIT’d it. When your resolver looks back into the JIT to search for definitions (a standard first step for a resolver) it’ll find anything you’ve mapped.
I prefer customizing the resolver (as in the example I gave in the other thread), rather than using the mapping layer. It feels cleaner to keep a distinction between the symbols owned by the JIT (which are the only ones you can find with findSymbols, unless you add the mapping layer), and the symbols the JIT needs (i.e. those found via the resolver). That said, there’s no real performance difference and it’s easy to switch between one scheme and the other once you have something that works, so go with whatever works for you.
Cheers,
Lang.