I’m trying to develop tranforms pass out-of-source. I create the Project folder out of the llvm source, and link the cmake to the llvm include header. After I compile the pass to DLL successfully , I found I can’t load it to the clang or opt.exe.
How can I make the pass as a DLL to let clang load on Windows platform? (I just want to make the pass project out of source to separate)， tks
Hello
This is quite non-trivial thing on Windows as DLL needs to import lots of things from the executable which is not what Windows binaries are designed for. You try to compile LLVM into DLL, but still there are lots of issues on the way.
1 Like
Hi,
Making this work on Windows is much tricker than on Linux or Mac OS. My suggestion would be to try on either Linux or Mac OS first.
This should in principle work on Windows too. I don’t have a Windows machine so I can’t really verify myself, but this patch: https://reviews.llvm.org/D18826 added LLVM_EXPORT_SYMBOLS_FOR_PLUGINS. You need to use that flag when building LLVM (yes, you need to build from sources) and hopefully that will be enough for the build system to export all the symbols that you may need.
I haven’t really tested this myself, so I would be really interested to hear if you get it to work 
Kind regards,
Andrzej
1 Like
I suddenly find the loadable modules don’t support my Windows when I cmake
And I check the
AddLLVM.cmake again, find that I should define ARG_PLUGIN_TOOL as well adapted to LLVM_EXPORT_SYMBOLS_FOR_PLUGINS
Thanks a lot. In a nutshell, IT WORKS!!!
After a long time struggling to compile and test, finally I successfully load the pass dll to opt, LLVM_EXPORT_SYMBOLS_FOR_PLUGINS really works. Thanks again.
Anyway, I write and build code under the llvm source code following the llvm documents. But each time I build the pass with VisualStudio or Clion on either Linux or Windows costs a lot of time.
I wonder whether I can build the pass out-of-source. I have follow some articles about out-of-source-tree build, the tips are quite rare, and I always fail to build. Because If build out-of-source, I can’t use llvm cmake command like add_llvm_loadable_module. I wonder whether there are some tips to notice when build pass out-of-source ( i just upload my pass project here GitHub - HAPPY-archive/tmp_pass: pass with fatal error :)
[ 93%] Building CXX object CMakeFiles/LLVMObfuscation.dir/src/AddPasses/Register_Passes.cpp.obj
[100%] Linking CXX shared library libLLVMObfuscation.dll
CMakeFiles\LLVMObfuscation.dir/objects.a(Register_Passes.cpp.obj): In function `llvm::RegisterStandardPasses::RegisterStandardPasses(llvm::PassManagerBuilder::ExtensionPointTy, std::function<void (llvm::PassManagerBuilder const&, llvm::legacy::PassManagerBase&)>)':
I:/llvm_research/llvm/include/llvm/Transforms/IPO/PassManagerBuilder.h:212: undefined reference to `llvm::PassManagerBuilder::addGlobalExtension(llvm::PassManagerBuilder::ExtensionPointTy, std::function<void (llvm::PassManagerBuilder const&, llvm::legacy::PassManagerBase&)>)'
libLLVMObfuscationCommon.a(PassScheduler.cpp.obj): In function `initializeObfuscationPassOnce':
I:/llvm_research/HikariCore/src/PassScheduler.cpp:225: undefined reference to `llvm::PassRegistry::registerPass(llvm::PassInfo const&, bool)'
libLLVMObfuscationCommon.a(PassScheduler.cpp.obj): In function `llvm::Value::assertModuleIsMaterialized() const':
I:/llvm_research/llvm/include/llvm/IR/Value.h:318: undefined reference to `llvm::Value::assertModuleIsMaterializedImpl() const'
libLLVMObfuscationCommon.a(PassScheduler.cpp.obj): In function `llvm::raw_ostream::operator<<(llvm::StringRef)':
I:/llvm_research/llvm/include/llvm/Support/raw_ostream.h:174: undefined reference to `llvm::raw_ostream::write(char const*, unsigned long long)'
libLLVMObfuscationCommon.a(PassScheduler.cpp.obj): In function `llvm::cl::parser<bool>::parser(llvm::cl::Option&)':
I:/llvm_research/llvm/include/llvm/Support/CommandLine.h:865: undefined reference to `llvm::cl::basic_parser<bool>::basic_parser(llvm::cl::Option&)'
libLLVMObfuscationCommon.a(PassScheduler.cpp.obj): In function `llvm::cl::parser<unsigned long long>::parser(llvm::cl::Option&)':
I:/llvm_research/llvm/include/llvm/Support/CommandLine.h:965: undefined reference to `llvm::cl::basic_parser<unsigned long long>::basic_parser(llvm::cl::Option&)'
libLLVMObfuscationCommon.a(PassScheduler.cpp.obj): In function `llvm::Obfuscation::runOnModule(llvm::Module&)':
I:/llvm_research/HikariCore/src/PassScheduler.cpp:123: undefined reference to `llvm::GlobalValue::isDeclaration() const'
I:/llvm_research/HikariCore/src/PassScheduler.cpp:146: undefined reference to `llvm::GlobalValue::isDeclaration() const'
I:/llvm_research/HikariCore/src/PassScheduler.cpp:164: undefined reference to `llvm::errs()'
1 Like
Hello @HAPPY,
I’m glad that it has worked! Could you please share your CMake command for building LLVM?
As for out-of-tree pass development, I’ve tried my best to document it here (it’s up-to-date):
- GitHub - banach-space/llvm-tutor: A collection of out-of-tree LLVM passes for teaching and learning
- 2019 LLVM Developers’ Meeting: A. Warzynski “Writing an LLVM Pass: 101” - YouTube
As you’ll notice, I don’t cover Windows. I have no access to any Windows machines ATM, but I would be very keen to learn what the additional requirements are.
Btw, in your GitHub project you don’t seem to use add_llvm_loadable_module - is that the latest version of your pass? And yes, you do get access to that CMake method even when working out-of-tree. It’s made available by calling find_package(LLVM REQUIRED CONFIG), which you already do.
I hope that this helps!
-Andrzej
Thanks for reply.
After set LLVM_EXPORT_SYMBOLS_FOR_PLUGINS ON in HandleLLVMOptions.cmake file, I build with the following cmake command.
cmake ../llvm -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_TARGETS_TO_BUILD="X86" -Wno-dev
Just to confirm and as a future reference for others - I have also been able to make it to work with LLVM-10. Here’s my CMake invocation for LLVM:
cmake -G Ninja -DLLVM_ENABLE_PROJECTS="clang" -DLLVM_EXPORT_SYMBOLS_FOR_PLUGINS=On ..\llvm
I used ninja, but AFAIK the generator doesn’t really matter here.
As for a CMake script for an out-of-tree pass for Windows: https://github.com/banach-space/llvm-tutor/blob/master/HelloWorld/CMakeLists.txt_for_windows (the HelloWorld directory contains everything that’s required to build the pass).
Hope this helps!
1 Like
Hello @banach-space and HAPPY!
I was following your instructions to build a sample plugin on Windows, and it works with LLVM_EXPORT_SYMBOLS_FOR_PLUGINS.
The bad news is that if I add any code to the plugin, I have linker issues, like some functions are not exported.
So if I add command line parsing option (cl::opt) code or AssumptionCacheTracker related code, I have a linker issues like this:
Error LNK2001 unresolved external symbol "class llvm::cl::OptionCategory llvm::cl::GeneralCategory" (?GeneralCategory@cl@llvm@@3VOptionCategory@12@A)
I’m building LLVM branch release/10.x, Hello sample. CMakefile contains Core and Support as link components, like this:
if(WIN32 OR CYGWIN)
set(LLVM_LINK_COMPONENTS Core Support)
endif()
Here is my modified HelloWorld plugin - Hastebin: Send and Save Text or Code Snippets for Free | Toptal®
I have annotated my code with the “Added by Alex” comment.
Could you give me hint where to look at? It seems LLVM_EXPORT_SYMBOLS_FOR_PLUGINS is not enough to get all required symbols for Windows plugin.
Thanks!
Hi @Alex-llvm,
Apologies for not responding earlier. I was looking for a good reference for this, but couldn’t find it and then forgot to reply 
.
This is based on my limited understanding of how dynamic linking works on Windows. Basically, on Windows every *.dll defines its externally visible symbols. If a symbol is not defined as externally visible, you cannot use it. However, there’s a limit: you can only export 2^16 symbols. Sadly, LLVM defines more than that and it does not specify what is an internal and what is an external symbol (i.e. it exports everything). See e.g. here.
In terms of plugins on Windows - if you are lucky then the *.dll that you are using exports everything that you need. Otherwise, there’s not much you can do as the limit of exported symbols has already been reached. For this reasons plugins are considered unsupported on Windows.
I hope that this makes sense. As I said - this is just my limited understanding I might have missed sth. I’m still trying to find a good reference!
Hey Andrzej,
Thanks for the update.
I found the root cause of the problem, which is this script - https://github.com/llvm/llvm-project/blob/master/llvm/utils/extract_symbols.py
Script is missing some of the symbols that need to be exported in opt.exe. I have made quick hack for the missed symbols, but this require a proper fix for the proper solution.
Btw, it seems we are still good with exported symbols limit in opt.exe, as now it is a little more than 2^15.
2 Likes
Hey Alex,
I’m glad that you managed to solve your problem! Btw, I found this discussion about extract_symbol.py quite helpful in understanding the context: https://reviews.llvm.org/D18826:
This Python script is a wonderful, horrible hack
All in all, this works (and is indeed wonderful), but sadly is not super robust
Would you please send your exact full command for compiling your llvm pass out of source on windows?
Thanks