Hi all,
I'm working to transition rust-bindgen from libclang to using the
clang libtooling C++ interface directly. This is unfortunately
necessary because there is a great deal of details about AST nodes
that we need but isn't exposed by libclang. I'm comfortable using and
maintaining a library interface on top of the C++ APIs, that's not a
problem.
However, I am hitting a wall on distribution for Windows. Would it at
all a possibility to include the new libclang-cpp library and headers
on Windows in the pre-built binaries? The only real alternative I see
is making users build LLVM+Clang from source, which seems prohibitive
in many cases, or distributing my own binary build, which I prefer not
to resort to for a single open-source Rust tool.
Adding C++ libraries on Windows potentially introduces issues by
exposing the C++ ABI directly, so I think we would have to specify
which C++ library and toolchain each release was built with.
Thanks,
Stephen
Including pre-build libtooling DLL in Windows binaries would be really helpful.
Right now we are building libtooling static libraries from the source code, it takes
several hours (debug/release/x86/x64 targets).
Best regards
Jan
As far as I know, these libraries are not supported on Windows.
Windows DLLs have a limit of 2^16 exported symbols, that LLVM
unfortunately exceeds.
Michael
Would the individual libclang* libraries (e.g. libclangAST,
libclangParse, etc.) exceed this limit? I'll do some experimental
builds and see what I can determine.
- stephen
What about DLL with simple COM-like interface (abstract class with pure virtual methods)?
Best regards
Jan
BUILD_SHARED_LIBS is also not supported on Windows, but I don't know
whether for the same reason.
Michael
This would be a new interface that someone had to out work into.
Note that in llvm/utils/extract_symbols.py there is a script that
tries to export only symbols that are actually needed (for limited
plugin support under Windows). In my experience, it filters out too
many symbols that are needed for non-trivial/example use cases (in my
case: Polly).
Michael
I just noticed this. I found an explanation from 2017:
http://lists.llvm.org/pipermail/llvm-dev/2017-June/113925.html Anyone
able to elaborate more on this?
I'm not super familiar with development on Windows, so I hadn't even
thought about exports. I'm testing out the cmake
WINDOWS_EXPORT_ALL_SYMBOLS property to see if it's at all possible to
build shared libs on Windows. I'm assuming distributing static
libraries is too fragile to consider?
- stephen
It's not supported when building with MSVC (or with Clang in MSVC mode), as it requires dllexport attributes on all symbols to be visible in the external interface of the DLL (or a def file listing all the exported symbols), and the interfaces of all the internal libraries aren't decorated with such attributes.
If building with a MinGW toolchain (either GCC/binutils based, or Clang/lld based), it can export all symbols if none are marked specifically for export with dllexport attributes - and BUILD_SHARED_LIBS should work in this configuration (at least builds with it enabled succeeded last I tested maybe 7 weeks ago).
// Martin
Cmake also supporte the option WINDOWS_EXPORT_ALL_SYMBOLS for msvc. It
just creates a .def (list of symbols to export when linking a dll)
file including all symbols . LLVM's extract_symbols.py does something
similar.
Michael
I was able to successfully build and use both x86 and x64 DLLs on
Windows 10 (MSVC 2019) with the following cmake invocation:
cmake ../llvm -GNinja -DLLVM_ENABLE_PROJECTS=llvm;clang
-DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PDB=ON
-DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_TARGETS_TO_BUILD=X86
-DCMAKE_C_COMPILER=cl -DCMAKE_CXX_COMPILER=cl -DLLVM_ENABLE_RTTI=Off
-DLLVM_BUILD_LLVM_DYLIB=On -DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=TRUE
I had to make a few changes to the build system (mostly commenting out
checks preventing building DLLs on Windows). I'd be happy to prep
patches to enable this mode properly in the LLVM cmake scripts, so we
can discuss the merits of that option separately.
- stephen
Great! We are looking forward to test your patches and DLLs.
Best regards
Jan
I've been trying to get some sort of usable C++ API DLLs built on
Windows, but I'm giving up for now. My next question: would shipping
static C++ libraries and the corresponding headers on Windows be an
option at all? We already do this for the Linux binary distributions.
Not sure who would be best to weigh in on that. Hans, Tom, any
thoughts on this?
Over the last week I've spent quite a bit of time attempting to build
reasonably useful LLVM and clang-cpp DLLs on Windows. I got LLVM 9 to
build with just the frontend, no targets, but the number of symbols
has grown since then and exceeds 64K. It looks like clang-cpp is still
under the threshold, but not by much. We could probably pick out a
minimal set of LLVM frontend libraries necessary to link the clang
frontend DLL, but that doesn't seem like a good solution to me. So
unfortunately, unless someone spends time improving the
extract_symbols.py or another similar approach, DLLs don't look viable
on Windows.
Thanks,
Stephen
I've been trying to get some sort of usable C++ API DLLs built on
Windows, but I'm giving up for now. My next question: would shipping
static C++ libraries and the corresponding headers on Windows be an
option at all? We already do this for the Linux binary distributions.
Not sure who would be best to weigh in on that. Hans, Tom, any
thoughts on this?
Over the last week I've spent quite a bit of time attempting to build
reasonably useful LLVM and clang-cpp DLLs on Windows. I got LLVM 9 to
build with just the frontend, no targets, but the number of symbols
has grown since then and exceeds 64K. It looks like clang-cpp is still
under the threshold, but not by much. We could probably pick out a
minimal set of LLVM frontend libraries necessary to link the clang
frontend DLL, but that doesn't seem like a good solution to me. So
unfortunately, unless someone spends time improving the
extract_symbols.py or another similar approach, DLLs don't look viable
on Windows.
If you look at this commit[1], I went through the targets
and explicitly marked all the symbols that needed to be public.
I'm not sure if this does better or worse than extract_symbols.py,
but I think it would be reasonable to start doing this in more
places throughout the code, e.g. in llvm/lib/CodeGen.
As LLVM maintainers for Fedora/RHEL, we also have an interest in reducing
the number of exported symbols as this helps make LTO optimizations
more effective.
-Tom
[1] https://github.com/llvm/llvm-project/commit/0dbcb3639451a7c20e2d5133b459552281e64455
That sounds great. I'm happy to help with that as well. In the
meantime, would distributing static libraries on Windows be a
reasonable option? The *NIX prebuilt tarballs already include the
static libraries for libClang*. I think that would also require
distributing static libs for LLVM as well to link libClang libs
against, but if we can trim the LLVM dylib down enough before the next
release that becomes a moot point.
Thanks,
- stephen