Exported symbols from LLDB build products

Hello,

On Mac OS X, when using the xcode projects, there is some machinery to manage what symbols are exported from various things:

  • liblldb only exports the public API.

  • debugserver only exports _DNB* and __DNB*

  • argdumper exports nothing

  • lldb-server exports nothing

This machinery only exists within the xcode projects though and isn’t used in the cmake or autotools build systems.

I’m not interested in adding it to the autotools build system, but I would like to get this set up in the cmake build system.

The LLVM cmake code provides a helper function for managing this already in a cross-platform manner. The main difference is that it expects one fewer leading underscores in the exports files and a particular naming scheme for the files.

Managing the files and their names is pretty easy and I can handle that.

The other difference is a bit harder to manage unless we set up a script to run in the xcode build to perform the mapping (not hard for someone that knows xcode, I guess).

So:

  • Is this something we want on all platforms? I think the most important of these is limiting liblldb to only exporting the public API.

  • Should any other targets have their exported symbols similarly restricted? (Perhaps lldb-mi?)

  • Is someone willing to assist with a minor xcode change so that we don’t have to have a separate copy of 2 of the files for the xcode build?

Thanks!

  • Bruce

Hello,

On Mac OS X, when using the xcode projects, there is some machinery to manage what symbols are exported from various things:

  • liblldb only exports the public API.

It currently exports:

_ZN4lldb*
_ZNK4lldb*
init_lld*

  • debugserver only exports _DNB* and __DNB*

That is due to legacy stuff and debugserver actually doesn't need to export anything...

  • argdumper exports nothing
  • lldb-server exports nothing

No executable tools need to ever really export anything, so this can and should almost always be the rules we use in CMake to create executables, not shared libraries, but just executables.

This machinery only exists within the xcode projects though and isn't used in the cmake or autotools build systems.

We should fix this.

I'm not interested in adding it to the autotools build system, but I would like to get this set up in the cmake build system.

The LLVM cmake code provides a helper function for managing this already in a cross-platform manner. The main difference is that it expects one fewer leading underscores in the exports files and a particular naming scheme for the files.

Managing the files and their names is pretty easy and I can handle that.

The other difference is a bit harder to manage unless we set up a script to run in the xcode build to perform the mapping (not hard for someone that knows xcode, I guess).

Why? The http://llvm.org/svn/llvm-project/lldb/trunk/resources/lldb-framework-exports file is so simple and it won't change. I don't see the need to do anything special in Xcode to convert some random file into another format.

So:

  • Is this something we want on all platforms? I think the most important of these is limiting liblldb to only exporting the public API.

The lldb.so should limit its exports. It makes the shared library cleaner and easier for the dynamic loaders to deal with.

  • Should any other targets have their exported symbols similarly restricted? (Perhaps lldb-mi?)

lldb-mi should export nothing just like argdumper and lldb-server.

  • Is someone willing to assist with a minor xcode change so that we don't have to have a separate copy of 2 of the files for the xcode build?

I am not sure how this would work. I would rather not have something generating the http://llvm.org/svn/llvm-project/lldb/trunk/resources/lldb-framework-exports file as these files aren't too difficult to create, even if there are two copies. It currently contains:

__ZN4lldb*
__ZNK4lldb*
_init_lld*

These could easily be added to the linker flags for command line builds and they aren't going to be changing. So I don't see the need to jump through any hoops to make this happen since they won't change. Just make it happen in our CMake/make builds.

Greg

I was doing some more digging into this and experimentation and found that the Windows build appears to handle this already, but that’s because symbol visibility is different over there.

It looks like on non-Windows, we should be using the attribute((visibility(“hidden”))) for pretty much all of the LLDB classes, apart from the public API. With that set up correctly, we don’t need linker flags at all, and it gives the compiler additional information to generate better code (which linker options / scripts won’t provide).

llvm’s include file llvm/Support/Compiler.h provides a preprocessor definition LLVM_LIBRARY_VISIBILITY which can be applied to everything that is supposed to be internal. This is what clang and LLVM do to hide some things, but they provide a much broader API than LLDB does.

Would you be averse to LLVM_LIBRARY_VISIBILITY being added to all of our internal class definitions? (I don’t mind doing the work.) That seems like the better and more correct way to handle this.

If we can do that, we’d want to include “llvm/Support/Compiler.h” in just about everything to pick up the LLVM_LIBRARY_VISIBILITY definition. Would we be able to put that in one of the LLDB headers that is already included everywhere?

  • Bruce

Hmm, while the below would help, it wouldn’t prevent the llvm and clang symbols from being re-exported from the LLDB shared library.

I still think it would be good to do, but we’d still need the linker options (but that’s easy for Darwin, a bit more involved for Linux and FreeBSD, I think).

  • Bruce

I was doing some more digging into this and experimentation and found that the Windows build appears to handle this already, but that's because symbol visibility is different over there.

It looks like on non-Windows, we should be using the __attribute__((visibility("hidden"))) for pretty much all of the LLDB classes, apart from the public API. With that set up correctly, we don't need linker flags at all, and it gives the compiler additional information to generate better code (which linker options / scripts won't provide).

llvm's include file llvm/Support/Compiler.h provides a preprocessor definition LLVM_LIBRARY_VISIBILITY which can be applied to everything that is supposed to be internal. This is what clang and LLVM do to hide some things, but they provide a much broader API than LLDB does.

Would you be averse to LLVM_LIBRARY_VISIBILITY being added to all of our internal class definitions? (I don't mind doing the work.) That seems like the better and more correct way to handle this.

I would rather not do this because with all of the current work going into abstracting TypeSystem's so that they are pluggable so we can support multiple languages, we might end up making an extra lldb_internal.so that exports all lldb_private::* so that we can actually make directories that can be checked out from other repositories so that we can make dynamic plug-ins for languages that can be just inserted into the LLDB code base and will just compile. These would link against the internal LLDB code and the API for this would be expected to constantly change, there would be no guarantee of API compatability. We can't do dynamic plug-ins through our public interface (lldb::*). This would mean that LLDB.framework or lldb.so would link against this lldb_internal.so along with any dynamic plug-ins.

This might mean that we have a clang.so which completely encapsulates a Language plug-in that links against the internal lldb_private::* functions, but the shared library boundary can be used to hide different versions of llvm and clang. Think about one language plug-in that has its own copy of llvm::* and clang::*, but doesn't export any llvm or clang symbols like "clang.so", and another like "swift.so" that also contains a full copy of llvm and clang (different revisions of llvm or clang that found in clang.so). All llvm/clang code is hidden behind the shared library boundary with nothing from llvm/clang exported. I doubt we will go this way, but with all the work going on right now to make this happen we need to keep this option open. So please don't decorate any classes. It should be up to the build system to determine how the files are used, not up to classes to pre-determine this. Especially since the llvm/clang/lldb build system makes a bunch of static libraries (.a files), we really don't want to pre-bake any notions of how the code is to be used. This is a job for the linker.

If we can do that, we'd want to include "llvm/Support/Compiler.h" in just about everything to pick up the LLVM_LIBRARY_VISIBILITY definition. Would we be able to put that in one of the LLDB headers that is already included everywhere?

So I would prefer to use linker flags to control exports for now if we can. I know windows uses the declspec(__dllexport) to do this, but I do prefer to do this after the fact with linker flags if we can get away with it.

Greg