How to build shared and static libclang* files

I'm trying to build LLVM, clang and lld release/8.x from
https://github.com/llvm/llvm-project.

I'm on Arch Linux system using gcc:

  $ uname -a
  Linux wink-desktop 5.0.7-arch1-1-ARCH #1 SMP PREEMPT Mon Apr 8
10:37:08 UTC 2019 x86_64 GNU/Linux

  $ gcc --version
  gcc (GCC) 8.2.1 20181127
  Copyright (C) 2018 Free Software Foundation, Inc.
  This is free software; see the source for copying conditions. There is NO
  warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

One way I've been able to do this is to first build LLVM, clang and
lld which creates the static libraries, libclang*.a:

  $ git clone https://github.com/llvm/llvm-project && \
  cd llvm-project/ && \
  git checkout release/8.x && \
  cd llvm && \
  mkdir build && \
  cd build && \
  cmake .. -G Ninja -DCMAKE_INSTALL_PREFIX=$HOME/local
-DCMAKE_BUILD_TYPE=Release -DLLVM_BUILD_LLVM_DYLIB=ON
-DLLVM_ENABLE_PROJECTS="clang;lld" && \
  ninja install

At this point we have libclang*.a and libclang.so. But this
libclang.so only exposes the 380 C API globals and there are no
libclang*.so matching the libclang*.a files:

  wink@wink-desktop:~/local/lib
  $ ls libclang*
  libclangAnalysis.a libclangFrontendTool.a
libclang.so.8 libclangTidyModernizeModule.a
  libclangApplyReplacements.a libclangHandleCXX.a
libclangStaticAnalyzerCheckers.a libclangTidyMPIModule.a
  libclangARCMigrate.a libclangHandleLLVM.a
libclangStaticAnalyzerCore.a libclangTidyObjCModule.a
  libclangAST.a libclangIncludeFixer.a
libclangStaticAnalyzerFrontend.a libclangTidyPerformanceModule.a
  libclangASTMatchers.a libclangIncludeFixerPlugin.a
libclangTidy.a libclangTidyPlugin.a
  libclangBasic.a libclangIndex.a
libclangTidyAbseilModule.a libclangTidyPortabilityModule.a
  libclangChangeNamespace.a libclangLex.a
libclangTidyAndroidModule.a libclangTidyReadabilityModule.a
  libclangCodeGen.a libclangMove.a
libclangTidyBoostModule.a libclangTidyUtils.a
  libclangCrossTU.a libclangParse.a
libclangTidyBugproneModule.a libclangTidyZirconModule.a
  libclangDaemon.a libclangQuery.a
libclangTidyCERTModule.a libclangTooling.a
  libclangDoc.a libclangReorderFields.a
libclangTidyCppCoreGuidelinesModule.a libclangToolingASTDiff.a
  libclangDriver.a libclangRewrite.a
libclangTidyFuchsiaModule.a libclangToolingCore.a
  libclangDynamicASTMatchers.a libclangRewriteFrontend.a
libclangTidyGoogleModule.a libclangToolingInclusions.a
  libclangEdit.a libclangSema.a
libclangTidyHICPPModule.a libclangToolingRefactor.a
  libclangFormat.a libclangSerialization.a
libclangTidyLLVMModule.a
  libclangFrontend.a libclang.so
libclangTidyMiscModule.a
  wink@wink-desktop:~/local/lib
  $ objdump -tC libclang.so.8 | grep ' g ' | wc -l
  380

Now, if I do a second build from within clang and use BUILD_SHARED_LIBS=ON:

  $ cd ../../clang && \
  mkdir build && \
  cd build && \
  cmake .. -G Ninja -DCMAKE_INSTALL_PREFIX=$HOME/local
-DCMAKE_PREFIX_PATH=$HOME/local -DCMAKE_BUILD_TYPE=Release
-DBUILD_SHARED_LIBS=ON && \
  ninja install

And now have both libclang*.so and libclang*.a:

  wink@wink-desktop:~/local/lib
  $ ls libclang*
  libclangAnalysis.a libclangEdit.a
libclangReorderFields.a libclangTidyFuchsiaModule.a
  libclangAnalysis.so libclangEdit.so
libclangRewrite.a libclangTidyGoogleModule.a
  libclangAnalysis.so.8 libclangEdit.so.8
libclangRewriteFrontend.a libclangTidyHICPPModule.a
  libclangApplyReplacements.a libclangFormat.a
libclangRewriteFrontend.so libclangTidyLLVMModule.a
  libclangARCMigrate.a libclangFormat.so
libclangRewriteFrontend.so.8 libclangTidyMiscModule.a
  libclangARCMigrate.so libclangFormat.so.8
libclangRewrite.so libclangTidyModernizeModule.a
  libclangARCMigrate.so.8 libclangFrontend.a
libclangRewrite.so.8 libclangTidyMPIModule.a
  libclangAST.a libclangFrontend.so
libclangSema.a libclangTidyObjCModule.a
  libclangASTMatchers.a libclangFrontend.so.8
libclangSema.so libclangTidyPerformanceModule.a
  libclangASTMatchers.so libclangFrontendTool.a
libclangSema.so.8 libclangTidyPlugin.a
  libclangASTMatchers.so.8 libclangFrontendTool.so
libclangSerialization.a libclangTidyPortabilityModule.a
  libclangAST.so libclangFrontendTool.so.8
libclangSerialization.so libclangTidyReadabilityModule.a
  libclangAST.so.8 libclangHandleCXX.a
libclangSerialization.so.8 libclangTidyUtils.a
  libclangBasic.a libclangHandleCXX.so
libclang.so libclangTidyZirconModule.a
  libclangBasic.so libclangHandleCXX.so.8
libclang.so.8 libclangTooling.a
  libclangBasic.so.8 libclangHandleLLVM.a
libclangStaticAnalyzerCheckers.a libclangToolingASTDiff.a
  libclangChangeNamespace.a libclangHandleLLVM.so
libclangStaticAnalyzerCheckers.so libclangToolingASTDiff.so
  libclangCodeGen.a libclangHandleLLVM.so.8
libclangStaticAnalyzerCheckers.so.8 libclangToolingASTDiff.so.8
  libclangCodeGen.so libclangIncludeFixer.a
libclangStaticAnalyzerCore.a libclangToolingCore.a
  libclangCodeGen.so.8 libclangIncludeFixerPlugin.a
libclangStaticAnalyzerCore.so libclangToolingCore.so
  libclangCrossTU.a libclangIndex.a
libclangStaticAnalyzerCore.so.8 libclangToolingCore.so.8
  libclangCrossTU.so libclangIndex.so
libclangStaticAnalyzerFrontend.a libclangToolingInclusions.a
  libclangCrossTU.so.8 libclangIndex.so.8
libclangStaticAnalyzerFrontend.so libclangToolingInclusions.so
  libclangDaemon.a libclangLex.a
libclangStaticAnalyzerFrontend.so.8 libclangToolingInclusions.so.8
  libclangDoc.a libclangLex.so
libclangTidy.a libclangToolingRefactor.a
  libclangDriver.a libclangLex.so.8
libclangTidyAbseilModule.a libclangToolingRefactor.so
  libclangDriver.so libclangMove.a
libclangTidyAndroidModule.a libclangToolingRefactor.so.8
  libclangDriver.so.8 libclangParse.a
libclangTidyBoostModule.a libclangTooling.so
  libclangDynamicASTMatchers.a libclangParse.so
libclangTidyBugproneModule.a libclangTooling.so.8
  libclangDynamicASTMatchers.so libclangParse.so.8
libclangTidyCERTModule.a
  libclangDynamicASTMatchers.so.8 libclangQuery.a
libclangTidyCppCoreGuidelinesModule.a

So it "worked", but seems harder than it should be.

Is there a way to have both the libclang*.a and libclang*.so files
built at the same time while building LLVM?

-- Wink

I'm trying to build LLVM, clang and lld release/8.x from
https://github.com/llvm/llvm-project.

I'm on Arch Linux system using gcc:

  $ uname -a
  Linux wink-desktop 5.0.7-arch1-1-ARCH #1 SMP PREEMPT Mon Apr 8
10:37:08 UTC 2019 x86_64 GNU/Linux

  $ gcc --version
  gcc (GCC) 8.2.1 20181127
  Copyright (C) 2018 Free Software Foundation, Inc.
  This is free software; see the source for copying conditions. There is NO
  warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

One way I've been able to do this is to first build LLVM, clang and
lld which creates the static libraries, libclang*.a:

  $ git clone https://github.com/llvm/llvm-project && \
  cd llvm-project/ && \
  git checkout release/8.x && \
  cd llvm && \
  mkdir build && \
  cd build && \
  cmake .. -G Ninja -DCMAKE_INSTALL_PREFIX=$HOME/local
-DCMAKE_BUILD_TYPE=Release -DLLVM_BUILD_LLVM_DYLIB=ON
-DLLVM_ENABLE_PROJECTS="clang;lld" && \
  ninja install

At this point we have libclang*.a and libclang.so. But this
libclang.so only exposes the 380 C API globals and there are no
libclang*.so matching the libclang*.a files:

  wink@wink-desktop:~/local/lib
  $ ls libclang*
  libclangAnalysis.a libclangFrontendTool.a
libclang.so.8 libclangTidyModernizeModule.a
  libclangApplyReplacements.a libclangHandleCXX.a
libclangStaticAnalyzerCheckers.a libclangTidyMPIModule.a
  libclangARCMigrate.a libclangHandleLLVM.a
libclangStaticAnalyzerCore.a libclangTidyObjCModule.a
  libclangAST.a libclangIncludeFixer.a
libclangStaticAnalyzerFrontend.a libclangTidyPerformanceModule.a
  libclangASTMatchers.a libclangIncludeFixerPlugin.a
libclangTidy.a libclangTidyPlugin.a
  libclangBasic.a libclangIndex.a
libclangTidyAbseilModule.a libclangTidyPortabilityModule.a
  libclangChangeNamespace.a libclangLex.a
libclangTidyAndroidModule.a libclangTidyReadabilityModule.a
  libclangCodeGen.a libclangMove.a
libclangTidyBoostModule.a libclangTidyUtils.a
  libclangCrossTU.a libclangParse.a
libclangTidyBugproneModule.a libclangTidyZirconModule.a
  libclangDaemon.a libclangQuery.a
libclangTidyCERTModule.a libclangTooling.a
  libclangDoc.a libclangReorderFields.a
libclangTidyCppCoreGuidelinesModule.a libclangToolingASTDiff.a
  libclangDriver.a libclangRewrite.a
libclangTidyFuchsiaModule.a libclangToolingCore.a
  libclangDynamicASTMatchers.a libclangRewriteFrontend.a
libclangTidyGoogleModule.a libclangToolingInclusions.a
  libclangEdit.a libclangSema.a
libclangTidyHICPPModule.a libclangToolingRefactor.a
  libclangFormat.a libclangSerialization.a
libclangTidyLLVMModule.a
  libclangFrontend.a libclang.so
libclangTidyMiscModule.a
  wink@wink-desktop:~/local/lib
  $ objdump -tC libclang.so.8 | grep ' g ' | wc -l
  380

Now, if I do a second build from within clang and use BUILD_SHARED_LIBS=ON:

  $ cd ../../clang && \
  mkdir build && \
  cd build && \
  cmake .. -G Ninja -DCMAKE_INSTALL_PREFIX=$HOME/local
-DCMAKE_PREFIX_PATH=$HOME/local -DCMAKE_BUILD_TYPE=Release
-DBUILD_SHARED_LIBS=ON && \
  ninja install

And now have both libclang*.so and libclang*.a:

  wink@wink-desktop:~/local/lib
  $ ls libclang*
  libclangAnalysis.a libclangEdit.a
libclangReorderFields.a libclangTidyFuchsiaModule.a
  libclangAnalysis.so libclangEdit.so
libclangRewrite.a libclangTidyGoogleModule.a
  libclangAnalysis.so.8 libclangEdit.so.8
libclangRewriteFrontend.a libclangTidyHICPPModule.a
  libclangApplyReplacements.a libclangFormat.a
libclangRewriteFrontend.so libclangTidyLLVMModule.a
  libclangARCMigrate.a libclangFormat.so
libclangRewriteFrontend.so.8 libclangTidyMiscModule.a
  libclangARCMigrate.so libclangFormat.so.8
libclangRewrite.so libclangTidyModernizeModule.a
  libclangARCMigrate.so.8 libclangFrontend.a
libclangRewrite.so.8 libclangTidyMPIModule.a
  libclangAST.a libclangFrontend.so
libclangSema.a libclangTidyObjCModule.a
  libclangASTMatchers.a libclangFrontend.so.8
libclangSema.so libclangTidyPerformanceModule.a
  libclangASTMatchers.so libclangFrontendTool.a
libclangSema.so.8 libclangTidyPlugin.a
  libclangASTMatchers.so.8 libclangFrontendTool.so
libclangSerialization.a libclangTidyPortabilityModule.a
  libclangAST.so libclangFrontendTool.so.8
libclangSerialization.so libclangTidyReadabilityModule.a
  libclangAST.so.8 libclangHandleCXX.a
libclangSerialization.so.8 libclangTidyUtils.a
  libclangBasic.a libclangHandleCXX.so
libclang.so libclangTidyZirconModule.a
  libclangBasic.so libclangHandleCXX.so.8
libclang.so.8 libclangTooling.a
  libclangBasic.so.8 libclangHandleLLVM.a
libclangStaticAnalyzerCheckers.a libclangToolingASTDiff.a
  libclangChangeNamespace.a libclangHandleLLVM.so
libclangStaticAnalyzerCheckers.so libclangToolingASTDiff.so
  libclangCodeGen.a libclangHandleLLVM.so.8
libclangStaticAnalyzerCheckers.so.8 libclangToolingASTDiff.so.8
  libclangCodeGen.so libclangIncludeFixer.a
libclangStaticAnalyzerCore.a libclangToolingCore.a
  libclangCodeGen.so.8 libclangIncludeFixerPlugin.a
libclangStaticAnalyzerCore.so libclangToolingCore.so
  libclangCrossTU.a libclangIndex.a
libclangStaticAnalyzerCore.so.8 libclangToolingCore.so.8
  libclangCrossTU.so libclangIndex.so
libclangStaticAnalyzerFrontend.a libclangToolingInclusions.a
  libclangCrossTU.so.8 libclangIndex.so.8
libclangStaticAnalyzerFrontend.so libclangToolingInclusions.so
  libclangDaemon.a libclangLex.a
libclangStaticAnalyzerFrontend.so.8 libclangToolingInclusions.so.8
  libclangDoc.a libclangLex.so
libclangTidy.a libclangToolingRefactor.a
  libclangDriver.a libclangLex.so.8
libclangTidyAbseilModule.a libclangToolingRefactor.so
  libclangDriver.so libclangMove.a
libclangTidyAndroidModule.a libclangToolingRefactor.so.8
  libclangDriver.so.8 libclangParse.a
libclangTidyBoostModule.a libclangTooling.so
  libclangDynamicASTMatchers.a libclangParse.so
libclangTidyBugproneModule.a libclangTooling.so.8
  libclangDynamicASTMatchers.so libclangParse.so.8
libclangTidyCERTModule.a
  libclangDynamicASTMatchers.so.8 libclangQuery.a
libclangTidyCppCoreGuidelinesModule.a

So it "worked", but seems harder than it should be.

Is there a way to have both the libclang*.a and libclang*.so files
built at the same time while building LLVM?

I don't think this is possible.

-Tom

Txs for the reply, hard to believe its impossible, llvm itself can
build both static and shared libraries if -DLLVM_BUILD_LLVM_DYLIB=on.
And actually, as I mentioned in my original post, there is already a
clang shared library, libclang.so.8. The problem with it is that it
only exposes the 380 C API entry points, not the full set of entry
points available with the libclang[[:alpha:]]+.a versions.

Is there something I'm missing?

Tom, I misread your email and I dropped the "don't", please accept my apology.

Is there any reason NOT to try to build both for most/all subprojects?

Txs for the reply, hard to believe its impossible, llvm itself can
build both static and shared libraries if -DLLVM_BUILD_LLVM_DYLIB=on.
And actually, as I mentioned in my original post, there is already a
clang shared library, libclang.so.8. The problem with it is that it
only exposes the 380 C API entry points, not the full set of entry
points available with the libclang[[:alpha:]]+.a versions.

This is just a difference between llvm and clang. llvm has a special
llvm-shlib target that builds libLLVM.so when LLVM_BUILD_LLVM_DYLIB
is enabled. We could just as easily add this to clang, it's just
that no one has yet.

-Tom

OK, I'm going to give building both a try.

So I've got a proof of concept for building both SHARED and STATIC libraries.
The simple implementation that creates libclangARCMigrate.a and
libclangARCMigrate.so
is on my llvm-project fork:
https://github.com/winksaville/llvm-project/commit/fb314421c1dd404648e7046c6ef074e93d28e4c1

As it turns out llvm_add_library already supported building both, but
I had to have it build the shared library first rather than the static library.
Without this change clang can be built but it fails when run:

  $ ./bin/clang-9 --version
  : CommandLine Error: Option 'use-dbg-addr' registered more than once!
  LLVM ERROR: inconsistency in registered CommandLine options

The reason it fails is because it was linked with libclangARCMigrate.so.9svn
rahter than the correct library, libclangARCMigrate.a.

As you can see in the link above the code changes were fairly small:

  clang/cmake/modules/AddClang.cmake add_clang_library:

   - Add support for STATIC, SHARED or both
   - Add support for OUTPUT_NAME

  llvm/cmake/modules/AddLLVM.cmake llvm_add_library:

   - Changed so when both SHARED and STATIC are passed SHARED is built
     first. Without this change llvm_add_library causes the shared libraries
     to be linked to clang rather than the static library. This compiles
     successfully but clang fails when run with:

  clang/lib/ARCMigrate/CMakeLists.txt:

   - Added STATIC SHARED so both libclangARCMigrate.a and
     libclangARCMigrate.so are built

The result of building with the change is both static and shared libraries
of libclangARCMigrate are created:

  $ ls -al lib/*ARCM*
  -rw-r--r-- 1 wink users 8541496 May 7 17:37 lib/libclangARCMigrate.a
  lrwxrwxrwx 1 wink users 26 May 7 17:37
lib/libclangARCMigrate.so -> libclangARCMigrate.so.9svn
  -rwxr-xr-x 1 wink users 37597048 May 7 17:37 lib/libclangARCMigrate.so.9svn

And clang runs:

  $ ./bin/clang --version
  clang version 9.0.0 (git@github.com:winksaville/llvm-project
fb314421c1dd404648e7046c6ef074e93d28e4c1)
  Target: x86_64-unknown-linux-gnu
  Thread model: posix
  InstalledDir:
/home/wink/prgs/llvm/llvm-project/build-Support-building-shared-and-static-libraries-simultaneously/./bin

Feed back welcome!

-- Wink

I've updated my POC,
https://github.com/winksaville/llvm-project/tree/Support-building-shared-and-static-libraries-simultaneously,
so it now provides cmake options for building static and shared
libclang* libraries. By default its doing both. This is modeled after
how libunwind works.

I've updated my POC,
https://github.com/winksaville/llvm-project/tree/Support-building-shared-and-static-libraries-simultaneously,
so it now provides cmake options for building static and shared
libclang* libraries. By default its doing both. This is modeled after
how libunwind works.

Can you submit your patch via phabricator?

-Tom

Can you submit your patch via phabricator?

Will do, later today or tomorrow.

Will do, later today or tomorrow.

I’ve created https://reviews.llvm.org/D61804