libLTO on linux

I'm looking into the possibility of building the libLTO plugin on Linux, and got tangled up in the build system.

Currently, tools/Makefile only includes 'lto' when the OS equals Darwin, while tools/lto/Makefile has an if-statement that builds libLTO as a static library on non-Darwin systems.

If I try to build libLTO as a dynamic library on Linux, it works fine, but I get an enormous warning from libtool:
*** Warning: Linking the shared library /home/nicholas/llvm-commit/Debug/lib/libLTO.la against the non-libtool
*** objects /home/nicholas/llvm-commit/Debug/lib/LLVMCppBackend.o /home/nicholas/llvm-commit/Debug/lib/LLVMMSIL.o /home/nicholas/llvm-commit/Debug/lib/LLVMCBackend.o /home/nicholas/llvm-commit/Debug/lib/LLVMXCore.o /home/nicholas/llvm-commit/Debug/lib/LLVMPIC16.o /home/nicholas/llvm-commit/Debug/lib/LLVMCellSPUCodeGen.o /home/nicholas/llvm-commit/Debug/lib/LLVMCellSPUAsmPrinter.o /home/nicholas/llvm-commit/Debug/lib/LLVMMips.o /home/nicholas/llvm-commit/Debug/lib/LLVMARMAsmPrinter.o /home/nicholas/llvm-commit/Debug/lib/LLVMARMCodeGen.o /home/nicholas/llvm-commit/Debug/lib/LLVMIA64.o /home/nicholas/llvm-commit/Debug/lib/LLVMAlphaCodeGen.o /home/nicholas/llvm-commit/Debug/lib/LLVMAlphaAsmPrinter.o /home/nicholas/llvm-commit/Debug/lib/LLVMPowerPCAsmPrinter.o /home/nicholas/llvm-commit/Debug/lib/LLVMPowerPCCodeGen.o /home/nicholas/llvm-commit/Debug/lib/LLVMSparcCodeGen.o /home/nicholas/llvm-commit/Debug/lib/LLVMSparcAsmPrinter.o /home/nicholas/llvm-commit/Debug/lib/LLVMX86AsmPrinter.o /home/nicholas/llvm-commit/Debug/lib/LLVMX86CodeGen.o is not portable!

If we're going to build a .so file, we need those .o's to be built with -fPIC. As far as I can see, configuring LLVM with --enable-pic actually does that already, so perhaps we can make it the default.

But that warning isn't about PIC, it's about the fact that the .o's weren't compiled with libtool. It's looking for the matching .lo files.

It's not clear to me what we ought to do here. Put 'lto' inside an if ENABLE_PIC and suppress the warning? Build everything in LLVM with libtool --mode=compile? Is there a simple way to limit that to just the .o files we end up installing?

Nick

I'm looking into the possibility of building the libLTO plugin on Linux,
and got tangled up in the build system.

Currently, tools/Makefile only includes 'lto' when the OS equals Darwin,
while tools/lto/Makefile has an if-statement that builds libLTO as a
static library on non-Darwin systems.

If I try to build libLTO as a dynamic library on Linux, it works fine,
but I get an enormous warning from libtool:
*** Warning: Linking the shared library
/home/nicholas/llvm-commit/Debug/lib/libLTO.la against the non-libtool
*** objects /home/nicholas/llvm-commit/Debug/lib/LLVMCppBackend.o
/home/nicholas/llvm-commit/Debug/lib/LLVMMSIL.o
/home/nicholas/llvm-commit/Debug/lib/LLVMCBackend.o
/home/nicholas/llvm-commit/Debug/lib/LLVMXCore.o
/home/nicholas/llvm-commit/Debug/lib/LLVMPIC16.o
/home/nicholas/llvm-commit/Debug/lib/LLVMCellSPUCodeGen.o
/home/nicholas/llvm-commit/Debug/lib/LLVMCellSPUAsmPrinter.o
/home/nicholas/llvm-commit/Debug/lib/LLVMMips.o
/home/nicholas/llvm-commit/Debug/lib/LLVMARMAsmPrinter.o
/home/nicholas/llvm-commit/Debug/lib/LLVMARMCodeGen.o
/home/nicholas/llvm-commit/Debug/lib/LLVMIA64.o
/home/nicholas/llvm-commit/Debug/lib/LLVMAlphaCodeGen.o
/home/nicholas/llvm-commit/Debug/lib/LLVMAlphaAsmPrinter.o
/home/nicholas/llvm-commit/Debug/lib/LLVMPowerPCAsmPrinter.o
/home/nicholas/llvm-commit/Debug/lib/LLVMPowerPCCodeGen.o
/home/nicholas/llvm-commit/Debug/lib/LLVMSparcCodeGen.o
/home/nicholas/llvm-commit/Debug/lib/LLVMSparcAsmPrinter.o
/home/nicholas/llvm-commit/Debug/lib/LLVMX86AsmPrinter.o
/home/nicholas/llvm-commit/Debug/lib/LLVMX86CodeGen.o is not portable!

If we're going to build a .so file, we need those .o's to be built with
-fPIC. As far as I can see, configuring LLVM with --enable-pic actually
does that already, so perhaps we can make it the default.

But that warning isn't about PIC, it's about the fact that the .o's
weren't compiled with libtool. It's looking for the matching .lo files.

My question is this, why use libtool to build the .so? I don't see
what libtool helps with here, and it is a nightmare to use in general.
Why not directly use the gcc driver?

-Chandler

I'd love to rip out libtool. What benefit does it provide?

-Chris

If we're going to build a .so file, we need those .o's to be built
with
-fPIC. As far as I can see, configuring LLVM with --enable-pic
actually
does that already, so perhaps we can make it the default.

But that warning isn't about PIC, it's about the fact that the .o's
weren't compiled with libtool. It's looking for the matching .lo
files.

My question is this, why use libtool to build the .so? I don't see
what libtool helps with here, and it is a nightmare to use in general.
Why not directly use the gcc driver?

I'd love to rip out libtool. What benefit does it provide?

Several that seem worth noting (although none seem like show stoppers):

- management of the linker options (this is simplified by the
presence of "g{cc,++} -shared" driver-based linking)
- "automatic" library naming rules (lib- prefix, .so versioning, etc,
nothing terrible here)
- automatic LD_LIBRARY_PATH setup for binaries build against shared
libraries and executed prior to the libraries' installation into a
system path

The last one is probably the most relevant, but even then, the
solution is horribly ugly: libtool produces shell scripts rather than
actual programs that setup the LD_LIBRARY_PATH variables and then
execute a hidden executable. Ugh. For a *shared* library, I've never
understood why you wouldn't just re-link the executable upon install
(and have the build tree variant be linked explicitly against the full
path to the .so). Re-linking an executable's shared libraries should
be extremely fast.

My 2 cents. This isn't trivial, but it also gives you freedom (as
evidenced by Nick's issues above). It could also be done
incrementally, as you probably don't need all of that all at once as
LLVM is primarily built as static libraries today. The first
increments shouldn't be too rough.

-Chandler