I tried building llvm, clang, lld, lldb from the 3.9 svn release
branch with LTO, and some of the results were unexpected.
I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by
providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I
fell back to building with -DLLVM_ENABLE_LTO=On and using the system
CC/CXX (gcc 6.1).
The resulting installed build is many times bigger than the non-LTO
version. Is this to be expected? I thought LTO would reduce size by
detecting more unused code.
The bindist .tar.xz archives are as follows:
LTO=Off: 195MB
LTO=On : 953MB
Now, I am of course aware that falling back to gcc's lto support is
not the same as llvm's lto or thinlto, but I only fell back after it
failed to build that way, and I wasn't aware of gcc6 providing almost
5x large code than without lto.
Also, given a fresh 3.9 install, on an otherwise gcc/libstdc++ system,
how can I configure and build with -DLLVM_ENABLE_LTO=Thin? Is this
supported, or does it require the use of some libXX too?
I tried building llvm, clang, lld, lldb from the 3.9 svn release
branch with LTO, and some of the results were unexpected.
I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by
providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I
fell back to building with -DLLVM_ENABLE_LTO=On and using the system
CC/CXX (gcc 6.1).
The resulting installed build is many times bigger than the non-LTO
version. Is this to be expected? I thought LTO would reduce size by
detecting more unused code.
LTO also inline more, thus increasing the size. But usually it is quite slight.
If you’re concerned about binary size, disabling clang plugin may help:
cmake -DCLANG_PLUGIN_SUPPORT=OFF
The bindist .tar.xz archives are as follows:
LTO=Off: 195MB
LTO=On : 953MB
Now, I am of course aware that falling back to gcc's lto support is
not the same as llvm's lto or thinlto, but I only fell back after it
failed to build that way, and I wasn't aware of gcc6 providing almost
5x large code than without lto.
GCC may be using “fat” intermediate objects by default, i.e. it generates both the binary and the IR in the intermediates files and the static archives.
What is the size of the final clang binary itself?
Also, given a fresh 3.9 install, on an otherwise gcc/libstdc++ system,
how can I configure and build with -DLLVM_ENABLE_LTO=Thin? Is this
supported, or does it require the use of some libXX too?
You need to have Gold installed, but that is the same requirement as for LTO I believe.
Then if you’re using 3.9 as a host compiler, it should “just work”.
I tried building llvm, clang, lld, lldb from the 3.9 svn release
branch with LTO, and some of the results were unexpected.
I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by
providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I
fell back to building with -DLLVM_ENABLE_LTO=On and using the system
CC/CXX (gcc 6.1).
The resulting installed build is many times bigger than the non-LTO
version. Is this to be expected? I thought LTO would reduce size by
detecting more unused code.
LTO also inline more, thus increasing the size. But usually it is quite slight.
If you’re concerned about binary size, disabling clang plugin may help:
cmake -DCLANG_PLUGIN_SUPPORT=OFF
Curious to know why this would result in smaller binaries.
The bindist .tar.xz archives are as follows:
LTO=Off: 195MB
LTO=On : 953MB
Now, I am of course aware that falling back to gcc's lto support is
not the same as llvm's lto or thinlto, but I only fell back after it
failed to build that way, and I wasn't aware of gcc6 providing almost
5x large code than without lto.
GCC may be using “fat” intermediate objects by default, i.e. it
generates both the binary and the IR in the intermediates files
and the static archives.
Are you saying I should strip files?
What is the size of the final clang binary itself?
Interestingly the non-LTO clang-3.9 is 49MB while the LTO one if 58MB.
Also, given a fresh 3.9 install, on an otherwise gcc/libstdc++ system,
how can I configure and build with -DLLVM_ENABLE_LTO=Thin? Is this
supported, or does it require the use of some libXX too?
You need to have Gold installed, but that is the same requirement as for LTO I believe.
Then if you’re using 3.9 as a host compiler, it should “just work”.
$ c++ -v
gcc version 6.2.1 20160830 (GCC)
$ ld.gold --version
GNU gold (GNU Binutils 2.27) 1.12
$ clang -v
clang version 3.9.0 (branches/release_39 279763)
I tried building llvm, clang, lld, lldb from the 3.9 svn release
branch with LTO, and some of the results were unexpected.
I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by
providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I
fell back to building with -DLLVM_ENABLE_LTO=On and using the system
CC/CXX (gcc 6.1).
The resulting installed build is many times bigger than the non-LTO
version. Is this to be expected? I thought LTO would reduce size by
detecting more unused code.
LTO also inline more, thus increasing the size. But usually it is quite slight.
If you’re concerned about binary size, disabling clang plugin may help:
cmake -DCLANG_PLUGIN_SUPPORT=OFF
Curious to know why this would result in smaller binaries.
Supporting Plugin implies that every function has to available, even if inlined everywhere they can’t be dead-stripped. Because the plugin may call into it.
Disabling plugin allows to remove all this dead code.
The bindist .tar.xz archives are as follows:
LTO=Off: 195MB
LTO=On : 953MB
Now, I am of course aware that falling back to gcc’s lto support is
not the same as llvm’s lto or thinlto, but I only fell back after it
failed to build that way, and I wasn’t aware of gcc6 providing almost
5x large code than without lto.
GCC may be using “fat” intermediate objects by default, i.e. it
generates both the binary and the IR in the intermediates files
and the static archives.
Are you saying I should strip files?
I’m just saying, it depends what you’re measuring: what matters is the final binary usually, not the size of the intermediate files.
What is the size of the final clang binary itself?
Interestingly the non-LTO clang-3.9 is 49MB while the LTO one if 58MB.
Also, given a fresh 3.9 install, on an otherwise gcc/libstdc++ system,
how can I configure and build with -DLLVM_ENABLE_LTO=Thin? Is this
supported, or does it require the use of some libXX too?
You need to have Gold installed, but that is the same requirement as for LTO I believe.
Then if you’re using 3.9 as a host compiler, it should “just work”.
$ c++ -v
gcc version 6.2.1 20160830 (GCC)
$ ld.gold --version
GNU gold (GNU Binutils 2.27) 1.12
$ clang -v
clang version 3.9.0 (branches/release_39 279763)
So ThinLTO should “just work” if you’re building with clang.
You may need to use llvm-ar/llvm-ranlib though:
cmake -DCMAKE_AR=path/to/llvm-ar -DCMAKE_RANLIB=path/to/llvm-ranlib
I tried building llvm, clang, lld, lldb from the 3.9 svn release
branch with LTO, and some of the results were unexpected.
I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by
providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I
I've been able to do a ThinLTO build of clang for some time with gold. I'm
not sure if there was a regression when the 3.9 release was cut? Can you
send me the error?
Thanks,
Teresa
I'll share the result and any reproduceable error (if there are any), after
the next build tests.
Anything else I should override, when I'm already using more than CC and CXX
Supporting Plugin implies that every function has to available, even if
inlined everywhere they can’t be dead-stripped. Because the plugin may call
into it.
Disabling plugin allows to remove all this dead code.
That makes sense, thanks.
I’m just saying, it depends what you’re measuring: what matters is the final
binary usually, not the size of the intermediate files.
I had only measured the size of the installed tree and assumed any
stripping that might happen would be the same as with LTO=OFF.
Let's see if I can get ThinLTO+No_PLUGINS to build and the results of
that. Might take a while.
Cc’ing llvm-dev again in case someone knows what is going on.
That error is coming from a polly configure script, which I don’t have cloned into my own tree. Not sure why polly would behave differently when configuring for ThinLTO. Does polly configure ok when configuring the same way, but for regular LTO?
Teresa
Cc'ing llvm-dev again in case someone knows what is going on.
That error is coming from a polly configure script, which I don't
have cloned into my own tree. Not sure why polly would behave
differently when configuring for ThinLTO. Does polly configure ok
when configuring the same way, but for regular LTO?
Yes it does, and I don't remember when I've added it, meaning it's
been working when bootstrapping llvm with the host gcc toolchain, for
a long time.
Can you look for the specific errors in the CMake log and error files?
CMake is very good at not presenting the real issue…
Here it is trying to build very simple programs to check features availability. Like:
#include <strings.h>
int main() { ffs(0); return 0; }
>
> Cc'ing llvm-dev again in case someone knows what is going on.
>
> That error is coming from a polly configure script, which I don't
> have cloned into my own tree. Not sure why polly would behave
> differently when configuring for ThinLTO. Does polly configure ok
> when configuring the same way, but for regular LTO?
Yes it does, and I don't remember when I've added it, meaning it's
been working when bootstrapping llvm with the host gcc toolchain, for
a long time.
You mentioned in your original email:
I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by
providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I
fell back to building with -DLLVM_ENABLE_LTO=On and using the system
CC/CXX (gcc 6.1).
I assume this error when you try to bootstrap using llvm-3.9 and ThinLTO
then? What about bootstrapping with llvm-3.9 and -DLLVM_ENABLE_LTO=On?
If the latter works, I'll have to clone polly and see if I can reproduce it
and see what is different between a -DLLVM_ENABLE_LTO=Thin
and -DLLVM_ENABLE_LTO=On build then.
Teresa
In CMakeError.log there are errors like:
CheckIncludeFile.c:1:10: fatal error: 'malloc/malloc.h' file not found
#include <malloc/malloc.h>
...
CheckIncludeFile.c:1:10: fatal error: 'ndir.h' file not found
#include <ndir.h>
...
CheckIncludeFile.c:1:10: fatal error: 'sys/ndir.h' file not found
#include <sys/ndir.h>
...
fatal error: 'intrin.h' file not found
These seem like tests that are expected to fail.
...
The interesting test
int main() { __builtin_ffs(0); return 0; }
fails like this:
/usr/bin/ld[...]llvm/bin/../lib/LLVMgold.so:
cannot open shared object file: No such file[...]
I have two questions here:
Is there anything I might have missed when configuring the gcc-built
llvm install for it to be incomplete in the linker department? I mean,
the system linker is used (not lld) and that via CC=clang is looking
for the gold linker plugin, which seems to be missing. Probably
something got lost in my transition from autoconf to cmake?
Anything else to pass to cmake so that it, in addition to llvm-ar and
llvm-ranlib, also used lld? Or isn't that a requirement for building
with ThinLTO?
Same error, and as mentioned in reply to Mehdi, I'll have to verify it there
isn't an underlying issue of my local llvm install.
However, I would certainly welcome a success report and binary size info.
Can you look for the specific errors in the CMake log and error files?
CMake is very good at not presenting the real issue…
Here it is trying to build very simple programs to check features
availability. Like:
#include <strings.h>
int main() { ffs(0); return 0; }
In CMakeError.log there are errors like:
CheckIncludeFile.c:1:10: fatal error: 'malloc/malloc.h' file not found
#include <malloc/malloc.h>
...
CheckIncludeFile.c:1:10: fatal error: 'ndir.h' file not found
#include <ndir.h>
...
CheckIncludeFile.c:1:10: fatal error: 'sys/ndir.h' file not found
#include <sys/ndir.h>
...
fatal error: 'intrin.h' file not found
These seem like tests that are expected to fail.
...
The interesting test
int main() { __builtin_ffs(0); return 0; }
fails like this:
/usr/bin/ld[...]llvm/bin/../lib/LLVMgold.so:
cannot open shared object file: No such file[…]
I assume that "/usr/bin/ld[...]llvm/bin” is the path where clang is installed right?
It seems that when you built the stage1 clang, it didn’t find Gold installed, and then didn’t build the plugin.
So you need to go back to the first build (not the ThinLTO one, the one with the system compiler), and make sure that CMake finds gold.
I have two questions here:
Is there anything I might have missed when configuring the gcc-built
llvm install for it to be incomplete in the linker department? I mean,
the system linker is used (not lld) and that via CC=clang is looking
for the gold linker plugin, which seems to be missing. Probably
something got lost in my transition from autoconf to cmake?
Anything else to pass to cmake so that it, in addition to llvm-ar and
llvm-ranlib, also used lld? Or isn't that a requirement for building
with ThinLTO?
No requirement for lld.
You probably missed -DLLVM_BINUTILS_INCDIR.
See: http://llvm.org/docs/GoldPlugin.html
plugin-api.h is in /usr/include, so I'd expect it to be found, but I
can explicitly set BINUTILS_INCDIR and re-bootstrap with gcc 6.2.1.
I have ld.gold, but I'm not sure if /usr/bin/ld uses it, though I'd expect
it to since it's been in for a couple releases now.
$ ld -v
GNU ld (GNU Binutils) 2.27
$ ld.bfd -v
GNU ld (GNU Binutils) 2.27
$ ld.gold -v
GNU gold (GNU Binutils 2.27) 1.12
I assume that "/usr/bin/ld[...]llvm/bin” is the path where clang is installed right?
Correct.
It seems that when you built the stage1 clang, it didn’t find Gold installed,
and then didn’t build the plugin.
So you need to go back to the first build (not the ThinLTO one, the one with
the system compiler), and make sure that CMake finds gold.
Going to re-bootstrap since the cmake transition somehow dropped that here.
Looks like your default ld is GNU ld.bfd not ld.gold. You can either change
your /usr/bin/ld (which probably is a link to /usr/bin/ld.bfd) to point
instead to /usr/bin/ld.gold, or if you prefer, set your PATH before the
stage1 compile to a location that has ld linked to ld.gold.
Teresa