LTO on FreeBSD

Hi.

I'm testing out clang on FreeBSD & OS X and have a few quick questions.

On the Mac, I've managed to compile some basic C code with no issues. I can compile with the -O4 or -flto flags and everything works as expected.
On the FreeBSD side, I can compile with clang just fine, so long as I keep it at -O3 or lower. If I try to use any of the linktime optimization flags, I get the error:

clang: error: 'i386-portbld-freebsd9.0': unable to pass LLVM bit-code files to linker

From my admittedly limited knowledge of clang/llvm, my understanding is that it's trying to link using the standard GNU binutils/ld instead of llvm-ld. GNU ld then chokes, not knowing what to do with the llvm-generated data. Is this right?

What is the OS X version finding and using that the FreeBSD version isn't? I see mention on the llvm site of compiling and using the ld gold plugins, but that appears to be more for llvm-gcc than clang.

I'd appreciate any pointers in this area.

Thanks,
SK

Exactly that. The FreeBSD linker does not support LTO. Gold is GPLv3, so is unlikely to ever make it into the FreeBSD base system. If you want to use LTO on FreeBSD, then you have to do it manually, via llvm-link + opt + llc. So far, no one has been sufficiently motivated to make the clang driver able to do this itself.

David

Just theoretically, would it be relatively easy to make the driver do
it? It sure would make -O4 a lot more enticing.

What about llvm-ld? Should it not be useable as a linker on FreeBSD?

Erik

llvm-ld is half-baked at best, it doesn't know a lot of options that your system linker does. It may work for simple things, but probably not for largish real world programs.

-Chris

What about llvm-ld? Should it not be useable as a linker on FreeBSD?

llvm-ld is half-baked at best, it doesn't know a lot of options that your system linker does. It may work for simple things, but probably not for largish real world programs.

Ah, I had been curious about this as well. Is the long-term goal for llvm-ld to be a full drop-in replacement for the system linker?

So, my original assumption that the OS X clang is using ld with the gold plugins was correct? If so, I'm curious where it's hidden away. I see a lot of libLLVMContainer.dylib files under the /Developer tree, but no files relating to "gold" anywhere.

It sounds like my only option on the FreeBSD side is to try and compile the llvm gold plugins. I've pulled down the cvs source and will try my hand at compiling it. Has anyone had any luck with compiling this package under FreeBSD or will I have to resign myself to a lot of hand-tweaking?

As there's some talk of migrating over to clang in the long-term, I'd be interested to see if/how the FreeBSD folks are planning to implement LTO without using any GNU code.

Thanks for all the helpful info!
-SK

No. Apple ld is not GNU ld and it does not use the gold plugin. It uses libLTO.dylib directly (if present).

-- Jean-Daniel

What about llvm-ld? Should it not be useable as a linker on FreeBSD?

llvm-ld is half-baked at best, it doesn't know a lot of options that your system linker does. It may work for simple things, but probably not for largish real world programs.

Ah, I had been curious about this as well. Is the long-term goal for llvm-ld to be a full drop-in replacement for the system linker?

I don't think anyone is working on llvm-ld anymore. This was the original goal, but it turns out that every system has a different set of linker options so it's not worth the effort. It might be worth FreeBSD forking llvm-ld and producing a version that can drive FreeBSD's ld and implements all of the options that this linker implements, but probably not.

So, my original assumption that the OS X clang is using ld with the gold plugins was correct?

Nope. OS X clang uses the Darwin linker, which has native LLVM support via libLTO.dylib

If so, I'm curious where it's hidden away. I see a lot of libLLVMContainer.dylib files under the /Developer tree, but no files relating to "gold" anywhere.

As espected.

It sounds like my only option on the FreeBSD side is to try and compile the llvm gold plugins. I've pulled down the cvs source and will try my hand at compiling it. Has anyone had any luck with compiling this package under FreeBSD or will I have to resign myself to a lot of hand-tweaking?

I don't know if gold supports FreeBSD. Last time I tried it, it compiled fine, but the result of using it was a load of nonsense (even without the plugin).

As there's some talk of migrating over to clang in the long-term, I'd be interested to see if/how the FreeBSD folks are planning to implement LTO without using any GNU code.

There are plans as part of the elftoolchain project to write a BSD-licensed linker, which will probably eventually replace GNU ld in FreeBSD. I'd imagine that, like the Darwin linker, this will end up with native support for LTO via LLVM.

David

So, my original assumption that the OS X clang is using ld with the gold plugins was correct?

No, OS X ld is not connected to gold in any way. It's completely
separate implementation.

Ah, I had been curious about this as well. Is the long-term goal for llvm-ld to be a full drop-in replacement for the system linker?

Looks hard, as it would have to be really generic to support all the
different systems.

So, my original assumption that the OS X clang is using ld with the gold plugins was correct? If so, I'm curious where it's hidden away. I see a lot of libLLVMContainer.dylib files under the /Developer tree, but no files relating to "gold" anywhere.

There is a gold plugin. That should work on freebsd, but yes, gold is
gpl3. If working on the bsd linker, you should be able to implement a
plugin support just use libLTO directly.

Note that you probably also have to update nm and ar. The GNU ones can
use the same plugin as gold.

Thanks for all the helpful info!
-SK

Cheers,