LTO and Optimized libraries don't mix

When using the ARM cross compiler we’ve run into an issue with LTO and optimized libraries.

Consider you have an optimized library opt.a, which contains a version of memcpy.

Compiling with LTO (something like),

clang myTest.c opt.a –flto –o myTest

causes myTest.c to get compiled to bitcode.

Then the bitcode gets passed to the linker. The linker looks through the bitcode (via the gold plugin) searching for symbols it needs to resolve. But any memcpy() calls in myTest.c got converted to llvm.memcpy() calls in the bitcode. Any symbols that start with ‘llvm.’ get ignored in LTOModule.cpp. So the linker doesn’t know that it needs memcpy. Now once the linker processes opt.a it doesn’t know that it needs memcpy, so the linker doesn’t keep the definition of memcpy from opt.a.

After the (full) compilation of myTest is complete, the linker winds up trying to pull memcpy from libc. But once it hits opt.a to process again, it sees a multiple definition issue.

It appears that optimized libraries and LTO don’t currently mix well. Does anyone have any insight about how this should be accomplished?

Daniel

Daniel,

This has been a recurring issue with LTO on darwin too. To fix most issues, I added an enhancement to the linker whenever it was dealing with LTO. After the LTO codegen was done, if any new undefined symbols appeared, the link made another pass looking for definitions in dylibs (DSO). This solved almost all issues because memcpy and friends are implemented in libSystem.dylib on darwin. The remaining cases were for static programs (embedded, EFI, etc) which don’t use dylibs. For them, the current work around is to add -U as needed on the linker command line (e.g. -U _memcpy) which forces the linker to retain a copy of that runtime utility function.

-Nick

I don't think gold has an option for doing a full second pass. What
is has is the ADD_INPUT_LIBRARY callback which should cause it to look
at the provided library again.

It is probably a good idea to try to find a convenient way to use it
from llvm. If the performance impact is not too horrible, it might
make sense to change gold-plugin.cpp to use that callback for every .a
that it sees that contains at least one native object file.

Cheers,
Rafael

I had tried to use the add_input _library as

(*add_input_library)("opt.a");

but got a seg fault every time. I'm not sure exactly how to use it, apparently.

I also noticed that in gold_plugin.cpp:onload(), that add_input_library is defined as

case LDPT_ADD_INPUT_LIBRARY:
add_input_library = tv->tv_u.tv_add_input_file;
break;

Seems odd that add_input_library doesn't actually use the tv_add_input_library. I changed mine locally to point to add_input_library, but still got a crash.

Am I using the callback correctly?

Actually I used it as

add_input_library("opt"), without the .a, as the documents say use it as you would with a -l. But the result is still the same. I also get a crash when I try to use the add_input_file() as well.

My memory is a bit fuzzy, but I think the options are
* Pass the library name (the stuff after the -l) to add_input_library.
* Pass the full file path (/home/me/llvm/the_lib.a) to add_input_file.

Can you try passing the full file path? Where is the crash coming
from? gold or the plugin?

Cheers,
Rafael