[lld] bug detecting undefined symbols in shared libraries

I have a test case where lld-5.0 fails to detect an undefined symbol (this bug also happens with clang/lld 4.0). I haven’t narrowed down exactly all the circumstances when this can occur, but in this case it was for a virtual method in a class defined in a shared library. If I build the executable with the raw object files, the linker notices the missing method (see test1 vs test2). This was tested on Ubuntu 14.04 using the llvm/clang-5.0 downloaded from the official apt repository.

If I use gold (from binutils 2.26), it finds the error during link time.

********** DEMO OUTPUT

$ make clean && make
rm -f main.o virtual.o virtual.so test1 test2
clang+±5.0 -fPIC -c main.cc
clang+±5.0 -fPIC -c virtual.cc
clang+±5.0 -fPIC -fuse-ld=lld-5.0 -shared -o virtual.so virtual.o
clang+±5.0 -fPIC -fuse-ld=lld-5.0 -Wl,-R$ORIGIN. -o test1 main.o virtual.so
clang+±5.0 -fPIC -fuse-ld=lld-5.0 -Wl,-R$ORIGIN. -o test2 main.o virtual.o
/usr/bin/ld.lld-5.0: error: undefined symbol: foo::bar()

referenced by virtual.cc
virtual.o:(vtable for foo)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [test2] Error 1

Note that running test1 results in an error:
$ ./test1
./test1: symbol lookup error: ./virtual.so: undefined symbol: _ZN3foo3barEv

********** CODE main.cc

#include “virtual.h”
int main() {
foo baz;
return 0;
}

********** CODE virtual.h
struct foo {
virtual ~foo();
virtual void bar();
};

********** CODE virtual.cc
#include “virtual.h”
foo::~foo() { }

********* CODE Makefile
all: test1 test2

CC = clang+±5.0 -fPIC
LD = ${CC} -fuse-ld=lld-5.0

test1: main.o virtual.so
${LD} -Wl,-R$$ORIGIN. -o test1 main.o virtual.so

test2: main.o virtual.o
${LD} -Wl,-R$$ORIGIN. -o test2 main.o virtual.o

virtual.o: virtual.cc virtual.h
${CC} -c virtual.cc

virtual.so: virtual.o
${LD} -shared -o virtual.so virtual.o

main.o: main.cc virtual.h
${CC} -c main.cc

clean:
rm -f main.o virtual.o virtual.so test1 test2

Hi Scott,

I think you are saying that lld doesn’t report an error for undefined symbols that exist in a .so which cannot be resolved at link-time. That’s intentional. Trying to resolve undefined symbols in a .so at link-time doesn’t make sense because we can just link fine without resolving such symbols, and the static linker doesn’t really know whether it’s actually an error condition or not. The real failure could only happen at load-time when the undefined symbols cannot be resolved using other DSOs. So we do not care about undefined symbols in DSOs in most cases.

With shared libraries, it is always possible that you end up running with a different set of libraries than you actually linked with. However, how often is that actually the case? It seems at the very least there should be an option to tell the user that unless they switch libraries, their code won’t be runnable. In most cases, everything is available to the linker to detect the situation (as both ld and gold do).

gold has a set of options to vary their behavior - --[no-]allow-shlib-undefined, --no-undefined, -z defs, --warn-unresolved-symbols, --error-unresolved-symbols, --unresolved-symbols ignore-all/report-all/ignore-in-object-files/ignore-in-shared-libs. It seems that lld is implementing --allow-shlib-undefined (test1 builds, test2 fails to build). Is it possible for lld to implement --[no-]allow-shlib-undefined to switch the behavior for undefined symbols in shared libraries?

$ make clean && make -k
rm -f main.o virtual.o virtual.so test1 test2
clang+±5.0 -fPIC -c main.cc
clang+±5.0 -fPIC -c virtual.cc
clang+±5.0 -fPIC -fuse-ld=gold -Wl,–allow-shlib-undefined -shared -o virtual.so virtual.o
clang+±5.0 -fPIC -fuse-ld=gold -Wl,–allow-shlib-undefined -Wl,-R$ORIGIN. -o test1 main.o virtual.so
clang+±5.0 -fPIC -fuse-ld=gold -Wl,–allow-shlib-undefined -Wl,-R$ORIGIN. -o test2 main.o virtual.o
virtual.o:virtual.cc:vtable for foo: error: undefined reference to ‘foo::bar()’
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [test2] Error 1
make: Target `all’ not remade because of errors.

Sure. I understand why you want it. I don’t want to make a conclusion right now, but at least we should evaluate the impact of adding --no-allow-shlib-undefined to lld. Filed your feature request as https://bugs.llvm.org/show_bug.cgi?id=35635 to keep this in my todo list.

Scott Smith via llvm-dev <llvm-dev@lists.llvm.org> writes:

With shared libraries, it is always possible that you end up running with a
different set of libraries than you actually linked with. However, how
often is that actually the case? It seems at the very least there should
be an option to tell the user that unless they switch libraries, their code
won't be runnable. In most cases, everything is available to the linker to
detect the situation (as both ld and gold do).

Your best option for checking that during link time is to use -z defs
when linking the shared library.

Cheers,
Rafael

I tried that. It doesn’t work in my application due to odd circular references between shared libraries. I need the validation to happen at executable link time, not shared library link time.