LLVM r201936 restores the use of a relative rpath but this breaks LLDB
on FreeBSD. For now I've disabled CMAKE_BUILD_WITH_INSTALL_RPATH
locally until the fix can be found, in the top-level LLVM
CMakeLists.txt:
if(NOT ${CMAKE_SYSTEM_NAME} MATCHES FreeBSD )
set(CMAKE_BUILD_WITH_INSTALL_RPATH ON)
endif()
It seems probable that either ld flags aren't being set correctly to
indicate that $ORIGIN is in use, or that there's an issue in FreeBSD's
handling of $ORIGIN (on at least FreeBSD 9.2).
Which library fails to load? Is the rpath set correctly? Does it work
with the configure build? Which rpath do you get with it?
Cheers,
Rafael
LLVM r201936 restores the use of a relative rpath but this breaks LLDB
on FreeBSD. For now I've disabled CMAKE_BUILD_WITH_INSTALL_RPATH
locally until the fix can be found, in the top-level LLVM
CMakeLists.txt:if(NOT ${CMAKE_SYSTEM_NAME} MATCHES FreeBSD )
set(CMAKE_BUILD_WITH_INSTALL_RPATH ON)
endif()It seems probable that either ld flags aren't being set correctly to
indicate that $ORIGIN is in use, or that there's an issue in FreeBSD's
handling of $ORIGIN (on at least FreeBSD 9.2).
Hi Rafael - normally I would wait until I had useful information to
post something like this, but I wanted to make sure anyone else who
runs into the same problem is aware that something's up and can work
around it in the short term.
Which library fails to load?
liblldb - here's what lld shows:
bin/lldb:
liblldb.so.3.5 => not found (0)
librt.so.1 => /usr/lib/librt.so.1 (0x80082f000)
libtinfo.so.5.9 => /usr/local/lib/libtinfo.so.5.9 (0x800a34000)
<and system and Python-related libraries>
Is the rpath set correctly?
It looks like it:
feynman% readelf -d bin/lldb | grep PATH
0x000000000000000f (RPATH) Library rpath: [$ORIGIN/../lib]
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib]
Does it work with the configure build?
The buildbot uses the configure build and is green, so it "works" I
guess -- but I do not know if it's using $ORIGIN or an absolute rpath,
and will have to wait until I can complete a configure build locally
to find out.
Ed
liblldb - here's what lld shows:
bin/lldb:
liblldb.so.3.5 => not found (0)
librt.so.1 => /usr/lib/librt.so.1 (0x80082f000)
libtinfo.so.5.9 => /usr/local/lib/libtinfo.so.5.9 (0x800a34000)
<and system and Python-related libraries>Is the rpath set correctly?
It looks like it:
feynman% readelf -d bin/lldb | grep PATH
0x000000000000000f (RPATH) Library rpath: [$ORIGIN/../lib]
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib]Does it work with the configure build?
The buildbot uses the configure build and is green, so it "works" I
guess -- but I do not know if it's using $ORIGIN or an absolute rpath,
and will have to wait until I can complete a configure build locally
to find out.
That is so strange. The Makefile.rules used by the configure build has
LD.Flags += $(RPATH) -Wl,'$$ORIGIN/../lib
so I would expect it to behave the same as the current cmake build.
Cheers,
Rafael
This at least has a straightforward explanation -- for the configure
build my binary ends up with an rpath containing both the
$ORIGIN-based relative path as well as an absolute path:
feynman% readelf -d ./Debug+Asserts/bin/lldb | grep PATH
0x000000000000000f (RPATH) Library rpath:
[$ORIGIN/../lib:/tank/emaste/src/llvm/build-make/Debug+Asserts/lib]
0x000000000000001d (RUNPATH) Library runpath:
[$ORIGIN/../lib:/tank/emaste/src/llvm/build-make/Debug+Asserts/lib]
OK. We should probably get the same to happen with cmake. What is the
command line used to link ./Debug+Asserts/bin/lldb? I assume the
relative path is being added by llvm's Makefile.rules. Is the absolute
one being added by lldb's?
Brand, is there a way to add to rpath from an inner CMakeLists.txt?
Cheers,
Rafael
The CMAKE_INSTALL_RPATH variable is not actually a global setting
but is rather used to initialize the INSTALL_RPATH target property:
on each library or executable target as it is created. One may
append to an individual target's INSTALL_RPATH property or may
set CMAKE_INSTALL_RPATH in the current directory scope to influence
targets in the directory and below.
IIUC the reason LLVM can use BUILD_WITH_INSTALL_RPATH is because
it is careful to lay out the build tree binaries the same way as
they appear in the install tree so $ORIGIN/../lib works in both
places. If LLDB is building externally then its binaries will not
appear in the right relative location to use that RPATH. It could
consider using INSTALL_RPATH_USE_LINK_PATH:
so that LLDB binaries are built and installed with the RPATH pointing
at locations used at link time to search for external libraries.
Alternatively it could not use BUILD_WITH_INSTALL_RPATH and instead
allow CMake to handle the build-tree RPATH as it likely did before
this change.
-Brad
LLVM r201936 restores the use of a relative rpath but this breaks LLDB
on FreeBSD. For now I've disabled CMAKE_BUILD_WITH_INSTALL_RPATH
locally until the fix can be found, in the top-level LLVM
CMakeLists.txt:
I found out what the problem is -- the -zorigin flag isn't being
passed to the linker, so the ORIGIN flag isn't set. I guess either
the GNU runtime linker doesn't check, or a later version of GNU ld
sets the flag automatically; either way, FreeBSD needs -zorigin. I
applied this patch:
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 2ce214a..86bb4c5 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -445,6 +445,7 @@ if (APPLE)
else(UNIX)
if(NOT DEFINED CMAKE_INSTALL_RPATH)
set(CMAKE_INSTALL_RPATH "\$ORIGIN/../lib")
+ set(CMAKE_EXE_LINKER_FLAGS "-Wl,-zorigin")
endif(NOT DEFINED CMAKE_INSTALL_RPATH)
endif()
and LLDB works as the binary has the flag set:
%feynman% readelf -d bin/lldb | egrep 'PATH|FLAGS'
0x000000000000000f (RPATH) Library rpath: [$ORIGIN/../lib]
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib]
0x000000000000001e (FLAGS) ORIGIN
0x000000006ffffffb (FLAGS_1) Flags: ORIGIN
I found out what the problem is -- the -zorigin flag isn't being
passed to the linker, so the ORIGIN flag isn't set. I guess either
the GNU runtime linker doesn't check, or a later version of GNU ld
sets the flag automatically; either way, FreeBSD needs -zorigin. I
applied this patch:
Interesting. Where is that coming from in the configure build? I don't
see it anywhere in llvm or lldb.
Cheers,
Rafael
It isn't being set in the configure build. With the configure build
rtld first fails on the $ORIGIN due to the missing flag, but still
finds the library via the the "backup" absolute path. It will still
need an equivalent change.
It isn't being set in the configure build. With the configure build
rtld first fails on the $ORIGIN due to the missing flag, but still
finds the library via the the "backup" absolute path. It will still
need an equivalent change.
Ah, OK.
So, this is not lldb specific (the -DBUILD_SHARED_LIBS=ON build is
probably broken on freebsd right now). It seems to be freebsd specific
(i don't see zorigin anywhere in binutils). Maybe what we should do is
apply the attached patch to llvm?
Cheers,
Rafael
t.patch (505 Bytes)
It's not FreeBSD-specific; here it is in the GNU ld manual:
-z keyword
`origin' Marks the object may contain $ORIGIN.
and it turns up in some Solaris references from the early 2000s as well.
It's not FreeBSD-specific; here it is in the GNU ld manual:
Options (LD)-z keyword
`origin' Marks the object may contain $ORIGIN.and it turns up in some Solaris references from the early 2000s as well.
Also w.r.t. the patch:
+ if( ${CMAKE_SYSTEM_NAME} MATCHES FreeBSD )
+ set(CMAKE_EXE_LINKER_FLAGS "-Wl,-zorigin")
+ endif()
CMAKE_EXE_LINKER_FLAGS is a user-settable value in the local cache
so instead of setting it outright you should prepend or append:
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,-zorigin")
-Brad
It's not FreeBSD-specific; here it is in the GNU ld manual:
Options (LD)-z keyword
`origin' Marks the object may contain $ORIGIN.and it turns up in some Solaris references from the early 2000s as well.
Oh, sorry. I was looking for it as a single option.
I have tested that the attached patch to llvm works on linux.
Cheers,
Rafael
t.patch (446 Bytes)
CMAKE_EXE_LINKER_FLAGS is a user-settable value in the local cache
so instead of setting it outright you should prepend or append:set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,-zorigin")
Ah, OK.
The attached patch should fix both issues.
Cheers,
Rafael
t.patch (472 Bytes)
Perhaps -zorigin should be used only when we are using $ORIGIN?
if(NOT DEFINED CMAKE_INSTALL_RPATH)
set(CMAKE_INSTALL_RPATH "\$ORIGIN/../lib")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,-zorigin")
endif()
If a user sets CMAKE_INSTALL_RPATH to include $ORIGIN then he/she
can also add -zorigin as necessary.
-Brad
True. My thinking was that there is probably no reason not to have it,
but the fact that it is not implicit in freebsd probably means there
is a reason for not having it sometimes.
Cheers,
Rafael
t.patch (426 Bytes)
Oh, that's likely my mistake. I suspect it's properly set as "-z
origin" or "-Wl,-z -Wl,origin".
How about the attached patch?
cmake_zorigin.patch (379 Bytes)
Perhaps "-Wl,-z,origin"?
-Brad
Aha, I think I've finally found a reference that makes sense of
DF_ORIGIN, in this decade-old SCO documentation:
http://www.sco.com/developers/gabi/2003-12-17/ch5.dynamic.html
Based on this I think that FreeBSD's rtld is actually in the wrong
here, and an executable's rpath that contains $ORIGIN should undergo
substitution, regardless of the state of the DF_ORIGIN flag. If I
understand correctly DF_ORIGIN should be necessary only in the case of
an executable that does not use $ORIGIN in an rpath, but later
dlopen()s a library that does.
That said, it still seems safe to just set the flag always.