[RFC] Prefer libraries installed next to Clang over those from GCC

Hi,

Background: I'm on CentOS 7 which has GCC 4.8 installed in /usr/bin and I'm
installing Clang to, let's say, /opt/clang. (I'm doing so with
CLANG_DEFAULT_CXX_STDLIB=libc++, just if you wonder why the following
matters to me...)
With the current behavior, -lunwind passed to Clang will pull in the system
default non-GNU /usr/lib/libunwind.so.8. IMO it should take LLVM's version
in /opt/clang/lib/libunwind.so.1

That's why I would like to change the default order for searching libraries
in Clang. Currently (and most probably due to historical reasons), the
Driver adds paths next to GCC first.
I propose to prefer paths next to the installed Clang. You can find the
patch doing the change here: https://reviews.llvm.org/D26244

However, this has a potential impact as Chandler pointed out in above patch
if Clang is installed to /opt/software together with other software.
With my proposed change, Clang will link against
/opt/software/lib/libunrelated.so when passed -lunrelated whereas before it
had preferred /usr/lib/libunrelated.so (assuming that the library exists in
both places).
Is that considered an important scenario or can we expect our users to
install Clang separately?

Any other feedback on that topic?
Jonas

I think this is an important usage scenario, but because of the complication Chandler mentioned, I don’t think we can enable it by default. I think it is very reasonable to have some CMake flag that enables this behavior. -Hal

Unless we’re also passing -rpath or similar, then this may also lead to confusion where you link against the /opt/clang/lib/libunwind.so, but end up using /usr/lib/libunwind.so. In this particular example, you’re relying on the fact that the two libunwinds have different SO versions, but that’s by no means guaranteed.

Libunwind is a very bad motivation for this in general because:

- It’s an implementation of a standard ABI
- The LLVM and non-GNU versions (and the GNU fork of the non-GNU version) are intended to be compatible
- Linking both will cause things to go very badly wrong at run time

Unless you can ensure that everything is compiled either purely with clang or purely with gcc, and that you never compile something with one that uses libraries compiled by the other, then this is going to cause far more problems than it solves.

David

Hi,

Background: I'm on CentOS 7 which has GCC 4.8 installed in /usr/bin and I'm
installing Clang to, let's say, /opt/clang. (I'm doing so with
CLANG_DEFAULT_CXX_STDLIB=libc++, just if you wonder why the following
matters to me...)
With the current behavior, -lunwind passed to Clang will pull in the system
default non-GNU /usr/lib/libunwind.so.8. IMO it should take LLVM's version
in /opt/clang/lib/libunwind.so.1

That's why I would like to change the default order for searching libraries
in Clang. Currently (and most probably due to historical reasons), the
Driver adds paths next to GCC first.
I propose to prefer paths next to the installed Clang. You can find the
patch doing the change here:
https://reviews.llvm.org/D26244

However, this has a potential impact as Chandler pointed out in above patch
if Clang is installed to /opt/software together with other software.
With my proposed change, Clang will link against
/opt/software/lib/libunrelated.so when passed -lunrelated whereas before it
had preferred /usr/lib/libunrelated.so (assuming that the library exists in
both places).
Is that considered an important scenario or can we expect our users to
install Clang separately?

Any other feedback on that topic?

I think this is an important usage scenario, but because of the complication Chandler mentioned, I don't think we can enable it by default. I think it is very reasonable to have some CMake flag that enables this behavior.

Unless we’re also passing -rpath or similar, then this may also lead to confusion where you link against the /opt/clang/lib/libunwind.so, but end up using /usr/lib/libunwind.so. In this particular example, you’re relying on the fact that the two libunwinds have different SO versions, but that’s by no means guaranteed.

Libunwind is a very bad motivation for this in general because:

- It’s an implementation of a standard ABI
- The LLVM and non-GNU versions (and the GNU fork of the non-GNU version) are intended to be compatible
- Linking both will cause things to go very badly wrong at run time

Unless you can ensure that everything is compiled either purely with clang or purely with gcc, and that you never compile something with one that uses libraries compiled by the other, then this is going to cause far more problems than it solves.

Good point. Actually, perhaps this is just another motivating case for https://reviews.llvm.org/D24933 (the Clang config files).

  -Hal

[Note: I think my previous reply didn't went when mail program crashed,
sorry if you get it twice]

Background: I'm on CentOS 7 which has GCC 4.8 installed in /usr/bin and I'm
installing Clang to, let's say, /opt/clang. (I'm doing so with
CLANG_DEFAULT_CXX_STDLIB=libc++, just if you wonder why the following
matters to me...)
With the current behavior, -lunwind passed to Clang will pull in the system
default non-GNU /usr/lib/libunwind.so.8. IMO it should take LLVM's version
in /opt/clang/lib/libunwind.so.1

That's why I would like to change the default order for searching libraries
in Clang. Currently (and most probably due to historical reasons), the
Driver adds paths next to GCC first.
I propose to prefer paths next to the installed Clang. You can find the
patch doing the change here: https://reviews.llvm.org/D26244

However, this has a potential impact as Chandler pointed out in above patch
if Clang is installed to /opt/software together with other software.
With my proposed change, Clang will link against
/opt/software/lib/libunrelated.so when passed -lunrelated whereas before it
had preferred /usr/lib/libunrelated.so (assuming that the library exists in
both places).
Is that considered an important scenario or can we expect our users to
install Clang separately?

Any other feedback on that topic?

I think the best way forward is to create a separate runtime libraries
directory for clang (similarly to how gcc has one). For example,
the x86 targets could use:

  /usr/lib/clang/5.0.0/lib/linux/i386
  /usr/lib/clang/5.0.0/lib/linux/x86_64

i.e. the same directory as compiler-rt uses right now + arch
subdirectory matching suffixes used by compiler-rt libraries.

Those (dedicated) directories could safely be prepended to library
search path without risking accidentally pulling in unintended
libraries. You could install LLVM libunwind there without risking it
getting mixed with system libunwind.

We already use a similar solution for clang-suite. If you'd like to
take a look at our patches, they're at [1,2]. Sadly, they're ugly
and I didn't find time so far to clean them up and submit.

[1]:https://github.com/pathscale/clang-suite/blob/master/patches/psclang/0005-Driver-Always-add-L-paths-for-clang-runtime-director.patch
[2]:https://github.com/pathscale/clang-suite/blob/master/patches/psclang/0006-Driver-Add-rpaths-for-internal-libdirs.patch

+1, let's do this! Putting the architecture in the name or our compiler
runtime libraries creates needless build system complexity.

From: Dr D. Chisnall [mailto:dc552@hermes.cam.ac.uk] On Behalf Of David
Chisnall
Sent: Monday, January 30, 2017 6:04 PM
To: Hal Finkel
Cc: Hahnfeld, Jonas; cfe-dev@lists.llvm.org
Subject: Re: [cfe-dev] [RFC] Prefer libraries installed next to Clang over those
from GCC

> I think this is an important usage scenario, but because of the complication
Chandler mentioned, I don't think we can enable it by default. I think it is
very reasonable to have some CMake flag that enables this behavior.

Unless we’re also passing -rpath or similar, then this may also lead to
confusion where you link against the /opt/clang/lib/libunwind.so, but end up
using /usr/lib/libunwind.so. In this particular example, you’re relying on the
fact that the two libunwinds have different SO versions, but that’s by no
means guaranteed.

On the systems I work with, I can ensure that LD_LIBRARY_PATH is set correctly. This will result in the correct libraries to be linked.

Libunwind is a very bad motivation for this in general because:

- It’s an implementation of a standard ABI
- The LLVM and non-GNU versions (and the GNU fork of the non-GNU
version) are intended to be compatible

Should we then teach libcxx and libcxxabi to link against an already installed libunwind? Currently, there's only LIBCXXABI_USE_LLVM_UNWINDER which requires LLVM's libunwind to be built...

- Linking both will cause things to go very badly wrong at run time

That's probably the point where I got it wrong last time: I had quite a lot of test failures...

Hi Hal,

Thanks for the idea: I will adapt the patch to introduce a new CMake flag and to not change the default behavior.

Thanks,

Jonas

From: Dr D. Chisnall [mailto:dc552@hermes.cam.ac.uk] On Behalf Of David
Chisnall
Sent: Monday, January 30, 2017 6:04 PM
To: Hal Finkel
Cc: Hahnfeld, Jonas; cfe-dev@lists.llvm.org
Subject: Re: [cfe-dev] [RFC] Prefer libraries installed next to Clang over those
from GCC

I think this is an important usage scenario, but because of the complication

Chandler mentioned, I don't think we can enable it by default. I think it is
very reasonable to have some CMake flag that enables this behavior.

Unless we’re also passing -rpath or similar, then this may also lead to
confusion where you link against the /opt/clang/lib/libunwind.so, but end up
using /usr/lib/libunwind.so. In this particular example, you’re relying on the
fact that the two libunwinds have different SO versions, but that’s by no
means guaranteed.

On the systems I work with, I can ensure that LD_LIBRARY_PATH is set correctly. This will result in the correct libraries to be linked.

Which is fine, if you’re (a) building everything with clang (no gcc-built shared libraries) and (b) able to [remember to] put that in your environment whenever you run a clang-built thing.

Libunwind is a very bad motivation for this in general because:

- It’s an implementation of a standard ABI
- The LLVM and non-GNU versions (and the GNU fork of the non-GNU
version) are intended to be compatible

Should we then teach libcxx and libcxxabi to link against an already installed libunwind? Currently, there's only LIBCXXABI_USE_LLVM_UNWINDER which requires LLVM's libunwind to be built…

Doesn’t it do this automatically if you don’t specify to use the LLVM unwinder? On FreeBSD, we use the unwinder from libgcc_s / libgcc_eh (which actually is the LLVM unwinder on head, but is called libgcc_* for compatibility) and libcxxrt as the C++ runtime library and all of this seems to just work.

- Linking both will cause things to go very badly wrong at run time

That's probably the point where I got it wrong last time: I had quite a lot of test failures...

Yes, and making it easier to get into this situation doesn’t sound like a good plan. The unwinder and C++ ABI library are both low-level system components and just ripping them out and replacing them needs almost as much thought as replacing libc. Making it a thing that end-users might to by mistake makes me very nervous.

David