How can I make clang/llvm use specific toolchain?

Hello list.

I wonder how I can make clang use specific linker.

I am now trying to compile a hello world c code on an Ubuntu box for
ARM and X86.
And I have successfully built for ARM but NOT ABLE TO build one for X86.

I am using a cross toolchain in the scratchbox environment of Tizen SDK.

You can refer to the url below for my testing environment.
http://blog.mikeasoft.com/2012/06/23/writing-native-tizen-applications-using-the-1-0-larkspur-sdk-scratchbox-and-efl/

Journeyer J. Joh wrote:

I wonder how I can make clang use specific linker.

I am now trying to compile a hello world c code on an Ubuntu box for
ARM and X86.
And I have successfully built for ARM but NOT ABLE TO build one for X86.

I am using a cross toolchain in the scratchbox environment of Tizen SDK.

<...>

The problem above is that clang tries to call "ld" but there is not a
effective path to it.
I now need to make clang use a specific "ld" in the cross toolchain of
the scratchbox.

Recently, I've started using scratchbox2 [1] for ARM cross-compilation and
met the same problem in invoking clang under scratchbox2: executing of
linker by clang wasn't redirected by scratchbox2 to cross-linker. Instead,
host system linker (/usr/bin/ld) was invoked. Also host assembler was
invoked when -integrated-as wasn't used.

I've found source of problem and a workaround. Hope it will help somebody
else.

In short:
* clang tries to find "<triple>-ld" or "ld" executable (as fallback) in a
  bunch of directories. <triple> could be (re-)defined by -target option.
* found executable is invoked by posix_spawn function (on systems, where
  it's supported) or fork/execve (as fallback). posix_spawn support is
  determined in process of LLVM compilation. posix_spawn is supported on
  many systems, Linux including.
* posix_spawn from glibc internally calls fork/__execve, but __execve isn't
  known as intercepting target to scratchbox2, so scratchbox2 couldn't
  change __execve arguments to point to cross-linker.
* and host linker is invoked.

Workaround: I was lucky enough to have cross-linker installed into
/usr/bin/arm-linux-gnueabi-ld. So I've passed following options to clang:

  -target arm-linux-gnueabi -march=armv7-a -mthumb -mtune=cortex-a8

As -target is defined, clang will try to find arm-linux-gnueabi-ld first and
it will be found. -march, -mthumb and -mtune options are overriding defaults
set after -target value.

And 'sb2 clang ...' with these options invokes cross-linker.

Used programs:
* clang 3.2 release from Debian Experimental (package clang 3.2-1~exp1)
* ARM cross-compiling gcc 4.7.2 from emdebian.net (package
  gcc-4.7-arm-linux-gnueabi 4.7.2-4)
* ARM cross-compiling binutils 2.22 from emdebian.net (package
  binutils-arm-linux-gnueabi 2.22-7.1)
* scratchbox2 from git (version 2.3.90, 2.3_development branch,
  commit 6439d12)

Just for reference, here is program flow for linker invoking:

clang: make ld invoking command

Hello Leonid Borisenko,

Workaround: I was lucky enough to have cross-linker installed into
/usr/bin/arm-linux-gnueabi-ld. So I've passed following options to clang:

  -target arm-linux-gnueabi -march=armv7-a -mthumb -mtune=cortex-a8

As -target is defined, clang will try to find arm-linux-gnueabi-ld first and
it will be found. -march, -mthumb and -mtune options are overriding defaults
set after -target value.

And 'sb2 clang ...' with these options invokes cross-linker.

- This means that you used cross-binutil to link ELFs. But (the
library and?) header files are of SB2?
- Did you execute clang under SB2 environment(context)? and used
cross-binutil out of SB2?

Thank you for your sharing of your knowledge.
The usage of the options above would be helpful.

Thank you very much!

Sincerely
Journeyer J. Joh

Journeyer J. Joh wrote:

Workaround: I was lucky enough to have cross-linker installed into
/usr/bin/arm-linux-gnueabi-ld. So I've passed following options to clang:

  -target arm-linux-gnueabi -march=armv7-a -mthumb -mtune=cortex-a8

As -target is defined, clang will try to find arm-linux-gnueabi-ld first and
it will be found. -march, -mthumb and -mtune options are overriding defaults
set after -target value.

And 'sb2 clang ...' with these options invokes cross-linker.

- This means that you used cross-binutil to link ELFs. But (the
library and?) header files are of SB2?
- Did you execute clang under SB2 environment(context)? and used
cross-binutil out of SB2?

Thank you for your sharing of your knowledge.
The usage of the options above would be helpful.

First of all, I'd like to note that I've installed scratchbox2 separately of
cross-toolchain and target rootfs and made sb2 target by myself with
'sb2-init ...'. So, it's not clear, whether I can say that some of my headers
and tools are "outside of sb2" or "inside of sb2". Certainly, my cross-tools
(gcc, as, ld etc.) are "outside of sb2" as they aren't belonged to
scratchbox2 installation or to target rootfs. On the other hand, they are
accessible from sb2 command context/shell, as scratchox2 automatically mapped
some paths (including paths to cross-tools) to real, not target rootfs. So
my cross-tools are "inside of s2". The same could be said about standard
headers provided by cross-toolchain.

Answering to your first question: yes, clang/clang++ uses the same
headers/libraries as gcc would use when invoked by 'sb2 gcc ...' (i.e.
headers/libraries "of sb2"), but by various reasons it's not handled
automatically by scratchbox2, so it should be set manually.

As a prerequisite, I had to pass to clang location of standard
cross-toolchain headers. By default clang uses headers from toolchain used
in it's own compilation process. As my clang was built for x86 architecture,
it uses host toolchain headers, not cross-toolchain headers. It could be
fixed with -isystem and -cxx-isystem options. Locations of cross-toolchain
standard headers could be found with the following commands:

  sb2 sh -c "echo| gcc -xc -E -v -" 2>&1 | awk -e '/<...>/,/^End/ {print}'
  sb2 sh -c "echo| gcc -xc++ -E -v -" 2>&1 | awk -e '/<...>/,/^End/ {print}'

First command shows paths to standard headers used by cross-toolchain in
compilation of C programs. Second command shows paths to standard headers
used by cross-toolchain in compilation of C++ programs.

I've used 'sb2 ...' just for convenience. Instead, full path to
cross-toolchain's gcc could be used, like:

  echo | /opt/cross/bin/gcc -xc -E -v -" 2>&1 | awk -e '/<...>/,/^End/ {print}

On my machine, first command outputs:
#include <...> search starts here:
/usr/local/include
/usr/include
/usr/lib/gcc/arm-linux-gnueabi/4.7/include
/usr/lib/gcc/arm-linux-gnueabi/4.7/include-fixed
/usr/arm-linux-gnueabi/include
End of search list.

And second command outputs:
#include <...> search starts here:
/usr/local/include
/usr/include
/usr/arm-linux-gnueabi/include/c++/4.7.2
/usr/arm-linux-gnueabi/include/c++/4.7.2/arm-linux-gnueabi
/usr/arm-linux-gnueabi/include/c++/4.7.2/backward
/usr/lib/gcc/arm-linux-gnueabi/4.7/include
/usr/lib/gcc/arm-linux-gnueabi/4.7/include-fixed
/usr/arm-linux-gnueabi/include
End of search list.

So, required options for clang are:

  -isystem /sb2/target/rootfs/usr/local/include
  -isystem /sb2/target/rootfs/usr/include
  -isystem /usr/lib/gcc/arm-linux-gnueabi/4.7/include
  -isystem /usr/lib/gcc/arm-linux-gnueabi/4.7/include-fixed
  -isystem /usr/arm-linux-gnueabi/include
  -cxx-isystem /usr/arm-linux-gnueabi/include/c++/4.7.2
  -cxx-isystem /usr/arm-linux-gnueabi/include/c++/4.7.2/arm-linux-gnueabi
  -cxx-isystem /usr/arm-linux-gnueabi/include/c++/4.7.2/backward

Paths from output of first command should be values of -isystem. Paths
from output of second command, that aren't already appeared in output of
first command, should be values of -cxx-isystem. I've also prepended
path to scratchbox2 target rootfs to /usr/local/include and /usr/include
paths, as mapping of these paths couldn't be handled by scratchbox2 because
clang driver calls clang compiler with posix_spawn (on my machine), so
compiler "escapes" from scratchbox2 monitoring.

To use libraries in target rootfs, absolute paths to libraries locations
should be passed to clang in following options:

  -L /sb2/target/rootfs/usr/lib
  -L /sb2/target/rootfs/lib
  -Wl,-rpath-link,/sb2/target/rootfs/usr/lib
  -Wl,-rpath-link,/sb2/target/rootfs/lib

-L and -rpath-link are described in 'man ld'. sb2-init automatically sets
applying of these options for each invocation of 'sb2 gcc ...', but for
clang these options should be applied manually.

Answering your second question: yes, I did execute clang under scratchbox2
environment (context) and used cross-binutil out of scratchbox2.

Answering your question about usage of -target and following options:
-target is a renamed -ccc-host-triple [1]. It's used for cross-compiling.
My clang had been built on x86 machine and, by default, produces code for
x86. As I want to cross-compile, I must provide clang with target
architecture by using -target option. You can read about target format in
[2] [3].

Other options (-march etc.) are target-specific options (in my case --
ARM-specific [4]).

[1] http://lists.cs.uiuc.edu/pipermail/cfe-dev/2012-January/019442.html
[2] http://www.sourceware.org/autobook/autobook/autobook_17.html
[3] http://airs.com/ian/configure/configure_4.html
[4] http://gcc.gnu.org/onlinedocs/gcc/ARM-Options.html

Hello Leonid Borisenko,

So, it's not clear, whether I can say that some of my headers and tools are "outside of sb2" or "inside of sb2".

If this is cleared, I believe your approach is very good workaround!
I believe header files and library and runtime must be of the same
toolchain. if it is mixed from several toolchain, the build system can
not be trust worthy.

I do cross compile using a toolchain in the tizen-sdk, which use
chroot environment, similar to the SB environment. The options needed
are

-ccc-host-triple $(CCC_HOST_TRIPLE_ARM) \
--sysroot=$(SYSROOT_ARM) \
-gcc-toolchain $(GCC_TOOLCHAIN)
LDFLAGS = -Wl,-dynamic-linker,/lib/ld-linux.so.3

In my understanding to use clang under SB2 environment, a porting is
needed because clang only works in known environment only. With the
knowledge of the OS clang finds header files and library and runtime.
But your workaround removes the necessity of porting!!

And the third question of mine was actually not a question. But your
answer is so much informative!!

I learned about the configuration triplet. I asked a question about
this on this list earlier but didn't receive any reply yet.

I really appreciate you for your fully informative answers.
I even feel sorry for my questions. __;

Thank you very much.

Sincerely
Journeyer