--sysroot and --gcc-toolchain: any docs etc.?

Can someone point to any detailed information about how these flags
work? I have a custom-compiled version of GCC on Linux and I'm trying
to build Clang 4.0.0 for this system as well. I would like it to use
libstdc++ from my custom-compiled GCC installation to allow linking of
other libraries built with libstdc++.

I also have a sysroot that I use with GCC so it compiles against a known
set of system headers and libraries (based on Red Hat 6.3 in this case).
This sysroot is directly extracted from a Red Hat system (technically,
it's extracted from the RPMs) so the layout is straightforward and using
the --sysroot for GCC locates it without any problems.

However, I can't get clang to work with either of these environments no
matter what options I provide it.

For --sysroot for example, I have this:

  /my/sysroot/lib
  /my/sysroot/lib64
/my/sysroot/usr/include
/my/sysroot/usr/lib
/my/sysroot/usr/lib64
/my/sysroot/usr/libexec

If I use "gcc --sysroot=/my/sysroot" all works fine. If I use "clang
--syroot=mysysroot" no system headers are found. I used strace on the
clang binary to try to figure out what it's looking for and it searches
for all of these things in my sysroot directory:

  /my/sysroot/usr/local/cuda
  /my/sysroot/usr/local/cuda-8.0
  /my/sysroot/usr/local/cuda-7.5
  /my/sysroot/usr/local/cuda-7.0
  /my/sysroot/lib/x86_64-linux-gnu
  /my/sysroot/lib/x86_64-generic-linux-gnu

/my/sysroot/lib/../lib64

  /my/sysroot/usr/lib/x86_64-generic-linux-gnu

/my/sysroot/usr/lib/../lib64

  /my/sysroot/usr/lib/x86_64-generic-linux-gnu/../../lib64

/my/sysroot/lib
/my/sysroot/usr/lib

Of these only the four marked exist in my sysroot. But the big thing
here is that at no time does clang look for /my/sysroot/usr/include, and
indeed when the preprocessor tries to find headers it doesn't look
there, and none of my system header files are found!

Do I have to add these myself with -isystem even when I specify
--sysroot and the header directory is right there? Is --sysroot not
intended to be used to locate header files?

For --gcc-toolchain the situation is similar although clang looks in
MANY more places to try to locate the GCC toolchain. However, I build
GCC myself and I have a completely standard installation: what you get
when you run make / make install after a configure --prefix. My GCC
installation looks like:

  /my/gcc/bin
  /my/gcc/lib/gcc/x86_64-generic-linux-gnu/7.1.0/include
  /my/gcc/lib64
  /my/gcc/libexec/gcc/x86_64-generic-linux-gnu/7.1.0
  /my/gcc/x86_64-generic-linux-gnu/bin
  /my/gcc/x86_64-generic-linux-gnu/include/c++/7.1.0/...
  /my/gcc/x86_64-generic-linux-gnu/include/c++/7.1.0/x86_64-generic-linux-gnu/...
  /my/gcc/x86_64-generic-linux-gnu/lib
  /my/gcc/x86_64-generic-linux-gnu/lib64

(I have a GCC 6.2 install too and the layout is basically identical.)
When I use --gcc-toolchain with clang it looks in about 50 different
directories but doesn't seem to be able to locate this: a standard
unmodified installation of GCC. I've tried numerous different values
for --gcc-toolchain but none seem to work.

Am I doing something wrong here? Or is this just not going to work and
I'll have to figure out all the include directories, etc. I need by hand
and add them via -isystem etc. myself?

Thanks for any help you can provide!

Hi,

we also had this problem and the thing it that clang checks for a
valid GCC/libstdc++ installation by looking for the crtbegin.o file in
the related lib directory. So it doesn't work if you just have the
headers for libstdc++, you also need to recreate the folder-structure
that leads to the crtbegin.o file, otherwise clang doesn't recognize
this as a valid sysroot.

There are some patches on the phabricator to fix this, but I don't
think any of them has landed yet.

Otherwise look in the /test/ directory and grep for "--sysroot" to get
a more examples.

Cheers,

- Raphael

Right at the end this comment seems to conflate the compiler install
with the sysroot, and these are not the same thing. Maybe it's just a
typo but if clang has that same confusion then maybe that's where my
problem lies: it's important to separate the sysroot (libc, system
headers, etc.) from the compiler (STL, libgcc or equivalent, etc.)
because multiple compilers can use the same sysroot, and the same
compiler can use multiple sysroots.

The crtbegin.o is part of the compiler, not part of the sysroot, so when
I specify a path for --sysroot clang should not be trying to find that
file there. From my strace logs it doesn't appear that it does look for
crtbegin.o in the sysroot so that's good. But of course it doesn't
explain why clang never tries to look for my system headers.

The GCC installation is a complete make / make install, it's not a
package installation, and so it definitely contains the entire compiler
installation, including crtbegin.o, not just the headers:

  /my/gcc/lib/gcc/x86_64-generic-linux-gnu/7.1.0/32/crtbegin.o
  /my/gcc/lib/gcc/x86_64-generic-linux-gnu/7.1.0/crtbegin.o

This is the standard location so I'm not sure what folder structure I
should be recreating...?

we also had this problem and the thing it that clang checks for a
valid GCC/libstdc++ installation by looking for the crtbegin.o file in
the related lib directory. So it doesn't work if you just have the
headers for libstdc++, you also need to recreate the folder-structure
that leads to the crtbegin.o file, otherwise clang doesn't recognize
this as a valid sysroot.

Right at the end this comment seems to conflate the compiler install
with the sysroot, and these are not the same thing. Maybe it's just a
typo but if clang has that same confusion then maybe that's where my
problem lies: it's important to separate the sysroot (libc, system
headers, etc.) from the compiler (STL, libgcc or equivalent, etc.)
because multiple compilers can use the same sysroot, and the same
compiler can use multiple sysroots.

Well, IIRC that's actually how the toolchain support in clang is
handled. You can grep for "crtbegin.o" if you want to recheck. I know
it's not what one expects from this flag, but I didn't wrote that
code, so don't shoot the messenger :slight_smile:

The crtbegin.o is part of the compiler, not part of the sysroot, so when
I specify a path for --sysroot clang should not be trying to find that
file there. From my strace logs it doesn't appear that it does look for
crtbegin.o in the sysroot so that's good. But of course it doesn't
explain why clang never tries to look for my system headers.

If you run this command it will show that it will check for crtbegin.o
in the sysroot at some point. Maybe in your scenario it's just failing
earlier at some other file; or it's maybe something completely
different, but usually my personal --sysroot failures are due to the
crtbegin.o not being there.

$ strace -f clang++ -fsyntax-only --sysroot /./ /dev/null -v 2>&1 |
grep "/./" | grep crtbegin.o

The GCC installation is a complete make / make install, it's not a
package installation, and so it definitely contains the entire compiler
installation, including crtbegin.o, not just the headers:

  /my/gcc/lib/gcc/x86_64-generic-linux-gnu/7.1.0/32/crtbegin.o
  /my/gcc/lib/gcc/x86_64-generic-linux-gnu/7.1.0/crtbegin.o

This is the standard location so I'm not sure what folder structure I
should be recreating...?

Well, I think the clang test suite is as said a better starting point
to see what clang expects from a sysroot, but for the reference, this
is how my working sysroot subtree of lib looks like:
https://pastebin.com/raw/UZ5GVFUe

(This tree was created by a tool, so I don't know the reason for
exactly those dirs/files being there).

- Raphael