Cross compile against sysroot of older linux distribution

Clang Frontend
1

Hi,

My (c++) linux application is distributed in binary form. In order to support older linux distros, I’m currently building on an Ubuntu 16 machine. This works, the application gets linked to the older glibc version and the binary is portable.

But I’m wondering if it’s also possible to build on a recent linux distro instead, and cross-compile against the sysroot of ubuntu 16. I can’t get this to work. I’ve installed the sysroot using debootstrap. Cmake fails to create a simple executable, complaining about missing _dl_find_object@GLIBC_2.35. Indeed, GLIBC_2.35 is my hosts’s version, while the ubuntu 16 sysroot has only GLIBC_2.23 (i think).

In other words, even though I’ve set it to use a sysroot, it looks like clang is still using something that’s linked to the host’s glibc.

Below is the cmake log, which shows the clang invocations and the full error.

Is my idea possible at all? Any pointers appreciated!

Thanks
Joost

clang: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
 (in-process)
 "/usr/lib/llvm-15/bin/clang" \
 -cc1 \
 -triple \
 x86_64-pc-linux-gnu \
 -emit-obj \
 -mrelax-all \
 --mrelax-relocations \
 -disable-free \
 -clear-ast-before-backend \
 -disable-llvm-verifier \
 -discard-value-names \
 -main-file-name \
 CMakeCCompilerABI.c \
 -mrelocation-model \
 pic \
 -pic-level \
 2 \
 -pic-is-pie \
 -mframe-pointer=all \
 -fmath-errno \
 -ffp-contract=on \
 -fno-rounding-math \
 -mconstructor-aliases \
 -funwind-tables=2 \
 -target-cpu \
 x86-64 \
 -tune-cpu \
 generic \
 -mllvm \
 -treat-scalable-fixed-error-as-warning \
 -debugger-tuning=gdb \
 -v \
 -fcoverage-compilation-dir=/tmp/5bLjjbaM8f/CMakeFiles/CMakeScratch/TryCompile-1MCqGI \
 -resource-dir \
 /usr/lib/llvm-15/lib/clang/15.0.7 \
 -dependency-file \
 CMakeFiles/cmTC_476ca.dir/CMakeCCompilerABI.c.o.d \
 -MT \
 CMakeFiles/cmTC_476ca.dir/CMakeCCompilerABI.c.o \
 -sys-header-deps \
 -isysroot \
 /home/joostn/toolchains/linux/x86_64/sysroot \
 -internal-isystem \
 /usr/lib/llvm-15/lib/clang/15.0.7/include \
 -internal-isystem \
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/local/include \
 -internal-isystem \
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1/../../../../x86_64-linux-gnu/include \
 -internal-externc-isystem \
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu \
 -internal-externc-isystem \
 /home/joostn/toolchains/linux/x86_64/sysroot/include \
 -internal-externc-isystem \
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/include \
 -fdebug-compilation-dir=/tmp/5bLjjbaM8f/CMakeFiles/CMakeScratch/TryCompile-1MCqGI \
 -ferror-limit \
 19 \
 -fgnuc-version=4.2.1 \
 -faddrsig \
 -D__GCC_HAVE_DWARF2_CFI_ASM=1 \
 -o \
 CMakeFiles/cmTC_476ca.dir/CMakeCCompilerABI.c.o \
 -x \
 c \
 /usr/share/cmake-3.25/Modules/CMakeCCompilerABI.c
clang -cc1 version 15.0.7 based upon LLVM 15.0.7 default target x86_64-pc-linux-gnu
ignoring nonexistent directory "/home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/home/joostn/toolchains/linux/x86_64/sysroot/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/llvm-15/lib/clang/15.0.7/include
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/local/include
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/include
End of search list.
[2/2] Linking C executable cmTC_476ca
FAILED: cmTC_476ca 
: && /usr/bin/clang --sysroot=/home/joostn/toolchains/linux/x86_64/sysroot -fuse-ld=lld --sysroot=/home/joostn/toolchains/linux/x86_64/sysroot -v  -rdynamic CMakeFiles/cmTC_476ca.dir/CMakeCCompilerABI.c.o -o cmTC_476ca   && :
Ubuntu clang version 15.0.7
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5
Found candidate GCC installation: /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1
Found candidate GCC installation: /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/6
Found candidate GCC installation: /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/6.0.0
Selected GCC installation: /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1
Candidate multilib: .;@m64
Selected multilib: .;@m64
 "/usr/bin/ld.lld" \
 --sysroot=/home/joostn/toolchains/linux/x86_64/sysroot \
 -pie \
 -export-dynamic \
 --hash-style=both \
 --build-id \
 --eh-frame-hdr \
 -m \
 elf_x86_64 \
 -export-dynamic \
 -dynamic-linker \
 /lib64/ld-linux-x86-64.so.2 \
 -o \
 cmTC_476ca \
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/x86_64-linux-gnu/Scrt1.o \
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/x86_64-linux-gnu/crti.o \
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1/crtbeginS.o \
 -L/home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1 \
 -L/home/joostn/toolchains/linux/x86_64/sysroot/lib/x86_64-linux-gnu \
 -L/home/joostn/toolchains/linux/x86_64/sysroot/lib/../lib64 \
 -L/home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/x86_64-linux-gnu \
 -L/home/joostn/toolchains/linux/x86_64/sysroot/lib \
 -L/home/joostn/toolchains/linux/x86_64/sysroot/usr/lib \
 CMakeFiles/cmTC_476ca.dir/CMakeCCompilerABI.c.o \
 -lgcc \
 --as-needed \
 -lgcc_s \
 --no-as-needed \
 -lc \
 -lgcc \
 --as-needed \
 -lgcc_s \
 --no-as-needed \
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1/crtendS.o \
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/x86_64-linux-gnu/crtn.o


ld.lld: 
error: undefined reference due to --no-allow-shlib-undefined: _dl_find_object@GLIBC_2.35
>>> referenced by /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1/libgcc_s.so
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
2

try add -ldl to the link line.

3

Thanks. Adding -ldl makes no difference unfortunately.

I’ve narrowed it down some more:

#!/bin/bash

set -e
set -x

cat << 'EOT' > /tmp/hello.c;
#include <stdio.h>
int main() {
    printf("GNU libc compile-time version: %u.%u\n", 
            __GLIBC__, __GLIBC_MINOR__);
   return 0;
}
EOT

rm -f /tmp/hello.o
rm -f /tmp/hello

"/usr/lib/llvm-15/bin/clang" \
  --sysroot=/home/joostn/toolchains/linux/x86_64/sysroot \
  -isysroot /home/joostn/toolchains/linux/x86_64/sysroot \
 -c \
  -o  /tmp/hello.o \
/tmp/hello.c

"/usr/lib/llvm-15/bin/clang" \
  -fuse-ld=lld \
  -ldl \
    -o /tmp/hello \
/tmp/hello.c

/tmp/hello

This succesfully links (because I’m not using the sysroot in the linker invocation).

This prints: GNU libc compile-time version: 2.37, which is the host version, not the sysroot version.

The sysroot has 2.23:

cat /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/features.h | grep _GLIBC
#define	__GLIBC__	2
#define	__GLIBC_MINOR__	23

Host has 2.37:

cat /usr/include/features.h | grep _GLIBC
#define	__GLIBC__	2
#define	__GLIBC_MINOR__	37

So apparently the compile command still uses includes from the host, despite the sysroot options. Clearly I’m missing something.

4

You can add -v to the Clang invocation to see exactly which directories it’s searching for headers in, which might shed some light upon what could be going wrong. I doubt it’s related, but I’d also remove the -isysroot; that’s more of an Apple thing, whereas Linux just uses --sysroot for everything.

5

Thanks. I’ve added -v -H, and this shows it’s using /usr/lib/llvm-15/lib/clang/15.0.7/include !

However, if I add -nostdinc and then add some paths manually:

"/usr/lib/llvm-15/bin/clang" \
 --sysroot=/home/joostn/toolchains/linux/x86_64/sysroot \
 -c \
 -nostdinc \
 -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/include \
 -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu \
 -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5/include \
 -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/linux \
  -o  /tmp/hello.o \
  -v -H \
/tmp/hello.c

I get this:

Ubuntu clang version 15.0.7
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-15/bin
Found candidate GCC installation: /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5
Found candidate GCC installation: /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1
Found candidate GCC installation: /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/6
Found candidate GCC installation: /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/6.0.0
Selected GCC installation: /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1
Candidate multilib: .;@m64
Selected multilib: .;@m64
 (in-process)
 "/usr/lib/llvm-15/bin/clang" -cc1 -triple x86_64-pc-linux-gnu -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name hello.c -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -v -H -sys-header-deps -fcoverage-compilation-dir=/tmp -nostdsysteminc -nobuiltininc -resource-dir /usr/lib/llvm-15/lib/clang/15.0.7 -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/include -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5/include -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/linux -isysroot /home/joostn/toolchains/linux/x86_64/sysroot -fdebug-compilation-dir=/tmp -ferror-limit 19 -fgnuc-version=4.2.1 -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/hello.o -x c /tmp/hello.c
clang -cc1 version 15.0.7 based upon LLVM 15.0.7 default target x86_64-pc-linux-gnu
#include "..." search starts here:
#include <...> search starts here:
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/include
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5/include
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/linux
End of search list.
. /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/stdio.h
.. /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/features.h
... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/stdc-predef.h
... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/sys/cdefs.h
.... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/bits/wordsize.h
... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/gnu/stubs.h
.... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/gnu/stubs-64.h
.. /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5/include/stddef.h
.. /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/bits/types.h
... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/bits/wordsize.h
... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/bits/typesizes.h
.. /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/libio.h
... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/_G_config.h
.... /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5/include/stddef.h
.... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/wchar.h
... /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5/include/stdarg.h
.. /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/bits/stdio_lim.h
.. /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/bits/sys_errlist.h
GNU libc compile-time version: 2.37

I see only includes from the sysroot, yet it’s still printing 2.37.

6

You’re sure you re-linked the binary? The command you posted will just build a new .o.

1 Like
7

This is normal. Clang includes a bunch of headers in its own resource directory (which is the path you’re seeing). Those headers do an #include_next to the system headers where needed, and that should pick up the sysroot you’ve specified. I wouldn’t recommend -nostdinc (which removes this include) because a bunch of things can break; you can use -nostdlibinc if you want to be explicit about your include paths instead, which will still include the resource directory but omit everything else. That being said, --sysroot should already cause the standard system paths to not be searched, and it’s much more ergonomic than specifying include paths manually.

8

Yes I did! Here’s the full script and its output:

#!/bin/bash

set -e
set -x


cat << 'EOT' > /tmp/hello.c;
#include <stdio.h>
#include <features.h>
int main() {
    printf("GNU libc compile-time version: %u.%u\n", 
            __GLIBC__, __GLIBC_MINOR__);
   return 0;
}
EOT

rm -f /tmp/hello.o
rm -f /tmp/hello

"/usr/lib/llvm-15/bin/clang" \
 --sysroot=/home/joostn/toolchains/linux/x86_64/sysroot \
 -c \
 -nostdlibinc \
 -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/include \
 -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu \
 -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5/include \
 -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/linux \
  -o  /tmp/hello.o \
  -v -H \
/tmp/hello.c

"/usr/lib/llvm-15/bin/clang" \
  -fuse-ld=lld \
  -ldl \
    -o /tmp/hello \
/tmp/hello.c

cat /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/features.h | grep "__GLIBC_"
cat /usr/include/features.h | grep "__GLIBC_"

/tmp/hello

OUtput:

+ cat
+ rm -f /tmp/hello.o
+ rm -f /tmp/hello
+ /usr/lib/llvm-15/bin/clang --sysroot=/home/joostn/toolchains/linux/x86_64/sysroot -c -nostdlibinc -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/include -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5/include -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/linux -o /tmp/hello.o -v -H /tmp/hello.c
Ubuntu clang version 15.0.7
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-15/bin
Found candidate GCC installation: /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5
Found candidate GCC installation: /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1
Found candidate GCC installation: /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/6
Found candidate GCC installation: /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/6.0.0
Selected GCC installation: /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1
Candidate multilib: .;@m64
Selected multilib: .;@m64
 (in-process)
 "/usr/lib/llvm-15/bin/clang" -cc1 -triple x86_64-pc-linux-gnu -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name hello.c -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -v -H -sys-header-deps -fcoverage-compilation-dir=/tmp -nostdsysteminc -resource-dir /usr/lib/llvm-15/lib/clang/15.0.7 -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/include -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5/include -I /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/linux -isysroot /home/joostn/toolchains/linux/x86_64/sysroot -internal-isystem /usr/lib/llvm-15/lib/clang/15.0.7/include -fdebug-compilation-dir=/tmp -ferror-limit 19 -fgnuc-version=4.2.1 -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/hello.o -x c /tmp/hello.c
clang -cc1 version 15.0.7 based upon LLVM 15.0.7 default target x86_64-pc-linux-gnu
#include "..." search starts here:
#include <...> search starts here:
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/include
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5/include
 /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/linux
 /usr/lib/llvm-15/lib/clang/15.0.7/include
End of search list.
. /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/stdio.h
.. /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/features.h
... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/stdc-predef.h
... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/sys/cdefs.h
.... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/bits/wordsize.h
... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/gnu/stubs.h
.... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/gnu/stubs-64.h
.. /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5/include/stddef.h
.. /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/bits/types.h
... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/bits/wordsize.h
... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/bits/typesizes.h
.. /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/libio.h
... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/_G_config.h
.... /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5/include/stddef.h
.... /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/wchar.h
... /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5/include/stdarg.h
.. /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/bits/stdio_lim.h
.. /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/x86_64-linux-gnu/bits/sys_errlist.h
+ /usr/lib/llvm-15/bin/clang -fuse-ld=lld -ldl -o /tmp/hello /tmp/hello.c
+ cat /home/joostn/toolchains/linux/x86_64/sysroot/usr/include/features.h
+ grep __GLIBC_
   The macros `__GNU_LIBRARY__', `__GLIBC__', and `__GLIBC_MINOR__' are
   on.  The use of this variable is deprecated.  Use __GLIBC__ and
   __GLIBC_MINOR__ now (see below) when you want to test for a specific
#define	__GLIBC__	2
#define	__GLIBC_MINOR__	23
#define __GLIBC_PREREQ(maj, min) \
	((__GLIBC__ << 16) + __GLIBC_MINOR__ >= ((maj) << 16) + (min))
+ cat /usr/include/features.h
+ grep __GLIBC_
   __GLIBC_USE (F)	Define things from feature set F.  This is defined
			__GLIBC_USE.
   The macros `__GNU_LIBRARY__', `__GLIBC__', and `__GLIBC_MINOR__' are
#undef	__GLIBC_USE_ISOC2X
#undef	__GLIBC_USE_DEPRECATED_GETS
#undef	__GLIBC_USE_DEPRECATED_SCANF
#define __GLIBC_USE(F)	__GLIBC_USE_ ## F
# define __GLIBC_USE_ISOC2X	1
# define __GLIBC_USE_ISOC2X	0
# define __GLIBC_USE_DEPRECATED_GETS 0
# define __GLIBC_USE_DEPRECATED_GETS 1
# define __GLIBC_USE_DEPRECATED_SCANF 1
# define __GLIBC_USE_DEPRECATED_SCANF 0
   on.  The use of this variable is deprecated.  Use __GLIBC__ and
   __GLIBC_MINOR__ now (see below) when you want to test for a specific
#define	__GLIBC__	2
#define	__GLIBC_MINOR__	37
#define __GLIBC_PREREQ(maj, min) \
	((__GLIBC__ << 16) + __GLIBC_MINOR__ >= ((maj) << 16) + (min))
+ /tmp/hello
GNU libc compile-time version: 2.37
9

This is not what you meant to do.

10

You’re right! But with the sysroot, lld fails:

+ /usr/lib/llvm-15/bin/clang --sysroot=/home/joostn/toolchains/linux/x86_64/sysroot -fuse-ld=lld -ldl -o /tmp/hello /tmp/hello.c
ld.lld: error: undefined reference due to --no-allow-shlib-undefined: _dl_find_object@GLIBC_2.35
>>> referenced by /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1/libgcc_s.so
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I think it does show that the problem occurs at compile time, my code seems to compile against headers from the host.

11

Although I’m wondering where the

@GLIBC_2.35

comes from. I guess this means that the linker links to an import library from the host? The compiler is not aware of the dynlib version.

But since GLIBC_MINOR is a macro, something must be wrong at compile time, right?

12

Well, that’s not linking the .o you just built, unless you’ve ditched that unnecessary step and are just using the --sysroot (which should work just fine, but up until now you have been explicitly building a .o).

From that error message though it sounds like your sysroot is inconsistent and that /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1/libgcc_s.so was built against a newer glibc than the one actually present in your sysroot.

1 Like
13

You’re right, that’s a stupid mistake…

"/usr/lib/llvm-15/bin/clang" \
 --sysroot=/home/joostn/toolchains/linux/x86_64/sysroot \
 -c \
  -o  /tmp/hello.o \
  -v -H \
  -O0 \
/tmp/hello_preprocessed.c

"/usr/lib/llvm-15/bin/clang" \
  -fuse-ld=lld \
  -ldl \
    -o /tmp/hello \
/tmp/hello.o

GNU libc compile-time version: 2.23

From that error message though it sounds like your sysroot is inconsistent and that /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1/libgcc_s.so was built against a newer glibc than the one actually present in your sysroot.

$ ll /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1/libgcc_s.so
lrwxrwxrwx 1 root root 35 Apr 13  2016 /home/joostn/toolchains/linux/x86_64/sysroot/usr/lib/gcc/x86_64-linux-gnu/5.3.1/libgcc_s.so -> /lib/x86_64-linux-gnu/libgcc_s.so.1

After replacing that symlink with a relative path, it works!

Mystery solved! Thanks everyone!