How to select a different runtime for linking with clang and ld.lld?

Hi!

I’m invoking the clang linker thru clang with those options:

$ /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/bin/clang -Xlinker --verbose --verbose -Xlinker --verbose -fuse-ld=/build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/bin/ld.lld -target aarch64-linux-android -L/build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/runtimes_ndk_cxx/ -L/build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/runtimes_ndk_cxx/aarch64 --sysroot=/build/pbourguignon/blob/utest-nkernel-sysroot -static -fprofile-instr-generate -fcoverage-mapping build/gcov/out/test_pic-gic_runner.o build/gcov/out/test_pic-gic.o build/gcov/out/mock_bsp.o build/gcov/out/mock_dtb.o build/gcov/out/mock_dtb-iter.o build/gcov/out/mock_hal.o build/gcov/out/mock_nkmem.o build/gcov/out/mock_nkmm.o build/gcov/out/mock_pm-s2r.o build/gcov/out/mock_params.o build/gcov/out/mock_trace-mocks.o build/gcov/out/unity.o build/gcov/out/bcopy.o build/gcov/out/compat_trace.o build/gcov/out/custom_trace.o build/gcov/out/CException.o build/gcov/out/cmock.o -o build/gcov/out/test_pic-gic.out

But it will the run ld.lld as:

    Android (8490178, based on r450784d) clang version 14.0.6 (https://android.googlesource.com/toolchain/llvm-project 4c603efb0cca074e9238af8b4106c30add4418f6)
    Target: aarch64-unknown-linux-android
    Thread model: posix
    InstalledDir: /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/bin
     "/build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/bin/ld.lld" --sysroot=/build/pbourguignon/blob/utest-nkernel-sysroot -EL --fix-cortex-a53-843419 --warn-shared-textrel -z now -z relro -z max-page-size=4096 --hash-style=gnu --hash-style=both --enable-new-dtags --eh-frame-hdr -m aarch64linux -static -o build/gcov/out/test_pic-gic.out /build/pbourguignon/blob/utest-nkernel-sysroot/usr/lib/crtbegin_static.o -L/build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/runtimes_ndk_cxx/ -L/build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/runtimes_ndk_cxx/aarch64 -L/build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/lib64/clang/14.0.6/lib/linux/aarch64 -L/build/pbourguignon/blob/utest-nkernel-sysroot/usr/lib --verbose --verbose build/gcov/out/test_pic-gic_runner.o build/gcov/out/test_pic-gic.o build/gcov/out/mock_bsp.o build/gcov/out/mock_dtb.o build/gcov/out/mock_dtb-iter.o build/gcov/out/mock_hal.o build/gcov/out/mock_nkmem.o build/gcov/out/mock_nkmm.o build/gcov/out/mock_pm-s2r.o build/gcov/out/mock_params.o build/gcov/out/mock_trace-mocks.o build/gcov/out/unity.o build/gcov/out/bcopy.o build/gcov/out/compat_trace.o build/gcov/out/custom_trace.o build/gcov/out/CException.o build/gcov/out/cmock.o -u__llvm_profile_runtime **/build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/lib64/clang/14.0.6/lib/linux/libclang_rt.profile-aarch64-android.a** --start-group /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/lib64/clang/14.0.6/lib/linux/libclang_rt.builtins-aarch64-android.a -l:libunwind.a -lc --end-group /build/pbourguignon/blob/utest-nkernel-sysroot/usr/lib/crtend_android.o
    ld.lld: /build/pbourguignon/blob/utest-nkernel-sysroot/usr/lib/crtbegin_static.o
    ld.lld: build/gcov/out/test_pic-gic_runner.o
    ld.lld: build/gcov/out/test_pic-gic.o
    ld.lld: build/gcov/out/mock_bsp.o
    ld.lld: build/gcov/out/mock_dtb.o
    ld.lld: build/gcov/out/mock_dtb-iter.o
    ld.lld: build/gcov/out/mock_hal.o
    ld.lld: build/gcov/out/mock_nkmem.o
    ld.lld: build/gcov/out/mock_nkmm.o
    ld.lld: build/gcov/out/mock_pm-s2r.o
    ld.lld: build/gcov/out/mock_params.o
    ld.lld: build/gcov/out/mock_trace-mocks.o
    ld.lld: build/gcov/out/unity.o
    ld.lld: build/gcov/out/bcopy.o
    ld.lld: build/gcov/out/compat_trace.o
    ld.lld: build/gcov/out/custom_trace.o
    ld.lld: build/gcov/out/CException.o
    ld.lld: build/gcov/out/cmock.o
    ld.lld: /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/lib64/clang/14.0.6/lib/linux/libclang_rt.profile-aarch64-android.a
    ld.lld: /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/lib64/clang/14.0.6/lib/linux/libclang_rt.builtins-aarch64-android.a
    ld.lld: /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/runtimes_ndk_cxx/aarch64/libunwind.a
    ld.lld: /build/pbourguignon/blob/utest-nkernel-sysroot/usr/lib/libc.a
    ld.lld: /build/pbourguignon/blob/utest-nkernel-sysroot/usr/lib/crtend_android.o
    ld.lld: error: undefined symbol: stderr
    >>> referenced by InstrProfilingFile.c
    >>>               InstrProfilingFile.c.o:(__llvm_profile_get_path_prefix) in archive /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/lib64/clang/14.0.6/lib/linux/libclang_rt.profile-aarch64-android.a
    >>> referenced by InstrProfilingFile.c
    >>>               InstrProfilingFile.c.o:(__llvm_profile_get_path_prefix) in archive /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/lib64/clang/14.0.6/lib/linux/libclang_rt.profile-aarch64-android.a
    >>> referenced by InstrProfilingFile.c
    >>>               InstrProfilingFile.c.o:(__llvm_profile_get_filename) in archive /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/lib64/clang/14.0.6/lib/linux/libclang_rt.profile-aarch64-android.a
    >>> referenced 63 more times

    ld.lld: detected cortex-a53-843419 erratum sequence starting at 24BFFC in unpatched output.

    ld.lld: detected cortex-a53-843419 erratum sequence starting at 24FFF8 in unpatched output.
    clang-14: error: linker command failed with exit code 1 (use -v to see invocation)

    $

Instead, I would want clang to invoke ld.lld as:

    $ /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/bin/ld.lld --sysroot=/build/pbourguignon/blob/utest-nkernel-sysroot -EL --fix-cortex-a53-843419 --warn-shared-textrel -z now -z relro -z max-page-size=4096 --hash-style=gnu --hash-style=both --enable-new-dtags --eh-frame-hdr -m aarch64linux -static -o build/gcov/out/test_pic-gic.out /build/pbourguignon/blob/utest-nkernel-sysroot/usr/lib/crtbegin_static.o -L/build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/runtimes_ndk_cxx/ -L/build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/runtimes_ndk_cxx/aarch64 -L/build/pbourguignon/blob/utest-nkernel-sysroot/usr/lib -T script.ldS --verbose build/gcov/out/test_pic-gic_runner.o build/gcov/out/test_pic-gic.o build/gcov/out/mock_bsp.o build/gcov/out/mock_dtb.o build/gcov/out/mock_dtb-iter.o build/gcov/out/mock_hal.o build/gcov/out/mock_nkmem.o build/gcov/out/mock_nkmm.o build/gcov/out/mock_pm-s2r.o build/gcov/out/mock_params.o build/gcov/out/mock_trace-mocks.o build/gcov/out/unity.o build/gcov/out/bcopy.o build/gcov/out/compat_trace.o build/gcov/out/custom_trace.o build/gcov/out/CException.o build/gcov/out/cmock.o -u__llvm_profile_runtime /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/runtimes_ndk_cxx/libclang_rt.profile-aarch64-android.a --start-group /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/lib64/clang/14.0.6/lib/linux/libclang_rt.builtins-aarch64-android.a -l:libunwind.a -lc --end-group /build/pbourguignon/blob/utest-nkernel-sysroot/usr/lib/crtend_android.o 
    ld.lld: /build/pbourguignon/blob/utest-nkernel-sysroot/usr/lib/crtbegin_static.o
    ld.lld: script.ldS
    ld.lld: build/gcov/out/test_pic-gic_runner.o
    ld.lld: build/gcov/out/test_pic-gic.o
    ld.lld: build/gcov/out/mock_bsp.o
    ld.lld: build/gcov/out/mock_dtb.o
    ld.lld: build/gcov/out/mock_dtb-iter.o
    ld.lld: build/gcov/out/mock_hal.o
    ld.lld: build/gcov/out/mock_nkmem.o
    ld.lld: build/gcov/out/mock_nkmm.o
    ld.lld: build/gcov/out/mock_pm-s2r.o
    ld.lld: build/gcov/out/mock_params.o
    ld.lld: build/gcov/out/mock_trace-mocks.o
    ld.lld: build/gcov/out/unity.o
    ld.lld: build/gcov/out/bcopy.o
    ld.lld: build/gcov/out/compat_trace.o
    ld.lld: build/gcov/out/custom_trace.o
    ld.lld: build/gcov/out/CException.o
    ld.lld: build/gcov/out/cmock.o
    ld.lld: /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/runtimes_ndk_cxx/libclang_rt.profile-aarch64-android.a
    ld.lld: /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/lib64/clang/14.0.6/lib/linux/libclang_rt.builtins-aarch64-android.a
    ld.lld: /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/runtimes_ndk_cxx/aarch64/libunwind.a
    ld.lld: /build/pbourguignon/blob/utest-nkernel-sysroot/usr/lib/libc.a
    ld.lld: /build/pbourguignon/blob/utest-nkernel-sysroot/usr/lib/crtend_android.o
    ld.lld: detected cortex-a53-843419 erratum sequence starting at 248FF8 in unpatched output.
    ld.lld: detected cortex-a53-843419 erratum sequence starting at 2534BC in unpatched output.
    ld.lld: detected cortex-a53-843419 erratum sequence starting at 25AFFC in unpatched output.
    ld.lld: detected cortex-a53-843419 erratum sequence starting at 25DFFC in unpatched output.
    ld.lld: detected cortex-a53-843419 erratum sequence starting at 2A6FFC in unpatched output.

    $ file  build/gcov/out/test_pic-gic.out
    build/gcov/out/test_pic-gic.out: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, with debug_info, not stripped

How can I archive that with clang options?

In short, I have a runtime in /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/lib64/clang/14.0.6/lib/linux/ that is selected using an absolute path, instead of search in the library directory list specified with -L ; I want to select an alternative runtime in /build/pbourguignon/src.devel/vlm-clang-prebuilts-android/clang-r450784d/runtimes_ndk_cxx/ because the former refers to stdout, stderr, etc, which I don’t have in the libc in my sysroot.
And using only clang options, since the command line is processed and used to generate a link script.

Note: I tried with -B instead of -L but it still links with the wrong profile runtime library.

Does ⚙ D98113 [Driver] Also search FilePaths for compiler-rt before falling back fix this? It’s been a while since I wrote that patch and don’t recall the details, but it’s certainly at least related.

Well, eventually I solved the problem by setting up a new sysroot with a more recent libc.a library that has the required symbols.