Strange clang behavior when compiled against musl

I have managed to compile llvm and clang against musl, but it behaves really strange:

At first I tried to launch the compiler with musl dynamic loader:

    $ LD_LIBRARY_PATH=/path/to/musl/lib /path/to/musl/lib/ld-musl-x86_64.so.1 /path/to/llvm/bin/clang -v
    clang version 4.0.0 (GitHub - llvm-mirror/clang: Mirror kept for legacy. Moved to https://github.com/llvm/llvm-project 40adebeca0f99006d407508653c2cbd270a1a51c) (https://github.com/llvm-mirror/llvm 943496ffc4e7cb9d7dd6f5119325a7583e2cc31f)
    Target: x86_64-pc-linux-musl
    Thread model: posix
    InstalledDir: /path/to/llvm/bin

It worked, but couldn't compile binaries:

    $ LD_LIBRARY_PATH=/path/to/musl/lib /path/to/musl/lib/ld-musl-x86_64.so.1 /path/to/llvm/bin/clang -v -c hello.c
    clang version 4.0.0 (GitHub - llvm-mirror/clang: Mirror kept for legacy. Moved to https://github.com/llvm/llvm-project 40adebeca0f99006d407508653c2cbd270a1a51c) (https://github.com/llvm-mirror/llvm 943496ffc4e7cb9d7dd6f5119325a7583e2cc31f)
    Target: x86_64-pc-linux-musl
    Thread model: posix
    InstalledDir: /path/to/llvm/bin
     "/path/to/musl/lib/ld-musl-x86_64.so.1" -cc1 -triple x86_64-pc-linux-musl -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name hello.c -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64 -v -dwarf-column-info -debugger-tuning=gdb -coverage-notes-file /path/to/hello/hello.gcno -resource-dir /path/to/musl/lib/clang/4.0.0 -isysroot /path/to/musl -internal-isystem /path/to/musl/usr/local/include -internal-isystem /path/to/musl/lib/clang/4.0.0/include -internal-externc-isystem /path/to/musl/include -internal-externc-isystem /path/to/musl/usr/include -fdebug-compilation-dir /path/to/hello -ferror-limit 19 -fmessage-length 80 -fobjc-runtime=gcc -fdiagnostics-show-option -o hello.o -x c hello.c
    /path/to/musl/lib/ld-musl-x86_64.so.1: cannot load -cc1: No such file or directory

As you can see it is trying to execute a compilation command with omitted clang binary path.

So I thought that it would work in chroot with nothing but musl and clang in it. So I chrooted to a newly created directory, put musl and clang in it, and it still didn't work:

    (chroot)$ clang -v -I/include -c hello.c
    clang version 4.0.0 (GitHub - llvm-mirror/clang: Mirror kept for legacy. Moved to https://github.com/llvm/llvm-project 40adebeca0f99006d407508653c2cbd270a1a51c) (https://github.com/llvm-mirror/llvm 943496ffc4e7cb9d7dd6f5119325a7583e2cc31f)
    Target: x86_64-pc-linux-musl
    Thread model: posix
    InstalledDir: /bin
     "" -cc1 -triple x86_64-pc-linux-musl -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name hello.c -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64 -v -dwarf-column-info -debugger-tuning=gdb -coverage-notes-file /hello/hello.gcno -resource-dir ../lib/clang/4.0.0 -I /include -isysroot /path/to/musl -internal-isystem /path/to/musl/usr/local/include -internal-isystem ../lib/clang/4.0.0/include -internal-externc-isystem /path/to/musl/include -internal-externc-isystem /path/to/musl/usr/include -fdebug-compilation-dir /hello -ferror-limit 19 -fmessage-length 139 -fobjc-runtime=gcc -fdiagnostics-show-option -o hello.o -x c hello.c
    error: unable to execute command: Executable "" doesn't exist!

So it still doesn't launch clang as it should. But if I just copy and paste the produced line replacing `""` with `clang` it works and produces fine object file.

What else can I try to make it work? Does clang have public bugzilla?

Where can I send my musl support patches?

Clang uses this logic in llvm/lib/Support/Unix/Path.inc to find itself (

):

...
#elif defined(__linux__) || defined(__CYGWIN__)
  char exe_path[MAXPATHLEN];
  StringRef aPath("/proc/self/exe");
  if (sys::fs::exists(aPath)) {
      // /proc is not always mounted under Linux (chroot for example).
      ssize_t len = readlink(aPath.str().c_str(), exe_path,
sizeof(exe_path));
      if (len >= 0)
          return std::string(exe_path, len);
  } else {
      // Fall back to the classical detection.
      if (getprogpath(exe_path, argv0))
        return exe_path;
  }
#elif defined(HAVE_DLFCN_H)
...

Presumably we are hitting the else, which would explain the bad behavior in
the first example, where clang uses the musl loader for the -cc1 action. I
don't know what went wrong in the chroot.

You can send patches to llvm-commits@lists.llvm.org and file bugs at
http://llvm.org/bugs.

Thanks for looking into this!

Try removing “/path/to/musl/lib/ld-musl-x86_64.so.1” from the invocation and see if that helps:

$ LD_LIBRARY_PATH=/path/to/musl/lib /path/to/musl/lib/ld-musl-x86_64.so.1 /path/to/llvm/bin/clang -v

because the shell thinks you want to run “”/path/to/musl/lib/ld-musl-x86_64.so.1" not “/path/to/llvm/bin/clang”

hth…
don

Actually, the problem is the if part. If you invoke the loader as the binary name, /proc/self/exe returns the path to the loader, not the path to clang.

E.g.:

$ /bin/ls -l /proc/self/exe

lrwxrwxrwx 1 jyknight eng 0 Dec 9 12:49 /proc/self/exe → /bin/ls

$ /lib64/ld-linux-x86-64.so.2 /bin/ls -l /proc/self/exe
lrwxrwxrwx 1 jyknight eng 0 Dec 9 12:49 /proc/self/exe → /lib/x86_64-linux-gnu/ld-2.19.so

Further, even if it DID return the path to the clang binary, you’d still be in trouble, since the exec would then bypass your loader choice when invoking the subprocess.

I’d suggest running patchelf to change the interpreter embedded in the binary, if that’s what you need to do, instead of running the loader manually.

No, it doesn’t help. It was my first thought as well, so tried to invoke clang inside chroot environment (I described it in the second part of my original message), but all I got was:

error: unable to execute command: Executable “” doesn’t exist!

Regards,
Dmitry

Thank you very much! Setting interpreter with patchelf actually did the trick and now it works.