clang built from source in Mac OSX 10.14 with apple-clang/Xcode missing stdlib from include search path

I recently cloned llvm from git and did a build of clang on my Macbook
(OSX 10.14.6). I have Xcode (11.0) installed and built using Xcode's
included apple-clang. The build worked fine but when I went to compile
my project I got an error where it was trying to #include a stdlib
header file. I'm not really sure but I think this is because in MacOS
Mojave by default the os has System Integrity Protection enabled which
among other things prevents modification to at least some locations
under /usr (root user unable to ln -s ... /usr/include/c++, tried that).
So Xcode does not install or symlink the standard library headers into
/usr/include. apple-clang as installed already includes the installed
stdlib header directories in its search path. Looking at the current git
source I don't see a way to build clang to set a directory to look in
for the stdlib. I did try -DGCC_INSTALL_PREFIX=... and that did not do
the trick, apparently because it does not end up keeping that path for
searching include <...>.

I made a local git branch and made some changes to allow me to build a
clang++ that includes the installed location in the include search path
but before I log a bug/submit a change for review I wanted to see what
constraints around this would end up being acceptable. Should the cmake
scripts automatically pick this up by default on OSX if building with
the Xcode-supplied compiler? Should it be structured/would it be useful
to introduce a top-level #define to set this default stdlib search path
in clang/include/clang/Config/config.h.cmake?

For reference, here's a comparison on a test.cpp file containing simple
#include <string>:

~/src/clang-build-20190828/bin/clang++ -v -c /tmp/test.cpp
clang version 10.0.0 (https://github.com/llvm/llvm-project.git
92ed86d239cdd6ed97dae3084f6537088da88677)
Target: x86_64-apple-darwin18.7.0
Thread model: posix
InstalledDir: /Users/user/src/clang-build-20190828/bin
"/Users/user/src/clang-build-20190828/bin/clang-10" -cc1 -triple
x86_64-apple-macosx10.14.0 -Wdeprecated-objc-isa-usage
-Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all -disable-free
-disable-llvm-verifier -discard-value-names -main-file-name test.cpp
-mrelocation-model pic -pic-level 2 -mthread-model posix
-mframe-pointer=all -masm-verbose -munwind-tables -target-cpu penryn
-dwarf-column-info -debugger-tuning=lldb -ggnu-pubnames
-target-linker-version 450.3 -v -coverage-notes-file
/Users/user/src/llvm-project/test.gcno -resource-dir
/Users/user/src/clang-build-20190828/lib/clang/10.0.0 -stdlib=libc++
-internal-isystem
/Users/user/src/clang-build-20190828/bin/../include/c++/v1
-internal-isystem /usr/include/c++/v1 -internal-isystem
/usr/local/include -internal-isystem
/Users/user/src/clang-build-20190828/lib/clang/10.0.0/include
-internal-externc-isystem /usr/include -fdeprecated-macro
-fdebug-compilation-dir /Users/user/src/llvm-project -ferror-limit 19
-fmessage-length 80 -stack-protector 1 -fblocks
-fencode-extended-block-signature -fregister-global-dtors-with-atexit
-fobjc-runtime=macosx-10.14.0 -fcxx-exceptions -fexceptions
-fmax-type-align=16 -fdiagnostics-show-option -fcolor-diagnostics -o
test.o -x c++ /tmp/test.cpp
clang -cc1 version 10.0.0 based upon LLVM 10.0.0svn default target
x86_64-apple-darwin18.7.0
ignoring nonexistent directory
"/Users/user/src/clang-build-20190828/bin/../include/c++/v1"
ignoring nonexistent directory "/usr/include/c++/v1"
#include "..." search starts here:
#include <...> search starts here:
/usr/local/include
/Users/user/src/clang-build-20190828/lib/clang/10.0.0/include
/usr/include
/System/Library/Frameworks (framework directory)
/Library/Frameworks (framework directory)
End of search list.
/tmp/test.cpp:1:10: fatal error: 'string' file not found
#include <string>
^~~~~~~~
1 error generated.

and with the Xcode clang:

clang++ -v -c /tmp/test.cpp
Apple clang version 11.0.0 (clang-1100.0.33.8)
Target: x86_64-apple-darwin18.7.0
Thread model: posix
InstalledDir:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang"
-cc1 -triple x86_64-apple-macosx10.14.0 -Wdeprecated-objc-isa-usage
-Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all -disable-free
-disable-llvm-verifier -discard-value-names -main-file-name test.cpp
-mrelocation-model pic -pic-level 2 -mthread-model posix
-mdisable-fp-elim -fno-strict-return -masm-verbose -munwind-tables
-target-cpu penryn -dwarf-column-info -debugger-tuning=lldb
-ggnu-pubnames -target-linker-version 512.4 -v -coverage-notes-file
/Users/jbj1/src/llvm-project/test.gcno -resource-dir
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.0
-stdlib=libc++ -internal-isystem
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1
-Wno-framework-include-private-from-public
-Wno-atimport-in-framework-header -Wno-extra-semi-stmt
-Wno-quoted-include-in-framework-header -fdeprecated-macro
-fdebug-compilation-dir /Users/jbj1/src/llvm-project -ferror-limit 19
-fmessage-length 80 -stack-protector 1 -mdarwin-stkchk-strong-link
-fblocks -fencode-extended-block-signature
-fregister-global-dtors-with-atexit -fobjc-runtime=macosx-10.14.0
-fcxx-exceptions -fexceptions -fmax-type-align=16
-fdiagnostics-show-option -fcolor-diagnostics -o test.o -x c++ /tmp/test.cpp
clang -cc1 version 11.0.0 (clang-1100.0.33.8) default target
x86_64-apple-darwin18.7.0
ignoring nonexistent directory "/usr/include/c++/v1"
#include "..." search starts here:
#include <...> search starts here:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1
/usr/local/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.0/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
/usr/include
/System/Library/Frameworks (framework directory)
/Library/Frameworks (framework directory)
End of search list.

I'm always confused when trying to get locally-built Clang to find
stdlib headers on my Mac.

Last time, after upgrading to Mojave, following the TL;DR advice from
this answer made it work for me: macos - Can't compile C program on a Mac after upgrade to Mojave - Stack Overflow

It would certainly be nice if this could somehow work out of the box.

Yeah it'd be great for it to work out of the box, the headers are
already there in the normal install of Xcode, so not like it's "missing"
some other component and it would be great to work out of the box. Based
on the (non-)responses I guess I won't know how receptive the project
will be until I submit a patch.

When building Clang the build system could run "xcrun --show-sdk-path" to identify the SDK path. Or to be even more flexible: when running Clang it could run the above code and it will avoid the need for hard coding the path to the SDK.

Yeah, there are lots of ways it /could/ be solved but I guess what I'm
proposing is that if there exists a default SDK that goes with the
compiler you're using to build clang and we know about it, shouldn't we
be providing the mechanism internally to find it (only in the case where
it cannot be found in the obvious places eg. /usr/include)? In the
ordinary case (where it is installed in /usr/include) the compiler would
already pick it up, so it's not like we're injecting strange behavior
right? Or I'm crazy? I don't know, but ending up with a good compiler
that cannot find the std lib, even though the previous compiler did, is
just kicking the problem "upstream". It seems a reasonable minimal
requirement that running clang++ without extra include paths should be
able to find the standard library.

I'm always confused when trying to get locally-built Clang to find
stdlib headers on my Mac.

Last time, after upgrading to Mojave, following the TL;DR advice from
this answer made it work for me: macos - Can't compile C program on a Mac after upgrade to Mojave - Stack Overflow

Seems that advice is obsolete. :frowning: Nothing else I found online worked either.

It would certainly be nice if this could somehow work out of the box.

Until then, anyone know how to make one's own build of clang find C++ headers on macOS?

Thanks,

Sean

Just replying to say that, BEFORE the move to the monorepo, I never had any trouble with this checklist: https://quuxplusone.github.io/blog/2018/04/16/building-llvm-from-source/
AFTER the move to the monorepo, I updated my blog post with something that I thought had worked for me, but it doesn’t seem to work really, and I have not had time (or probably the ability) to figure out what’s wrong. It appears that essentially, libc++ can no longer be built with OSX’s native compiler, and Clang itself can’t be built without libc++.
That is, as of 2019-12-10, the instructions here do not actually work for me: https://quuxplusone.github.io/blog/2019/11/09/llvm-from-scratch/

–Arthur

Apply my patch and build like :

cmake -DLLVM_ENABLE_PROJECTS=clang -G "Unix Makefiles"
-DCMAKE_BUILD_TYPE=Release
-DCLANG_XCODE_TOOLCHAIN_ROOT=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain
../llvm-project/llvm

patch.diff (2.13 KB)

Using Jens’ patch (attached), I managed to get make clang to build. However, I still can’t get make cxx to build.

cmake -G “Ninja” -DLLVM_ENABLE_PROJECTS=“clang;libcxx;libcxxabi” -DCMAKE_BUILD_TYPE=RelWithDebInfo
-DCLANG_XCODE_TOOLCHAIN_ROOT=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain
…/llvm
ninja cxx

[223/272] Linking CXX shared library lib/libc++abi.1.0.dylib
FAILED: lib/libc++abi.1.0.dylib
[…snip…]
Undefined symbols for architecture x86_64:
“__ZTIDu”, referenced from:
-exported_symbol[s_list] command line option
“__ZTIPDu”, referenced from:
-exported_symbol[s_list] command line option
“__ZTIPKDu”, referenced from:
-exported_symbol[s_list] command line option
“typeinfo for __float128 const*”, referenced from:
-exported_symbol[s_list] command line option
“typeinfo for __int128 const*”, referenced from:
-exported_symbol[s_list] command line option
“typeinfo for unsigned __int128 const*”, referenced from:
-exported_symbol[s_list] command line option
[…snip…]

Then I tried adding compiler-rt to LLVM_ENABLE_PROJECTS and building ninja compiler-rt, but it also errors:

cmake -G “Ninja” -DLLVM_ENABLE_PROJECTS=“clang;libcxx;libcxxabi;compiler-rt” -DCMAKE_BUILD_TYPE=RelWithDebInfo
-DCLANG_XCODE_TOOLCHAIN_ROOT=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain
-DCOMPILER_RT_HAS_WNON_VIRTUAL_DTOR_FLAG=yes
…/llvm
ninja compiler-rt

[533/839] Building CXX object projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_osx_dynamic.dir/rtl/tsan_interceptors_mac.cpp.o
[…]/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_mac.cpp:26:10: fatal error: ‘os/lock.h’ file not found
#include <os/lock.h>
^
1 error generated.

Anyone got any tips for building libc++ on Mac OSX?

–Arthur

patch.diff (2.13 KB)

Hmmm, unfortunately I don’t use libc++.

Jens,

I've finally tried your patch (against current master), and it indeed solves the problem for me.

Is there a ticket in bugzilla for this? I feel it should be a 10.0.0 release blocker.

Cheers,

Sean

Regrettably it seems that others do not see this as a legitimate bug
:-(. I'm happy to create a ticket, but as a noob to cfe-dev I hit the
mailing list first to test the waters and the reaction didn't seem to
indicate that anyone else regarded it as something that needed fixing.
I'm with you though, I regard this as straight-up broken: ending up with
a compiler that doesn't know where it's standard library is may be a
valid compiler but not a generally useful one.

I just built clang from git master again and ran into this issue again. Jens, did you ever create a ticket? I'd like to CC myself. Otherwise I can write one up.

Cheers,

Sean

  • stepan@

Hi guys! Any progress on that?
Thanks!
-Stepan

Issue reproduced on llvm-9, and not reproduced on llvm-8. Unfortunately I’ll have free time on next weekends only. Will try to bisect next weekends.

Stepan

I’m happy to get a patch into acceptable shape that would fix it, but it seemed like the initial response felt more like WONTFIX. Did someone else create a ticket?

I created one:
<https://bugs.llvm.org/show_bug.cgi?id=45880&gt;

No responses on it so far.

Cheers,

Sean

I used to rely on Jens’ patch in my “How to Build LLVM From Source” blog post; but as of March 2020 (or earlier), I found that the need for it had gone away, at least on OSX 10.14.6. I believe patch https://reviews.llvm.org/D69221 may have been related.
https://quuxplusone.github.io/blog/2019/11/09/llvm-from-scratch/

On the other hand, I have completely given up on trying to build LLVM/Clang on OSX 10.13.6 anymore.

–Arthur

That fixed it eh? Hmmmm, strange. The only change seems to be that it appends sysroot for seach paths that are not specified as absolute paths. Maybe something else fixed it? I’ll have to pull clean source and give it a try.

I just rebuilt from git master (without the patch) on macOS 10.13.6 and the resulting clang still can't find the STL headers. Then I rebuilt with the patch, and it finds the headers. So nothing seems fixed to me. :frowning:

Sean