Re-land -stdlib=libc++ support in clang-cl

Clang Frontend
, ,
1

Hi,

I was looking at adding support for -stdlib=libc++ in clang-cl - my goal was two-fold here, we wanted to experiment with libc++ on Windows for both LLVM itself and our own internal applications, and while we can use it with some flags (-nostdinc++ -nostdlib++ etc) it would be much more coinvent and consistent to support -stdlib=libc++ in clang-cl itself.

When researching this I came across this diff: :gear: D101479 [Driver] Support libc++ in MSVC (llvm.org) and then that it was reverted because it triggered some problems in a runtime build.

I wanted to see what needs to be done to finally be able to land this and use it on Windows.

Any guidance @rnk @mstorsjo @phosek?

Thanks,
Tobias

2 Likes
2

I think I understand the problem now.

When building a cross-compile toolchain and you set the default Stalin to libc++ it will after that patch also apply to clang-cl even if you only want it to apply to the cross-compile targets.

A few solutions:

The short term fix would be to split CLANG_DEFAULT_STDLIB to CLANG_CL_DEFAULT_STDLIB etc. but that feels a little specific for just this case.

We could expand on the config files so that they can be found by the target argument instead of just the name of the binary. So clang --target=x86_64-pc-linux-gnu Would look for the config file x86_64-pc-linux-gnu.cfg in the bin dir.

Or we could complicate the compile time options to allow a CLANG_DEFAULT_STDLIB_x86_64-pc-linux-gnu=libc++ and store that in the binary somehow waves hands.

I think the cfg file from target would be a great generic solution. What do you guys think?

3

The issue we hit was an error when using libc++ with -fno-exceptions on Windows. We’re trying to address that issue in ⚙ D103947 [libcxx] Fix using the vcruntime ABI with _HAS_EXCEPTIONS=0 defined. That change has been stalled for a while, but @ilovepi is making progress now and we’re hoping to land it soon. After it lands, we should be able to reland ⚙ D101479 [Driver] Support libc++ in MSVC since we’re not aware of any other issues.

1 Like
4

That sounds good. But I still think there might be a place for having the cfg files be found by target if you want to set different defaults. Any thoughts on this?

5

I like that idea but I’d suggest starting a separate topic for that proposal so it gets more visibility. In particular, Arm LLVM embedded toolchain relies heavily on configuration files and this would simplify their usage. This topic also came up in LLVM Embedded Toolchains Working Group. I expect that there would be a broader interest in having such a feature from other LLVM users.

6

FWIW I think this should be unblocked now that ⚙ D103947 [libcxx] Fix using the vcruntime ABI with _HAS_EXCEPTIONS=0 defined has landed.

7

I’ve been testing ⚙ D101479 [Driver] Support libc++ in MSVC before re-landing that change and I discovered a few more issues, some of which may need to be addressed first (or at least documented).

What I tried to do is building Clang toolchain that includes libc++ (stage 1), and then use it to build Clang toolchain against libc++ (stage 2), so basically a self-hosting test.

The first issue is missing std::set_new_handler on Windows. libc++ declares this function on non-Windows platforms at llvm-project/new at a6f621b8cacca926d809010c135be038fd05652c · llvm/llvm-project · GitHub, with the definition being provided by the C++ ABI library such as llvm-project/cxa_default_handlers.cpp at a6f621b8cacca926d809010c135be038fd05652c · llvm/llvm-project · GitHub. There’s no such definition provided on Windows though so we end up with undefined symbol linker error in llvm-project/ErrorHandling.cpp at a6f621b8cacca926d809010c135be038fd05652c · llvm/llvm-project · GitHub. I believe that on Windows, we need to provide our own definition that would be implemented using _set_new_handler | Microsoft Learn, this is approach is also used by STL STL/stdhndlr.cpp at 2f8342a3a57fb157d881c6a2d42a917d20e413f8 · microsoft/STL · GitHub.

When using clang-cl with CMake I ran into a challenge since CMake doesn’t use clang-cl for linking, instead it invokes the linker directly through the vs_link_exe or vs_link_dll CMake tool. This doesn’t include the /libpath: to libc++ installation, so we end up with link error because the linker fails to find libc++ which is referenced by the auto-linking directive in libc++ headers llvm-project/__config at bb1c8b1293b60444cab566c99463905f8ffe8c34 · llvm/llvm-project · GitHub. I worked around the issue by manually including /libpath:C:/path/to/stage1/lib/x86_64-pc-windows-msvc in CMAKE_EXE_LINKER_FLAGS, but that experience isn’t great. I don’t know if there’s a better solution though other than extending CMake to do this automatically.

Relatedly, when linking libc++ as a shared library, you also end up with an issue where the linked binary doesn’t know how to find c++.dll. I ran into this during the stage 2 build where clang-tblgen.exe would silently fail. I worked around the issue by manually copying c++.dll to C:/path/to/stage2/bin but this is again not particularly user friendly.

The last issue I haven’t yet found a solution for is building libc++ as a static library. This would be our preferred method of distribution since it avoids the library search order issues, but it doesn’t work at the moment. The problem is that libc++ relies on STL for its implementation of std::exception_ptr llvm-project/exception_pointer_msvc.ipp at 3e6f7ab867ac8c36f0c8a91e7fa1608742702681 · llvm/llvm-project · GitHub. This was introduced ⚙ D32927 [libc++] Implement exception_ptr on Windows and @EricWF mentioned in that change that this shouldn’t introduce any problems because the usages of msvcprt are kept to c++.dll, but this does seem to be causing issues in the static case. Specifically, since libc++.lib has undefined references to __ExceptionPtr* symbols provided by STL, libc++.lib users also need to manually link msvcprt.lib, but this results in link errors:

lld-link: error: bool __cdecl std::uncaught_exception(void) was replaced

A potential solution would be avoid the use of STL and __ExceptionPtr* symbols, and instead implement std::exception_ptr on Windows entirely within libc++ akin to STL STL/excptptr.cpp at 2f8342a3a57fb157d881c6a2d42a917d20e413f8 · microsoft/STL · GitHub. The advantage of this approach is that libc++ would become more self-contained, avoiding the dependency on STL. However, STL’s implementation is non-trivial so this strategy is likely going to require significant effort. A simpler alternative would be to declare std::exception_ptr as unsupported on Windows llvm-project/exception_pointer_unimplemented.ipp at 08d1c43c7023a2e955c43fbf4c3f1635f91521e0 · llvm/llvm-project · GitHub, but that might affect libc++'s usability (I don’t know how widely use std::exception_ptr is).

1 Like
8

I have managed to statically link Clang against libc++ by commenting out the following two lines:

diff --git a/libcxx/src/new.cpp b/libcxx/src/new.cpp
index 48d6f997fdda..1f598f4bcb36 100644
--- a/libcxx/src/new.cpp
+++ b/libcxx/src/new.cpp
@@ -30,7 +30,7 @@ namespace std
 {

 #ifndef __GLIBCXX__
-const nothrow_t nothrow{};
+//const nothrow_t nothrow{};
 #endif

 #ifndef LIBSTDCXX
diff --git a/libcxx/src/support/runtime/exception_msvc.ipp b/libcxx/src/support/runtime/exception_msvc.ipp
index 7e36c7068a28..63f058c362ea 100644
--- a/libcxx/src/support/runtime/exception_msvc.ipp
+++ b/libcxx/src/support/runtime/exception_msvc.ipp
@@ -76,7 +76,7 @@ void terminate() noexcept
 #endif // _LIBCPP_NO_EXCEPTIONS
 }

-bool uncaught_exception() noexcept { return uncaught_exceptions() > 0; }
+//bool uncaught_exception() noexcept { return uncaught_exceptions() > 0; }

 int uncaught_exceptions() noexcept {
     return __uncaught_exceptions();

to workaround the errors when linking msvcprt.lib.

I still think that avoiding the use of __ExceptionPtr* symbols and the dependency on STL would be cleaner, but it’s not a blocker, we just need to figure if omitting the two symbols on Windows is the right solution.

When statically linking libc++, I also had to manually include ucrt.lib, vcruntime.lib, msvcrt.lib and msvcprt.lib. When building libc++ as a shared library, these are included by the CMake build llvm-project/CMakeLists.txt at 2bb50a55b0f5a88bd432ad2691060d82748b5bc0 · llvm/llvm-project · GitHub, but that doesn’t happen when linking libc++ as a static library. I think a better solution would be to use the auto-linking directive for these.

1 Like
9

Thanks for looking into this! I think it’s important to make it easy for clang-cl users to opt into libc++. We skipped this process in Chromium, and as a result, we did not leave behind a well-lit path for other users of Clang on Windows.

I don’t have answers or ideas for your other questions, but regarding the linker /libpath:, I believe that what you did is the way forward, and we need to document it better. Clang actually has many of runtime libraries:

  • builtins (see i128 issues)
  • asan
  • ubsan
  • cfi
  • profile
  • libc++

They all live in that same library directory. Some of them (profile, I think) already rely on autolinking, which works with the library basename, not the full path. If we could make sure that all of our users have that directory on their linker search path, that would make a lot of things easier.

10

We discussed this issue with @tobiashieta at EuroLLVM and our proposal is to move the logic for constructing runtime libraries paths to the WindowsDriver library so it can be shared between clang-cl and lld-link, and set the appropriate library path directly in the linker. This approach follows the lld-link’s handling of MSVC toolchain which was implemented in [lld/coff] Make lld-link work in a non-MSVC shell, add /winsysroot: · llvm/llvm-project@b3b2538 · GitHub.

2 Likes