-fshort-wchar and std::wstring

Runtimes C++
1

Hi, I’m trying to compile an old C++ code on macOS that relies on having -fshort-wchar, so the code assumes sizeof(wchar_t) == 2 bytes.

The problem is that external libc++.dylib was not compiled with -fshort-wchar, so everything in this library still expects sizeof(wchar_t) to be 4 bytes. So, any usage of std::wstring (or anything else that somehow uses wchar_t) from a project compiled with -fshort-wchar ends up in making a lot of undefined behaviors, including out of bounds memory writes, and crashes.

Is there an option to somehow convince clang to link a short-wchar-enabled libc++ to my project? What I would need to do in order to make it happen? From what I was thinking, I would need to:

  1. Compile my own libc++ with -fshort-wchar enabled,
  2. Statically link it to my app,
  3. Continue using -fshort-wchar and std::wstring.

Is this even possible to achieve?

2

Why don’t you use std::u16string? Then you have a fixed-size character across platforms. Otherwise, the process you described is almost exactly what you have to do. It would probably also be a good idea to either

  1. define _LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS and compile with -fvisibility=hidden, or
  2. set LIBCXX_ABI_NAMESPACE to something different like __short_wchar

to avoid dynamically linking against the wrong symbol.
(BTW if you decide to link statically you can also look into enabling the unstable ABI)

1 Like
3

Thanks. Using u16string is a valid point, but the code organization makes this hard (it was written for Windows without thinking about portability).

I did something like this:

1. git -C temp/libcxx3/src clone https://github.com/llvm/llvm-project.git
2. git -C temp/libcxx3/src/llvm-project checkout llvmorg-14.0.6
3. cmake -S temp/libcxx3/src/llvm-project/runtimes -B temp/libcxx3/obj -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libunwind" -DCMAKE_CXX_FLAGS="-fshort-wchar -fvisibility=hidden" -DCMAKE_INSTALL_PREFIX=temp/libcxx3/out -DLIBCXX_ENABLE_SHARED=OFF -DLIBCXX_ABI_NAMESPACE=__short_wchar -D_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS=ON
4. cmake --build temp/libcxx3/obj -t cxx -t cxxabi -t unwind
5. cmake --build temp/libcxx3/obj -t install-cxx -t install-cxxabi -t install-unwind

The path passed in CMAKE_INSTALL_PREFIX now has includes and lib files with the newly compiled library. I’ve linked it to my project by:

1. Add `nostdinc++` to compile options,
2. Add `nostdlib++` to link options,
3. Link to the `libc++.a`, `libc++abi.a` files,
4. Add this include dir: of ${CMAKE_INSTALL_PREFIX}/include/c++/v1

And it seems that it does work as intended. Thanks.

4

Great that it works! You actually don’t have to pass -fvisibility=hidden -D_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS=ON to the build command. You’d have to pass the flags when building your application if you don’t change the ABI namespace while compiling libc++. (i.e. you don’t have to pass it at all). Also, if you want some performance for free, you can add -DLIBCXX_ABI_UNSTABLE=On when building libc++. You have to build everything that uses libc++ from source, but that shouldn’t be a problem, since you seem to already do that.

1 Like
5

Many of the libc++ functions which take wchar_t are simply wrappers around libc functions, so you’d most-likely need to build a new libc as well. Which is not easily doable on macos.

I suppose you may be lucky enough not to need to call those functions? It doesn’t seem like a great idea to depend on that, though…

6

That’s a good point. I haven’t encountered any problems probably because my project also uses -nostdlib. Lots of libc functions are reimplemented (like wcslen that expects 2-byte wchar_t). But without -nostdlib it probably wouldn’t work indeed.