[RFC] Project Hand In Hand (LLVM-libc/libc++ code sharing)

[michaelrj-google]

This is an RFC based on the design that was presented at the monthly libc meeting. For notes on our discussion, see the meeting notes: Monthly LLVM libc meeting - #11 by michaelrj-google

Context

Objective

The LLVM-libc and libc++ projects should be capable of sharing implementations to improve performance and reduce maintenance overhead.

Background

In functionality, libc and libc++ are very similar with slightly different interfaces. They both provide utilities for string manipulation, math, and general algorithms. Their interfaces are slightly different and incompatible, leading libc++ to be forced to either create an awkward translation layer (like in constexpr_c_functions.h), or to reimplement logic that already exists in libc (like in to_char’s floating point support). This isn’t bad design, just a result of the constraints they’re working with.

Additionally, LLVM-libc would like to support libc++, but some of the existing awkward translation layer uses obscure functions that LLVM-libc doesn’t yet support. (Like their use of strtoll_l with the C locale specified in locale)

Design

Overview

The core piece of this effort will be moving functionality shared by LLVM-libc and libc++ to a shared directory within the libc directory. This will allow LLVM-libc and libc++ to share code with a common interface, providing multiple benefits.

The shared code will be the same for both implementations, avoiding possible implementation mismatches. Small functions can be inlined into both libraries, and larger functions can be deduplicated with link time optimization when statically linking libc++ and libc. Implementation effort can be shared, reducing duplicated effort and improving developer velocity.

Detailed design

• Start of implementation is creating an interface in libc/shared
  • The code in libc/shared will depend on libc/src/__support, which may only depend on libc/include/llvm-libc-macros and libc/include/llvm-libc-types

• These folders should be header only, this is already mostly true. There are some cases of non-header files in support that will need to be cleaned up.

• The interface will be designed to fit libc++ needs, but otherwise mostly be a thin wrapper over libc’s existing support functions.

• Create libc++ dependency into libc/shared

  • This must be done so that none of the libc headers are even transitively included in the public libc++ headers
    • If the libc shared functions are leaked to the public headers, then the header only structure would cause it to pull in large parts of the libc support machinery.
      • We should have a header guard to ensure that this doesn’t happen, similar to what is currently done for headers in /usr/include/bits/
    • This may mean transforming an existing public libc call (such as the call to strtoll_l) into a call to a libc++ internal function, defined in a .cpp file, that calls the libc/shared function.
      • This may affect users who expect a given file to be header only, which files does that matter for?
  • This will be a build system break for anyone not on cmake
  • Might need to have a fallback implementation in libc++
  • Might also need a libcxx/shared directory with a readme explaining what the libc/shared directory is
  • The libc++ code must not include the libc shared interface from their public headers, since that would leak the libc internal code to userspace

• Modify libc++ to remove translation layers to libc where possible (e.g. calling islower_l(char, c locale) becomes internal::islower(char))

• Figure out what to do with __support/CPP since it contains standalone of implementations of libc++ functions

  • At the start, just leave it. This is a complex issue that will require input from the libcxx contributors.
  • Eventually we want to be able to share standalone code both ways between libc and libcxx
  • In future, create a design that doesn’t require building libcxx before building libc, possibly by using header-only libraries.
  • This may end up in a shared utility library, which has the same issues described below.
    • Notably, once the build system issues are resolved they won’t be an issue again.
    • It could be argued that not creating this shared library is taking on technical debt, and this would be paying that down.

Alternatives considered

Project management

Work estimates

This is a new interface that will require continuous support, but after the initial lift it should be fairly simple to keep up.

Initial cleanup/build will take 2-4 weeks, depending on how long gathering approvals from stakeholders and finalizing the design takes. If the design is approved as-is, then the first function (from_chars using string to float) will likely take 1-2 weeks, with an additional 1 week for writing new documentation. Each further group of functions (e.g. ctype) would likely take an additional week, though these can be parallelized.

Ongoing support will mostly focus on bug fixes that take no additional time (since they’ll be necessary for libc anyways). If libcxx only uses the smallest subset of libc code necessary, then ongoing support will likely take 1-2 weeks per year. If libcxx integrates more deeply with the libc internals, then 4-5 weeks per year may be necessary, though some of that should be handled by libcxx developers.

Documentation plan

Downstream vendors of libc++ with their own build systems (e.g. Fuchsia) will need to be provided documentation on which pieces of libc are necessary. This should be created alongside the initial patch and distributed directly to users who live at head, then distributed alongside any future releases.

Libc and libc++ developers should document their internal interface during the design process, and make documentation available on LLVM developer pages after the libc/shared directory is created.

[ldionne]

Thanks a lot for the proposal.

From the libc++ side of things, I think the main thing we have to gain here is to decouple ourselves from the underlying C Standard Library by clarifying what our “contact surface” with such a library is. I think there is a lot of value in doing that, however I believe we should start by identifying these internal libc++ APIs and clarifying them.

Then, we can proceed to move them to a shared place (e.g. llvm-libc/shared as discussed here or something else like under runtimes/). But that is a detail – IMO the really important bit here is to begin this work by figuring out what APIs we actually want to abstract away from the libc++ side.

I would also like for us to be really cautious about having fallbacks in libc++. If we create a nice API for e.g. localization, we should not have a fallback for other C Standard Libraries. We should unconditionally use that nice internal API we’ve created and shove the complexity elsewhere – otherwise we will simply make libc++ more complicated by supporting more ways to do things.

Overall, I think this proposal is definitely interesting so thanks for putting it together. I believe this can definitely work and be beneficial, but as I said above I would like for us to identify these “internal APIs” first and implement them in libc++ before we start taking any dependency on another directory or LLVM project.

[jyknight]

@ldionne

I would also like for us to be really cautious about having fallbacks in libc++. If we create a nice API for e.g. localization, we should not have a fallback for other C Standard Libraries.

It would be nice if we could eventually end up in a world where libc exposed sufficiently-expressive APIs such that we didn’t require a duplicate implementation of half of libc within libc++. (timezone database access is an unfortunate new example of this sort of thing…)

But that may well be effectively infeasible, given the decoupled nature of the C and C++ standards, and of their corresponding library implementations.

[mordante]

Thanks for the proposal. I feel this can indeed be very useful to avoid duplicating effort. I fully agree from_chars is a great way to start. Since this would be a pure addition for libc++ it would be a lot easier to experiment with and not risk breaking existing code.

@ldionne

If we create a nice API for e.g. localization, we should not have a fallback for other C Standard Libraries. We should unconditionally use that nice internal API we’ve created and shove the complexity elsewhere – otherwise we will simply make libc++ more complicated by supporting more ways to do things.

+1

In libc++ we require modern C++ compilers and use C++23 to compile our dylib. I wonder what the language and compiler restrictions for libc are. This is relevant if we want to move libc++ code the shared project. Timezones have been mentioned and I’m working on an implementation in libc++, where most of the code resides in the dylib.

As discussed during the libc++ monthly today we should look at how we can share code without ODR violations. I would prefer to avoid depending on LTO to remove duplicates. I think we should investigate that upfront.

[efriedma-quic]

I’m having trouble following how the proposed build works. Is the idea that you share the source code, but not binaries (i.e. build the code twice)? Or that libc++ calls APIs exposed by llvm-libc at runtime? Or that there’s a libc_shared library that’s a dependency for both libc and libc++?

[michaelrj-google]

Re: ldionne

The contact surface discussed in the libc++ meeting was mostly focused on these sections:

Locale-free char/string functions (e.g. isdigit(x, _LIBCPP_GET_C_LOCALE))
Floating point conversion (to_chars/from_chars, strtof/strfromf)
Threading (pthreads, C11 threads)
Timezones

These are pieces that LLVM-libc and libc++ can share relatively easily, with some additional complexity around timezones. Having a C++ to C++ interface allows for a nicer interface, using things like optional and stringref instead of returning null or passing a pointer and length.

Where we place this code is still somewhat to be determined. I think eventually we will want a separate shared directory inside of llvm-project, but to keep the proposal simple I have focused on code that exists in LLVM-libc and would be useful in libc++. For that code, placing it in libc/shared should be fine, and that allows us to put off a major refactor of our libc/__support directory.

For fallbacks in libc++, that’s ultimately up to you. I think that there are some places where you will still want to go through the public libc interface (e.g. math functions), but for places where we have a good internal API then we might not need a fallback.

Re: jyknight

My plan isn’t to replace the entire libc/libc++ contact surface with an internal API, this is more focused on places where libc and libc++ have a shared underlying goal (e.g. convert a string to a float) but incompatible interfaces (e.g. strtof vs from_chars). The decoupled nature of the C and C++ standards is exactly what causes this problem, and and my proposed solution is effectively coupling our libraries at a lower level so that it can be specialized for each public interface.

Re: mordante

Our compiler support is listed here: Compiler Support — The LLVM C Library
Currently we support Clang 11+ and GCC 12.2+, and I believe we use C++20, though I’m not entirely certain, and I’m not opposed to increasing the minimum required compiler/C++ version. We can discuss how to handle timezones, but I think having them as the first piece in a shared top level directory would work well. We would need it to be a build option to add it to our libc, at least at first, but requiring that for timezone support shouldn’t be a hard sell. Anyone who needs to touch timezones is advanced enough to flip some cmake switches.

As for sharing code without ODR violations, I think we’ll need to consider two cases:

1. libc++ is being built on its own (and a different libc is used)
2. LLVM-libc and libc++ are being built together

In case 1, we don’t need to worry about ODR at all. In case 2, the design I had been imagining would be to create a combined libc/libc++, though I think this is a good ways off. As a temporary measure, the LIBC_NAMESPACE could be defined differently by LLVM-libc and libc++ to avoid ODR violations. Additionally the plan for the shared code is for it to be in a header-only library, so it should all be marked inline anyways.

Re: efriedma-quic

My plan for the start is to stick to a header only shared library included internally, so sharing the source code and building it twice. In the future I’d like to make a combined libc/libc++ library, but that’s not part of this proposal nor do I expect it to happen anytime soon.

[codonell]

Just watching from the side lines here as a libc maintainer (glibc).

I think it is entirely a good thing if libc++ and libc can share code at a low-level where it makes sense, and doing that might result in innovative internal APIs that simplify maintenance and support.

The identified subsystems of char/string, floating point conversions, threading, and timezones are well understood and well structured. In libstdc++ and glibc we made the mistake several decades ago of trying to layer the standard IO subsystems and it was a lot of pain to dis-entangle (years of work that is still ongoing). So keep in mind that any actual ABI dependencies you create are going to be drag on the projects. However if you just share code at build time then that works out great.

There are many projects, look at go-lang where it reimplements some syscall assembly, that would really like a syscall layer that we can share amongst all the projects, but that’s a task nobody has taken up yet.

[michaelrj-google]

Thank you for the comment, it’s always great to hear from a fellow libc maintainer.

I’m glad to hear you think this is a good idea, do you have any more information on the libstdc++/glibc IO problems? I would love to learn more so we can avoid any pitfalls you ran into.

As for a syscall-only layer, that may be possible with our library. When building LLVM-libc there are options to let you set the list of functions you want to be provided. If you had a list that only contained the syscall wrapper functions then that would generate a library with just those functions.

[codonell]

Re:michaelrj-google

The tl;dr is that we allowed FILE* to be treated like a class when in fact it wasn’t and the ABI impact of that has created drag for the project.

Florian Weimer wrote this up here: LibioVtables - glibc wiki, and was one of the principal authors in cleaning some of this up including the vtable hardening which can detect new applications and harden the vtables.

My key point here is that the ABI dependencies, or even API depedencies create drag on the project, and this is a counter-balance to sharing that code. This is just a question you have to ask every time you share some code bewteen the projects. In this particularly case I think the developers in the past saw iostream and FILE* being similar enough to make them work together, but it didn’t pan out because the standards are really quite different and end up needing different things from the APIs. I think something lower level and internal only is the right way to go initially.

[cor3ntin]

In the clang frontend, we will need access to high-quality, correctly rounded implementations of the <cmath.h> function pretty soon
(to implement https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2023/p1383r2.pdf, we need to be able to constant evaluate maths builtins, and unlike GCC, we cannot rely on GMP or equivalent because of licence / portability).

It occured to me that using the llvm libc implementation would be a good solution.
Have you considered whether other llvm projects (such as clang) could reuse the shared implementation of libc (specifically math functions)

Thanks!

[michaelrj-google]

I think sharing the math functions would be great, this proposal is focused on libc++ to keep it simple but I expect that we’ll extend this more in the future.

As for using this in clang, there’s only one concern I have: We support a much narrower range of targets than LLVM as a whole does. Right now we only support x86_64, arm64/32, and RISC-V 64/32. I think the generic implementations of math functions would work on other targets (@lntue may have some comments) but these are the ones we currently test for accuracy/speed. Do you have a list of hosts the clang frontend has to support?

[efriedma-quic]

In general, we can’t use native floating-point types: we need to allow cross-compiling between targets that define long double differently. So we can’t directly use the libc code; we’d need to port it to use APFloat or equivalent. (With enough templates, it might still be possible to share the code.)

Given that, I don’t think portability is something we need to directly address: porting the code to use APFloat solves all portability issues.

[jcranmer]

In general, we can’t use natively floating-point types: we need to allow cross-compiling between targets that define long double differently. So we can’t directly use the libc code; we’d need to port it to use APFloat or equivalent. (With enough templates, it might still be possible to share the code.)

Strawman idea:

If libc implemented the long double not directly on a long double type but an x86_fp80, ppc_fp128, binary128 triad of types, we could have some header magic that makes it automatically use an implementation of APFloat if it’s not supported. E.g., something like this:

#ifdef FLOAT128_BUILTIN
using binary128 = __float128;
#elifdef FLOAT128_LONGDOUBLE
using binary128 = long double;
#else
struct binary128 {
  APFloat inner;
  friend binary128 operator*(binary128 lhs, binary128 rhs) {
     binary128 res;
     res.inner = lhs.inner;
     res.inner.multiply(rhs.inner, current_rounding_mode());
     return res;
};
#endif

I think there’s definitely something that can be done here to make an APFloat-based implementation of exotic types for libc possible, such that libm routines can be reused for APFloat helpers.

[pogo59]

The selection would have to be dynamic based on target triple, not a compile-time choice. The compile-time evaluation needs to be done using the target’s concept of arithmetic types.

[jyknight]

The math library doesn’t necessarily need to implement dynamically-format-agnostic functions, as long as it exposes functions taking a fixed format, which together implement the operations for all floating-point format we require – even those that aren’t exposed as C/C++ types on the compiler’s host platform.

Separately, it should be fine for the math library functions (and, indeed, APFloat in general) to use native types and instructions instead of a soft-float representation, where possible on a given host platform. We don’t do that today in APFloat (e.g. for 32 or 64-bit ieee floats), but I don’t think there’s any principled reason not to, as long as such a native implementation can be shown to produce identical results vs the current non-native APFloat code.

That could be helpful, if performance is a concern.

[jcranmer]

Separately, it should be fine for the math library functions (and, indeed, APFloat in general) to use native types and instructions instead of a soft-float representation, where possible on a given host platform. We don’t do that today in APFloat (e.g. for 32 or 64-bit ieee floats), but I don’t think there’s any principled reason not to, as long as such a native implementation can be shown to produce identical results vs the current non-native APFloat code.

The problem with using native types to implement APFloat operations is that native types tend to deviate from strict IEEE 754 compliance in random ways that aren’t necessarily easily controllable–think about someone compiling with fast-math or a system that defaults to FLT_EVAL_METHOD=2 or defaults to denormal flushing or is too happy to move floating point code around rounding mode changes.

[jyknight]

I mean…no reasonable person would build LLVM itself with -ffast-math, right?

I expect the APFloat unit tests cover enough tricky edge cases that you’d get test failures if you did do such a thing. More importantly, those tests should also catch if the default build configuration of LLVM had incorrectly chosen to enable native-float operations on a platform where they’re non-conforming.

[cor3ntin]

@michaelrj-google

I think sharing the math functions would be great, this proposal is focused on libc++ to keep it simple but I expect that we’ll extend this more in the future.

Thanks, this answer my question

@michaelrj-google

As for using this in clang, there’s only one concern I have: We support a much narrower range of targets than LLVM as a whole does. Right now we only support x86_64, arm64/32, and RISC-V 64/32. I think the generic implementations of math functions would work on other targets (@lntue may have some comments) but these are the ones we currently test for accuracy/speed. Do you have a list of hosts the clang frontend has to support?

I don’t think we want to derail too much the discussion here (sorry for that).
But in general, clang will not care deeply about speed, but accuracy is important. My hope would be that the generic implementation would be fine. As other have pointed out, we would want to use the same type width as the target platform, be independent of the host, and of the target rounding mode. It’s unclear to me how we will achieve all of that, but in the absence of a licence friendly GMP equivalent, I just wanted you to be aware of potential needs beyond libc++. Thanks!