[RFC] Libc++ taking a dependency on Boost.Math for the C++17 Math Special Functions

Runtimes C++
1

Summary

This RFC proposes permitting the use of Boost.Math within libc++ for the implementation of C++17 math special functions, a standard feature currently unimplemented in libc++. These functions are specified in <cmath> under the C++17 special functions section, including std::assoc_legendre, std::beta, std::ellint_1, and many others. This would involve creating a specific licensing carve-out to allow this dependency, with the understanding that it applies exclusively to this use case within libc++ and to Boost.Math (not arbitrary Boost libraries).

Motivation

Over the past few years, we’ve had multiple discussions about Boost.Math and the C++17 special math functions, where we’ve developed a fairly strong consensus on the desire to implement this feature by reusing an existing, high-quality implementation rather than writing our own from scratch. Importantly, libc++ contributors currently do not have the bandwidth or the domain expertise required to implement these functions from scratch, especially not with the level of correctness and performance that libc++ aims to provide.

Boost.Math is hence a great candidate, with the only issue being that the code is licensed under the Boost Software License instead of the LLVM license.

Integration Strategy

We propose integrating Boost.Math into libc++ in a way that fully encapsulates it as an implementation detail, with no ABI or header-level exposure to users of libc++.

Specifically:

  • Math special functions will be declared in libc++ headers using libc++’s own declarations, with appropriate visibility and ABI annotations.
  • The definitions of these functions will reside in the libc++ built library (whether static or shared).
  • Internally, these function definitions will call into Boost.Math, but this usage will be entirely contained within the translation units that implement the functions.
  • The Boost.Math code would be added to the LLVM monorepo under third-party/ (we already have e.g. GoogleBenchmark in there), with an explanation of where this code is allowed to be used.

This strategy has some important benefits:

  • No symbols from Boost.Math will be part of libc++'s public ABI surface.
  • No Boost.Math headers will be included or transitively exposed by libc++ headers.
  • The implementation of each function could be swapped independently in the future (e.g., if a better implementation becomes available).
  • No attribution or redistribution of Boost.Math source code is required, since we do not ship any Boost source files — only compiled object code generated from our own wrapper functions.

This model keeps the use of Boost.Math completely behind a stable libc++ interface, which should minimize the impact on downstreams and retain maximum flexibility for future evolution of libc++.

Licensing

We followed the LLVM Developer Policy and consulted the LLVM Foundation Board, who recommended that we create this RFC. The goal of this RFC is to gather feedback from vendors and downstream consumers of libc++ to gauge whether there is concern around introducing Boost-licensed code into libc++ in the specific way discussed above. Specifically, we would create a narrowly scoped carve-out to permit the use of Boost.Math in libc++. The carve-out would:

  • Apply only to the usage of Boost.Math within libc++ as described above
  • Not apply to any other Boost libraries or other LLVM subprojects

This is intended to be a targeted exception justified by the technical need and the lack of alternatives. In the future, we could potentially explore widening the carve-out, but we want to tackle any such discussion separately to avoid slowing down this specific effort.

Note that we also investigated the possibility of relicensing Boost.Math to be compatible with LLVM’s licensing policy. However, the library has had over 140 contributors across 24 years, and reaching out to all of them to relicense the code is unrealistic for the libc++ team to pursue.

Feedback Requested

We are particularly interested in feedback from packagers and downstream consumers of libc++ regarding packaging and distribution implications or concerns, and any potential blockers to adopting this dependency.

We propose a 6-week feedback window from the date of this post, which should leave enough time for downstreams and vendors to evaluate this RFC and voice concerns. If no major objections are raised during that time, we plan to integrate Boost.Math into libc++ under the licensing carve-out, as explained above.

Thanks,
Louis

7 Likes
2

I am very much in support of this direction. It allows us to finally implement one of the last features of C++17 with relatively little effort. A few things I want to highlight/reiterate here:

  • If there is ever a need for a different implementation (e.g. Boost.Math doesn’t support some platform, Boost.Math isn’t maintained anymore, etc.) there is no problem swapping out the underlying implementation.
  • The MSVC STL also uses Boost.Math to implement the mathematical special functions
  • Even if there is interest in an LLVM-owned implementation, using Boost.Math allows us to implement the feature now. We can still put in the effort to make an LLVM-owned implementation but there is no reason to wait until that happens. We can gradually replace all the Boost.Math code with our own implementation as that implementation matures.
4 Likes
3

Hello, downstream consumer of libc++ here. This is apocalyptic for us. We, and our users would strongly prefer missing C++17 features than a Boost.Math dependency.

If LLVM moves forward with this, we will very likely patch it out.

There are many reasons not to take on an additional dependency. Instead, please grow the implementation slowly without an external dependency. There is no rush to support C++17 Math Special Functions.

4

@andrewrk Could you expand a bit on what problem you have with this downstream?

Again, could you elaborate?

There is no guarantee that this will ever happen, since we don’t have people with the appropriate expertise. If nobody is going to provide resources to actually implement it this won’t grow. It didn’t for almost a decade so far.

5

Thanks for your feedback. Echoing Nikolas’ questions, can you please explain why that would be bad for your project? Did you read the details of how the code would be integrated into libc++? Unless you have some requirements unknown to us (which is partly why this RFC exists), the existence of Boost.Math as an implementation detail of libc++ should not even be noticeable for most.

Just for the record, C++17 Math Special Functions are actually a frequently requested feature. People who do audio, physics or video related code often need these functions.

6

This seems reasonable from a technical standpoint, as long as the use of boost is “fully contained”. As far as the licensing, if LLVM Foundation approves of it, that seems fine by me.

Because libc++ built as a static library doesn’t have symbol visibility control, I think in addition to the things you’ve already mentioned, you’ll also need to re-namespace the boost libraries to be within libcxx’s versioned-namespace as part of the source code import. As long as that’s done (along with a test to guarantee that no unexpected external symbols appear in the built object files), I don’t foresee any issues with this.

Regarding expertise: I think we do have people with the requisite expertise to implement this. Only problem is that they’re busy working on implementing all the other math functions in llvm-libc at the moment. And these new special math functions aren’t at the top of the priority list.

I do expect that eventually these special math functions will be implemented in the llvm-libc math library, in order to get similar bit-accuracy and high performance as we’re seeing from the other math-functions being implemented there. If that does happen, we should switch to using it in libc++ too. But, I agree with philnik: such potential future work does not mean we shouldn’t go forward with this proposal – as long as nobody will object to these functions returning slightly different results in the future.

7

I don’t think we provide any guarantees of this kind currently. We can definitely look into it, but I don’t know what the outcome will be at this point.

Yes, the llvm libc folks have this expertise. As you pointed out, they’re busy with other stuff. What I (and I think also Louis) meant is that we don’t have that expertise in libc++.

8

Even if there’s not an official guarantee, it’s currently true – and is an important property to uphold. Unless boost gets re-namespaced, we’d have libc++ generating symbol conflicts with the “normal” Boost.Math in any program that uses both, which I don’t think is an acceptable outcome.

9

I think we both agree that it’s a problem. However, I don’t think your claim about the status quo is correct. We actually have non-reserved names inside the built library and while most of them are probably inside namespace std, some of them probably aren’t and could clash with top-level symbols in user-defined code. We should arguably be using reserved names even inside the built library.

As far as Boost is concerned, I think we should be able to achieve that by applying an ABI tag (with a pragma) to their functions. We’ll have to investigate a little bit, but I think this is definitely surmountable.

10

We[1] (1, 2) and our users hold the opposite opinion. Libc++ is the last major C++ stdlib that hasn’t fully implemented C++17, and any efforts to close that gap are extremely welcome, because it would finally let people rely on universal availability of these features.

Again I disagree. While there might not be a “rush” (as in cut corners to get something out the door), it’s very important to get C++17 support finished. It’ll still take years for an LLVM version containing that support to be widely available, but the idea of waiting yet more years on top of that horrifies me.

I’d also be interested why it’d have such a big impact (it’s just more C++ code in the end? :thinking:). However, perhaps this concern could be addressed if libc++ provided a build option that toggles whether the math functions are built. That way you can avoid the additional dependency, as long as you don’t use any of the relevant functions.

  1. to the degree that I believe it represents the interests of a wide range of contributors and users across the two mentioned organizations ↩︎

3 Likes
11

We are also downstream users of these mathematical functions to ensure our implementations meet the precision and quality of the standard ones. As @h‑vetinari pointed out, libc++ is the only standard library that has not yet implemented them. We already depend on boost.math to get access to these functions anyway so we are not against the inclusion of boost inside libc++, at least as a first step.

2 Likes
12

I too am curious of the objection. My only real concern was licensing related, and I think this makes me not really concerned:

Boost has no real attribution requirement for binaries, so downstreams that ship Libc++ have no additional requirements, unless you ship source code (at which point… headers in our repo with the license should be sufficient?).

So I guess I don’t see concerns.

1 Like
13

@ldionne will this interact with making these functions constexpr in future?

14

It should not, not any more than if we implemented the functions with our own code in the dylib. If we made those constexpr in the future, I would expect us to have a compiler builtin to do the heavy lifting and simply dispatch to the builtin during constant evaluation, and to the dylib otherwise. Or maybe we could call the builtin at all times, but then that requires Clang to emit or provide (in compiler-rt) an efficient implementation of these functions for use at runtime, which gets us back to our current problem.

So TLDR, this implementation strategy doesn’t affect our ability to make the functions constexpr in the future, but there will be other (likely surmountable) challenges in doing so.

2 Likes
15

Microsoft ran into issues special_math.cpp: Statically linked library contains and depends on Boost symbols · Issue #362 · microsoft/STL · GitHub

The conclusion seems to be that boost math must be designed such that it can be used by standard libraries

In general

  • The symbols must not leak (which you address)
  • The symbols must not conflict with user-imported boost libraries

But in general I agree that reimplementing these functions from scratch is probably not a good use of time!

1 Like
16

As a downstream vendor and users of libc++, (we built packages and distribute it as the downstream), I just want to check

this means boost.math is the same as google test in LLVM repo, right? I mean, as a downstream packager, I don’t need to care anything else. I don’t need to introduce new dependency ourselves, right? Since technically, it means we add a lot of code to LLVM but no more external code. Do I understand right?

17

As long as you build libc++ in the monorepo (which is the only supported thing anyways) there will be no additional dependencies you have to manage. All the generated code will be part of the libc++ dylib.

1 Like
18

Then it sounds good from my position.

1 Like
19

BLAKE3 that we included in our repository adds a prefix to all global linker symbols. But using macros doesn’t seem a solution for Boost, since its headers willl be indirectly #included.

Btw, why isn’t BLAKE3 in the third-party directory?

20

I would disagree with this statement in the future :sneezing_face: