This has now been committed to the Itanium CXX ABI: ::= DF16b # C++23 std::bfloat16_t
So if we don’t introduce an extra type, we effectively already agreed on the type mangling.
However at Arm, this is what we anticipated, and we internally agreed that for Arm architectures we would deviate from this by keeping the old mangling name. __bf16 is already available in libraries that are used by our customers, and we’d rather not cause potential issues. The deviation is quite minor, so we can live with that.
Changing a type that forbids arithmetic to an arithmetic type is technically source-breaking in C++ — it can be detected by SFINAE — but the practical compatibility risk is low. If the architecture vendors are comfortable changing the behavior of __bf16, I think Clang should go along with it, and in that case there’s no reason to treat them as formally different types with different manglings. We should use the new mangling when introducing a new type on an architecture, but ARM’s request to continue using the vendor-extension mangling on ARM seems reasonable.
The implementation options for the actual arithmetic are basically the same as they were for _Float16: we can use native support on architectures that provide it, and otherwise we need to promote/demote to (presumably) float. When we’re doing the latter, we should probably use excess-precision arithmetic by default, but we can provide controls to disable that and force immediate promotion/demotion around individual operations.
Thank you for your valuable insights on this topic. Given your explanation, I’m considering adapting __bf16 to act as an arithmetic type in the context of C++23, as the semantics for this are well defined in the proposal. This would be applicable for hardware architectures that natively support bfloat16 arithmetic operations. In the event of a later decision to apply this change universally across all scenarios, I believe it could be handled in a separate patch.
I wanted to get your thoughts on this approach. Does it sound reasonable and in line with your understanding of the language and hardware constraints? Your guidance is greatly appreciated.
Are you suggesting you’d just make __bf16 arithmetic in the C++23 language mode? That seems unnecessary; it sounds like the architecture vendors are fine with it becoming unconditionally arithmetic.
Thank you for your prompt response. I understand your point about making __bf16 unconditionally arithmetic, but I do have some concerns. Specifically, I’m unsure about the semantics outside of C++23. For example, I’m uncertain about the expected behavior in binary arithmetic operations involving _Float16 and __bf16 in languages like C23 or earlier versions of C++.
Furthermore, my current implementation does not include any compiler-RT support to handle scenarios where the hardware does not natively support bfloat16. So, I was considering a more conservative approach initially, focusing on C++23 and hardware that natively supports bfloat16.
I’m eager to learn and navigate these complexities, and would appreciate any further guidance you might have on this matter.
Making __bf16 only conditionally arithmetic, but having the condition being hardware support, seems much more reasonable as an intermediate step than having the condition being C++23. We would then have to make sure not to make std::bfloat16_t available in C++23 when hardware support was not available.
Longer-term, I think the right way to implement this without hardware support is to promote to float. I believe there’s generic backend code for that for float16 which shouldn’t be hard to extend to another type. Operation-by-operation promotion/truncation is expensive, of course. Excess precision arithmetic helps a lot with that, and we have code for that in Clang, again for float16, which was written to be easily applied to different types.
Your insights have been incredibly helpful, and I appreciate your thoughtful guidance. I agree, making __bf16 arithmetic based on hardware support seems more reasonable than basing it on C++23.
I agree with your long-term solution for architectures that lack native bfloat16 support. The idea of promoting to float in these cases appears to be a practical and efficient approach. Your guidance is greatly appreciated. Thank you.
In response to @rjmccall’s suggestion of promoting __bf16 to float without hardware support, I am contemplating the implementation of excess precision arithmetic for bfloat16, leveraging existing support for Float16.
Would this approach, in your opinion, address the compiler-rt support concerns for hardware lacking native bfloat16 support?
I’ve not been keeping up on the conversation here very well (or on the review). If you can get the semantics to work properly between them (I thought math worked differently on them?), I would expect that to be acceptable. It would have to be well documented however.
To be clear, __bf16 is a 16-bit type with 8 bits of precision and 8 bits for the exponent. _Float16, half (AKA __fp16) are 16-bit types with 11 bits of precision and 5 bits for the exponent. Promoting these types to another type for arithmetic purposes requires a type that exceeds those requirements (as is the case for float when the IEEE binary32 format is used). I therefore assume you mean leveraging the existing support present for promotion in general (as opposed to promoting to/through _Float16).
@tahonermann, thank you for your input. You’re correct, I’m considering leveraging the general promotion support, particularly to float, not specifically through _Float16.
I’m aware that my current changes are somewhat extensive, and I’m eager to streamline the review process if possible. I’d appreciate your thoughts on the following two approaches:
Maintain the current change, which enables arithmetic on __bf16 when hardware support is available and implements the C++23 extended floating types (_Float16 and __bf16). A subsequent patch could introduce excess precision support for __bf16 on hardware lacking native support.
Separate out the arithmetic enablement for __bf16 into its own patch series. This could be further split into:
a) First patch enabling arithmetic on __bf16 when hardware support is available, and
b) Second patch implementing excess precision support for __bf16 when hardware support is lacking. Either approach would likely necessitate a flag in the target class to determine the level of bfloat16 support.
We could also combine 2a and 2b into one patch if it makes sense.
Regarding excess precision support, it promotes types like Float16 to float for usual arithmetic operations, followed by downcast in the end. This might not match exactly with how compiler-rt implements the type specification/semantics, i.e results may not reproduceable with excess precision support, edge cases may not be handled correctly, etc. But, precision for bfloat16 might not be very important for machine learning tasks, so using excess precision might be ok for now.
If I’m wrong or if there’s anything else I should consider, please let me know. Thanks again for your help.
I prefer this approach. It doesn’t matter to me if that patch is also split into the 2a and 2b parts.
I don’t feel sufficiently well informed to comment on this. I briefly reviewed P1467R9, but it isn’t clear to me whether an extended floating point type that computes values using excess precision is permissible as a type named by std::bfloat16_t. We should try to get a definitive answer to that.
@tahonermann It seems doing std::bfloat16_t arithmetic in 32-bit float might be ok as per one of the authors of P1467 - David Olsen: “My intention as one of the authors of P1467 is that compilers can implement std::bfloat16_t by doing all arithmetic in 32-bit float. What’s important is that code using std::bfloat16_t compiles and gets correct answers (for a reasonable definition of correct).”