RFC: HexFloat floating-point support

Clang Frontend
,
1

RFC: Support for Hexadecimal floating-point

This RFC concerns adding support for (IBM) hexadecimal floating-point, aka HexFloat, or HFP.

Some of the areas touched on here are also considered by the recent “RFC: Decimal floating-point support” (RFC: Decimal floating-point support (ISO/IEC TS 18661-2 and C23)).

What is HexFloat?

HexFloat is a format for encoding floating point numbers. It was first introduced with the IBM/360, and therefore predates the more contemporary and ubiquitous IEEE encoding… See here for an overview: IBM hexadecimal floating-point - Wikipedia of HexFloat.

The primary platform that supports HexFloat is the IBM System/Z running z/OS, and what follows is motivated by this platform. (HexFloat is not supported by Linux on IBM Z.)

User-visible impact and scope of changes

It is important to note that HexFloat affects only how values are represented; the programmer writes code using the standard float, double, and long double` types. Most changes, therefore, will be to the IR and the backend. Compilation units use either HexFloat or IEEE. Users can not mix .o’s compiled to use different modes in the same program.

The only action the programmer needs to take to use HexFloat is to specify a compiler flag to indicate that the standard linguistic types should be represented by HexFloat and appropriate code emitted. We propose the following flag:

-mzos-float-kind=<ieee|hex>

Again, it should be noted that all compile units in a program must be compiled with the same setting for this option, which should also be given in the link step.

This will be recognized only when the appropriate -target / -triple is given.
If the -mzos-float-kind option is not given, the appropriate default for the target will be used.

There are relatively few changes to the front-end, with most changes being restricted to the SystemZ target specific back end. Changes to the mid-levels are largely related to recognizing the new IR types (see below).

Pre-processor

The convention on z/OS is that when in IEEE mode, the __BFP__ (Binary Floating Point) macro is predefined by the compiler. There is no macro that directly indicates that HexFloat mode is active. Thus, to check that HexFloat is active, the idiom is to check that defined(__MVS__) && ! defined(__BFP__).

The values of the floating point limits defined by the compiler (e.g., the __DBL_MAX__ macro) are all updated to have the correct values for the floating point mode. The values in <float.h> will also be updated.

The main uses of the macro are in system/library headers, which use __BFP__ to control which declarations should be seen by the compiler… For the most part, user code will rarely, if ever, need to use this macro.

The APFloat family

A new HexFloat subclass of AFPloatBase will be introduced to APFloat to represent HexFloat values. All the methods necessary to support compile-time evaluation of expressions (e.g., during constant folding) will be implemented. As such, a HexFloat will be useable anywhere an existing APFloat is useable.

New IR types

New types for HexFloat will be added to the IR set of types:

IR type C type
hex_fp32 32-bit HexFloat, C type float
hex_f64 64-bit HexFloat, C type double
hex_fp128 128-bit HexFloat, C type long double

IR Literals

Literals in HexFloat format in the IR will be speciifed by a new 0XS prefix. The variant of HexFloat, i.e., whether the value is hex_fp32, hex_fp64 etc., will be determined by the length of the literal. The literal is the hexadecimal representation of the value in HexFloat format, and therefore encodes the sign, (biased) exponent and significand in their respective positions.

Example C → IR

This example shows the main elements discussed so far. The C code is conventional. The IR shows the new IR types, and the encoding of the float literal 2.0f. Note also that the standard fadd operation is used. The back-end will lower this to correct instruction for HexFloat.

    float plus2(float arg) {}
      return arg + 2.0f;
    }


    define hidden hex_fp32 @plus2(hex_fp32 noundef %arg) {
    entry:
      %arg.addr = alloca hex_fp32, align 4
      store hex_fp32 %arg, ptr %arg.addr, align 4
      %0 = load hex_fp32, ptr %arg.addr, align 4
      %add = fadd hex_fp32 %0, 0xS41200000
      ret hex_fp32 %add
    }

Compiler runtime

There are various routines in compiler-rt which manipulate floating point values, and which are dependent on the format of the floating point value. Examples include conversion to/from integer values. New variants to work with HexFloat will be provided. In the vast majority of cases the new variants are just wrappers that compile the existing code, but under a new name.

C++

Mangling is unaffected as the C++ types are unchanged. As far as the user code is concerned, the types are the standard types (float, double, etc.); the representational choice is independent of the language type encoded in the mangling. As a program must be entirely either HexFloat or IEEE, there is no need to encode the representational format.

Both HexFloat and IEEE will use the same typeinfo objects, which, again, is not problematic because programs are entirely HexFloat or IEEE. RTTI and exception handling, therefore, will work seamlessly.

libcxx and libcxxabi

A separate instance of the C++ library (libcxx) will need to be built for HexFloat. Although not identical, this is not altogether unlike supporting multiple instances for 32-bit and 64-bit modes.

libcxxabi will be shared between IEEE and HexFloat.

Some code in the library is sensitive to the floating format encoding. Where necessary, alternative implementations for HexFloat will be provided. Sensitive parts of the code may need to be guarded with the pre-processor macros described above so that the correct parts are included for the compilation units.

Examples of the types of changes that will be necessary include updating numeric_limits<> to have the correct values for HexFloat. Similarly, std::format will need to be modified to handle HexFloat. Again, it should be emphasized that in any one instance of the C++ library only one of the IEEE/HexFloats variants will be active.

DWARF

HexFloat values will tagged with the vendor specific type tag used in existing compilers.

The tags are:

  0xde    IBM_complex_float_hex
  0xdf    IBM_float_hex

As noted above, these tags are already in use by existing tools to describe HexFloat. Note also, that unlike with mangling, a debugger does need to know the representational format of the data.

2 Likes
2

Is there a plan how we can help users identify floating point format mismatch? Maybe there are arguments why such mismatch should be considered impossible, but I don’t see them myself, and don’t see them in an RFC. If we don’t help users, they will have “fun” time debugging this.

3

Thanks, Ariel! This looks great!

In the table in the New IR types section, I think that hex_f64 should be hex_fp64.

I was wondering where the 0xS prefix proposed for IR literals comes from, but it looks like S is just the next character in the sequence noted in the comments of LLLexer::Lex0x.

Inferring the HexFloat variant based on the length of the IR literal seems a bit novel. The existing function (LLLexer::Lex0x, LLLexer::LexDigitOrNegative, and LLLexer::LexPositive) produce an APFloat::IEEEDouble regardless of the length. Does the length include leading 0s?

4

I presume this will preserve consistency with the xlC compiler? IEEE will be the default for 64-bit and hex will be the default for 31-bit?

5

Actually I expect no changes are required for format. From the top of my head I do expect changes to to_chars (and from_chars), the stream operators (<< and >>), operator<=>, and comparison objects, and generate_canonical.

6

It is important to note that HexFloat affects only how values are represented; the programmer writes code using the standard float, double, and long double types. Most changes, therefore, will be to the IR and the backend. Compilation units use either HexFloat or IEEE. Users can not mix .o’s compiled to use different modes in the same program.

C23 adds _Float32, _Float64, etc. types (see Annex H) that are guaranteed to be the standard IEEE 754 floating-point types. Clang doesn’t support these types yet, but would it make sense to support _Float32 for IEEE 754 binary43 and hex-float float in the same translation unit? Conversely, in __BFP__ mode, is there utility for an extra floating-point type to represent hexfloat types?

[What this is really getting at is, in hardware, is it possible to have fadd float and fadd hex_fp32 in the same program without needing to mode-switch any control registers, and if so, does it make sense to expose some way to write such code in C/C++?]

Again, it should be noted that all compile units in a program must be compiled with the same setting for this option, which should also be given in the link step.

What effect does this option have in the link step?

hex_fp32
This is really getting into bikeshedding about names, but the existing standard for the processor-specific special types is to include the processor name. In that sense, the names should probably be more like ibm_fp32 or maybe ibm_hfp32.

7

SystemZ has different instructions for BFP, HFP, and DFP. They can be mixed together in a single function with no restrictions. IBM C probably allows this, but I can’t put my finger on the documentation this minute to be sure.

8

I don’t believe it does. My understanding, per the XL /C/C++ User’s Guide, is that the FLOAT(HEX) option (the default for ILP32) specifies that the HFP instructions are used for floating-point types while the FLOAT(IEEE) option (the default for LP64 and METAL) specifies that the BFP instructions are used. If alternate spellings/keywords for HFP/BFP types were available, I would expect them to be documented in the “Keywords” section of chapter 2 of the XL C/C++ Language Reference. It could be that such an extension is available but undocumented. I don’t work for IBM and don’t have access to any z/OS systems to explore further.

9

A compilation unit is either IEEE or HexFloat. Mismatches could only occur during linking. The binder/linker does not enforce any constraints that the all modules must have been built in the same mode.

Note also that the compilation flags field in the per-compilation unit PPA2 has a bit which indicates whether the module was compiled for IEEE or HexFloat.

In practice this is not a problem.

10

That is a good question. The XL compiler has been generating code like this for 30+ yrs and I’ve never seen a problem related to this. My theory on this is people just use the default representation and never come across the problem. The object files do have a flag to say if they were compiled with hex or IEEE. The binder could check this flag but since it is object file level you might have false failures (.o files with no floating point type have t be hexfloat or ieee).

11

Yes, that was a typo on my part.

Yes. The number of hexits includes the leading zeros. In the vast majority of cases there will be no leading zeros. (To have a leading zero, the value would need to be positive, and have an exponent less than -49 in base 16.)

12

Yes, the general idea is that the default floating mode is the default for that programming model: HexFloat for 31-bit, IEEE for 64-bit.

13

Thank you for this. The main point, though, is that there may be changes required to some of the C++ library. These will be taken care of.

14

This is interesting. If _Float32 etc do get mapped to the IR float etc, then it will be possible to have instances of both representations in the same translation unit. This RFC is primarily focused on how the basic C/C++ types are handled.

However, the normal usage model is that a program will use one representation or the other.

That would depend on the specific machine. In the immediate case, System/Z, there are different instructions for manipulating HexFloat, Binary Floating Point (IEEE), and Decimal Floating point.

If a machine does need to switch mode to handle the different types (I assume what you’re getting at is that the machine has one instruction for, say, floating point add, and whether this does a HexFloat or IEEE addition depends on the mode), switching mode could be wrapped in a library call. If that’s too heavyweight, an intrinsic or instruction could be added.

Note, however, the expected usage is for programs to use no more than one of HexFloat or
IEEE.

None.

15

My understanding is Float32 etc will be unique types in C. They will have their own set of library functions (eg. sinf32()), own printf/scanf specifiers (TBA in upcoming meetings), etc. The one thing that I’d like to understand is if the aliasing rules allow a float* to point to a Float32. I don’t that is allowed.

Given that Float32 is a unique type, there will be no problem mixing this with float being hex or IEEE. This new Float32 type would be as distinct in the language level as the decimal floating point types or even the integer types to the existing float types.

16

My understanding is Float32 etc will be unique types in C. They will have their own set of library functions (eg. sinf32()), own printf/scanf specifiers (TBA in upcoming meetings), etc. The one thing that I’d like to understand is if the aliasing rules allow a float* to point to a Float32. I don’t that is allowed.

As far as C is concerned, H.2.1p2 explicitly says

Each interchange floating type is not compatible with any other type.

(Type compatibility being the main prong of strict aliasing rules.)

17

HexFloat is a not-necessarily-normalized floating point representation. So, for example::

0x8020000 and 0x81020000 represent the same value the former being normalized
the later being unnormalized by 1 hex digit (hexFloat32)

One can use the exponent and non-normalized fraction to play tricks in the arithmetic
that are not possible in normalized arithmetic. For example one can add 0x96000000
(zero with an exponent that aligns the LoB with the binary point) and AINT() the value
{Fortran}.

My point is that it HexFloat also interacts with calculations in well understood ways;
which is beyond that of simply being a different representation. Programmers familiar with
HexFloat will be well aware of this, although LLVM compiler writers are unlikely to be fully
versed on the strange things non-normalized FP arithmetic can do.

18

HexFloat is a not-necessarily-normalized floating point representation. So, for example::

0x8020000 and 0x81020000 represent the same value the former being normalized the later being unnormalized by 1 hex digit (hexFloat32)

For the sake of clarity, is this different values like IEEE 754 decimal floating-point’s cohort and preferred exponent stuff, or is this different values like noncanonical values?

19

The first number (0x8020000) has just 7 digits present. Is that the intended number? Perhaps 0x80200000 was intended?

20

I see. I misunderstood the description of IR literals in the initial post.

This implies that the number of trailing zeros may then differentiate the types. For example:

0xS0.        // hex_fp32 with value 0? Or error due to no fraction bits?
0xS00.       // hex_fp32 with value 0? Or error due to no fraction bits?
0xS000.      // hex_fp32 with value 0?
...
0xS0000000.  // hex_fp32 with value 0?
0xS00000000. // hex_fp32 with value 0?
0xS000000000.        // hex_fp64 with value 0?
...
0xS000000000000000.  // hex_fp64 with value 0?
0xS0000000000000000. // hex_fp64 with value 0?
0xS00000000000000000.                // hex_fp128 with value 0?
...
0xS0000000000000000000000000000000.  // hex_fp128 with value 0?
0xS00000000000000000000000000000000. // hex_fp128 with value 0?

Since the exponent width is the same for all three formats, I guess trailing zeros can be ignored such that the format is inferred based solely on the significant digits?

What is the behavior for the unused part of the low-order part of hex_fp218? Are those required to be 0? Ignored? Preserved?

This approach differs from the behavior for binary FP literals (with prefix 0x<xdigit...>) where the type is always an IEEE double regardless of the number of hex digits. It looks like support for IEEE quad requires a 0xL prefix, IEEE half requires a 0xH prefix, and there is no syntax for a IEEE single.

I’m inclined towards either having three distinct prefixes or, perhaps distinct prefixes for hex_fp64 and hex_fp128 and no syntax for hex_fp32 (consistent with binary FP). It looks like differentiation is needed for hex_fp128 if implementation is intended to reuse HexIntToVal() and HexToIntPair().