[RFC] Having different behavior for __fp16 on RISC-V

Clang Frontend
#1

RISC-V has a draft extension for half-precision proposed last year[1],
so we think adding a new type for that would be great to make this
easier to use that extension.

We found there is _Float16 and __fp16 types are supported on GCC and
Clang, but _Float16 has C++ supporting issues on GCC site, so we think
support both types is a reasonable choice to us.

However we would like have slight different behavior for __fp16 other
than ACLE: The evaluation format of __fp16 set same as _Float16,
which means no promotion are performed if there is no hardware half-precision
supported.

The only concern to us is it’s defined differnt with clang’s document,
so we would put this RFC patch here, to make sure it’s OK for make
__fp16 has differnt behavior between differnt targets.

This patch contains document change only, implementation would be in
other patches.

[1] https://github.com/riscv/riscv-isa-manual/pull/496
[2] https://github.com/riscv/riscv-elf-psabi-doc/pull/172

Corresponding phabricator entry for this RFC:

https://reviews.llvm.org/D98012

#2

RISC-V has a draft extension for half-precision proposed last year[1],

so we think adding a new type for that would be great to make this
easier to use that extension.

We found there is _Float16 and __fp16 types are supported on GCC and
Clang, but _Float16 has C++ supporting issues on GCC site, so we think
support both types is a reasonable choice to us.

However we would like have slight different behavior for __fp16 other
than ACLE: The evaluation format of __fp16 set same as _Float16,
which means no promotion are performed if there is no hardware half-precision
supported.

The only concern to us is it’s defined differnt with clang’s document,
so we would put this RFC patch here, to make sure it’s OK for make
__fp16 has differnt behavior between differnt targets.

This patch contains document change only, implementation would be in
other patches.

[1] https://github.com/riscv/riscv-isa-manual/pull/496
[2] https://github.com/riscv/riscv-elf-psabi-doc/pull/172

Corresponding phabricator entry for this RFC:

https://reviews.llvm.org/D98012

#3

Sorry, I noticed the patch first and left a comment there.
But repeating what I wrote there: you’re redefining an existing type to behave as another type, but only for some targets. I think this is problematic and not going to help anyone. Your motivation is that the type you would like to use is not supported by GCC, but that seems like the proper fix to me.

Cheers,
Sjoerd.

#4

And to add one more thought to this, I am actually working on deprecating __fp16 because there should be no good use case for it anymore with _Float16. This means that we will deprecate the type in the ACLE, it should give a warning when it is used, and eventually we would like to remove it in favour of _Float16. It also means that GCC would have to support _Float16. Deprecating __fp16 is work in progress but just wanted to illustrate the direction of travel.

Cheers,
Sjoerd.

#5

Hi Sjoerd:
Thanks your reply, and apologize that I didn’t tell the full story behind this RFC:

  • Ideally the short float is one of candidates of half precision types, but it seems far from
    the way to standardize for including that in C or C++ std, so we didn’t go this way.

  • There is reason why we don’t using _Float16 as RISC-V’s proposal

  • _Float16 is belong with ISO/IEC TS 18661-3, which might included in C2X, but
    not belong any C++ std yet, and the spec also didn’t mention anything about C++ part,
    however ISO/IEC TS 18661-3 also contain other interchange types and extended types
    like _Float32_t, _Float32x_t, _Float64_t, _Float64x_t…and there is NO C++ document for
    mangling rule of those types, so we would like to prevent to made this global
    change in GCC side.

  • Default promotion rule is also one of our concerns, _Float16_t won’t be promoted to double,
    which is inconvenient for users, esp. for printf.

  • The reason why we choose __fp16 as the type name:

  • One major reason is __fp16 is widely used in this world rather than _Float16.

  • Why do we decide to have different behavior?

  • We would like to default fully utilized hardware capability - if there is half-precision
    then use it as possible.

  • We found some interesting things on __fp16 code gen behavior - native half-precision
    instruction still might be generated when optimization is enabled, so why not
    generated by default?

Thanks.

#6

Hello Kito,

Let me also first backup a little bit by saying that we want the same thing: a half-type that maps efficiently to modern hardware. But by definition, __fp16 isn’t that type.

There are different alternatives here: try changing the semantics of an existing type, or adoption of a proper half type. Your proposal is problematic on different levels: i) changing the behaviour of an existing type, and ii) only do that for some targets. Your arguments that __fp16 is widely used and is thus convenient is problematic for i) because of the change in behaviour/expectations, and ii) a type that behaves differently on different targets will be a source of confusion and non-portability. Thus, I don’t see how your narrow proposal to change the type for only some targets is going to be more acceptable than e.g. changing it for all targets. We actually have discussed changing the semantics of __16 recently:
https://lists.llvm.org/pipermail/cfe-dev/2020-December/067343.html

There you’ll see some pros/cons of doing this. But changing semantics is always going to have cons that will be awkward and there isn’t going to be a really solution. That’s why we are now taking the approach to deprecate __fp16.

To briefly comment on some your remarks:

  • You argue that there is a “standardisation gap” for _Float16, but that’s why it is interesting that you end up advocating for __fp16, for which that gap is even bigger as it is a non-standard type.
  • In Clang, _Float16 is defined in both C and C++ mode for the targets that support this type.
  • C++ mangling is defined for _Float16. It isn’t indeed for the other interchange types. But I don’t see that as a fundamental problem.
  • I don’t think your codegen example is representative: this is 1 addition that indeed avoids the up and down conversions. And while for some cases optimisations are possible, if you start looking at some bigger code examples that becomes more challenging, if possible at all in all cases, and it is all work and thing we can or should avoid.

The way forward is a native half type, which is _Float16. And what we need to work on is support for it in GCC, which has to happen sooner or later I guess. And yes, there might also be some more standardisation/implementation work to do for _Float16.

Cheers,
Sjoerd.

#7

Hi Sjoerd:

Hello Kito,

Let me also first backup a little bit by saying that we want the same thing: a half-type that maps efficiently to modern hardware. But by definition, __fp16 isn’t that type.

Yes, I believe we are on the same page for this.

There are different alternatives here: try changing the semantics of an existing type, or adoption of a proper half type. Your proposal is problematic on different levels: i) changing the behaviour of an existing type, and ii) only do that for some targets. Your arguments that __fp16 is widely used and is thus convenient is problematic for i) because of the change in behaviour/expectations, and ii) a type that behaves differently on different targets will be a source of confusion and non-portability. Thus, I don’t see how your narrow proposal to change the type for only some targets is going to be more acceptable than e.g. changing it for all targets. We actually have discussed changing the semantics of __16 recently:
https://lists.llvm.org/pipermail/cfe-dev/2020-December/067343.html

Thanks for providing this link, it’s a really great conversation there, one major reason I purpose for changing the behavior of some target (or more specific, RISC-V only at this moment) is because I know there is ACLE spec for ARM and AArch64, I think it’s not reasonable to change the behavior of all other target especially for ARM/AArch64.

So what about if add a new command-line option, I saw Richard Sandiford has mentioned this but seems no follow up discussion for this point.

#8

Hi Kito,

So what about if add a new command-line option, I saw Richard Sandiford has mentioned this but seems no follow up discussion for this point.

Perhaps I shouldn’t have mentioned this, but I believe this options is already there. :slightly_smiling_face:I think with cc1 option -fnative-half-type you’ll get the behaviour that you want. But having said this, my vote is against documenting and defining that this is how __fp16 works. This results in non-portable and ACLE breaking code.

And quoting from the original RFC, I want to reiterate that this looks to me like a very bad motivation to change __fp16 over here in Clang:

but_Float16has C++ supporting issues on GCC site

My alternative proposal is to fix this in GCC. This is something we want or need to fix anyway at some point, so I am guessing my Arm colleagues working on GCC would be happy to help here.

Cheers,
Sjoerd.

#9

Hi Sjoerd:

Thanks for your reply again :slight_smile:

So what about if add a new command-line option, I saw Richard Sandiford has mentioned this but seems no follow up discussion for this point.

Perhaps I shouldn’t have mentioned this, but I believe this options is already there. :slightly_smiling_face:I think with cc1 option -fnative-half-type you’ll get the behaviour that you want. But having said this, my vote is against documenting and defining that this is how __fp16 works. This results in non-portable and ACLE breaking code.

Yeah, I know the cc1 option, it can be passed by -Xclang -fnative-half-type, and we can hide this into the clang driver, but it kind of semantic change of __fp16 for RISC-V target, we would like to add a non-cc1 option -m or -f option there rather than a cc1 option here.

And I can fully understand breaking ACLE is not reasonable, I think if some guy wants to break that, the guys must come from ARM not a RISC-V guys :stuck_out_tongue:

And quoting from the original RFC, I want to reiterate that this looks to me like a very bad motivation to change __fp16 over here in Clang:

but_Float16has C++ supporting issues on GCC site

My alternative proposal is to fix this in GCC. This is something we want or need to fix anyway at some point, so I am guessing my Arm colleagues working on GCC would be happy to help here.

In fact there is kind of awkward story behind this, we chose __fp16 rather than _Float16, major reason is GCC supporting, and also we saw ARM/AArch64 deprecated that for a while, but _Float16 problem still there in GCC, so we decide to using __fp16 rather than _Float16.

And we also considering the compatible issue there, I mean the compatibility of ACLE’s __fp16, __fp16 isn’t a language extension for GCC, it’s a target specific type for ARM/AArch64, so if we don’t define that as a formal extension types, then we have broken support/compiler compatibility issue on this due to GCC didn’t support __fp16 as language extension.

RISC-V has no backward compatibility issue there so we are trying to take both compatibility and hw supporting utilization from __fp16.

#10

Was a bit confused about this message (duplicate of the first mail?), but I guess you wanted to ping the thread?
I will reply to your last mail in the original thread, which I indeed didn’t do yet.

#11

In fact there is kind of awkward story behind this, we chose __fp16 rather than _Float16, major reason is GCC supporting, and also we saw ARM/AArch64 deprecated that for a while, but _Float16 problem still there in GCC, so we decide to using __fp16 rather than _Float16.

I am not really following this.

And we also considering the compatible issue there, I mean the compatibility of ACLE’s __fp16, __fp16 isn’t a language extension for GCC, it’s a target specific type for ARM/AArch64, so if we don’t define that as a formal extension types, then we have broken support/compiler compatibility issue on this due to GCC didn’t support __fp16 as language extension.

And am not entirely sure about this.

I don’t think I have changed my mind on this topic, and I don’t know what else/more I can add to what I have already mentioned.
I won’t repeat my arguments, but the bottom line is that we want to deprecate __fp16, while you like to rebrand it to a native half-precision type, and I don’t see how this goes together very well. I also remain unconvinced about the arguments against _Float16, which I think is the solution.

I don’t know what your options are now. Maybe someone else can weigh in and shine a light on this, perhaps I am missing something. I don’t know if you can go ahead and make these changes if you get an LGTM from someone else, but I hope the community sees the problem of the same type behaving different on different targets.

Cheers,
Sjoerd.

#12

I don’t know why _Float16 isn’t implemented by GCC in C++ mode, but I’d agree that using _Float16 for RISC-V, and adding support for _Float16 to GCC’s C++ mode does seem like the better solution here, especially given that such support will also be needed for ARM.

#13

A thread has been started on the gcc mailing list. https://gcc.gnu.org/pipermail/gcc/2021-March/234962.html

#14

A thread has been started on the gcc mailing list. https://gcc.gnu.org/pipermail/gcc/2021-March/234962.html

#15

Hi Sjoerd:

Was a bit confused about this message (duplicate of the first mail?), but I guess you wanted to ping the thread?
I will reply to your last mail in the original thread, which I indeed didn’t do yet.

Apologize for the confusion, I didn’t subscribe to this mailing list for this email account when I sent the first mail, so my mail got blocked*, and then I sent again after I subscribed, and I guess why you see it twice is because it got approved…

  • message like this: Your message to cfe-dev awaits moderator approval
#16

Hi Craig and Kito,

Thanks for starting the GCC thread. I have added my thoughts there too. Looks like the GNU community is open about enabling _Float16, so I think that this is the approach that we should take.

Craig contacted me about different behaviour between GCC and Clang when FP16 isn’t enabled, for example: https://godbolt.org/z/vx6rnG. My colleague reminded me that this is related to FLT_EVAL_METHOD. With FLT_EVAL_METHOD=0, smaller types and expressions are evaluated in single-precision without the need for intermediate rounding. So, GCC is right here, and Clang is wrong. With FP16 enabled, we see that GCC switches to FLT_EVAL_METHOD=16, where as in Clang we still use 0 which is wrong: https://godbolt.org/z/j3KhGa.

I wanted to raise a ticket for this, but noticed that you already did: https://bugs.llvm.org/show_bug.cgi?id=43475. I will add some information to that a bit later.

Cheers,
Sjoerd.