Workaround UB when checking if a double fits in int64_t?

Hello,

I want to check if a double can be losslessly convert to int64_t in a NaN-safe manner. It turns out that this is non-trivial, as casting NaN to integer is UB, and this UB actually fires under clang (https://godbolt.org/z/zje7Wxrrv):

bool check(double d)
{
    int64_t i = static_cast<int64_t>(d);
    return (d == static_cast<double>(i));
}

int main()
{
    if (check(std::numeric_limits<double>::quiet_NaN()))
    {
        printf("1");
    }
    else
    {
        printf("0");
    }
}

(program outputs 1 under -O3, expects 0)

And if I manually add a check for nan, it turns out that Clang cannot optimize it away and generates inferior code (see https://godbolt.org/z/ccGvEoM6M):

bool check(double d)
{
    if (d != d) { return false; }
    int64_t i = static_cast<int64_t>(d);
    return (d == static_cast<double>(i));
}

generates

check(double):                              # @check(double)
        cvttsd2si       rax, xmm0
        cvtsi2sd        xmm1, rax
        xorpd   xmm2, xmm2
        cmpordpd        xmm2, xmm0
        cmpeqpd xmm1, xmm0
        andpd   xmm1, xmm2
        movd    eax, xmm1
        and     al, 1
        ret

while all is needed actually is “cvttsd2si rax, xmm0; cvtsi2sd xmm1, rax”

How can I generate optimal code while still avoiding the UB? Thanks!

1 Like

gcc also can’t get your expect assemble , Compiler Explorer

You are correct that gcc is also unable to generate best code in this case. However, this is unrelated to the fact that on x86-64 those extra checks and branches are unnecessary (due to the semantics of cvttsd2si), and that in the ideal world an ideal compiler should be able to remove these branches for x86-64 targets.