Integer -> Floating point -> Integer cast optimizations

I brought this up in IRC and was told to consult someone who knows more about floating point numbers; I propose an optimization as follows.

Let's say we have an int x, and we cast it to a float and back. Floats have 8 exponent bits and 23 mantissa bits.

If x matches the condition `countTrailingZeros(abs(x)) > (log2(abs(x)) - 23)`, then we can remove the float casts.

So, if we can establish that abs(x) is <= 2**23, we can remove the casts. LLVM does not currently perform that optimization on this C code:

int floatcast(int x) {
    if (abs(x) <= 16777216) { // abs(x) is definitely <= 2**23 and fits into our mantissa cleanly
        float flt = (float)x;
        return (int)flt;
    }
    return x;
}

Things get more interesting when you bring in higher integers and leading zeros. Floating point can't exactly represent integers that don't fit neatly into the mantissa; they have to round to a multiple of some power of 2. For example, integers between 2**23 and 2**24 round to a multiple of 2**1 - meaning that the result has *at least* 1 trailing zero. Integers between 2**24 and 2**25 round to a multiple of 2**2 - with the result having at least 2 trailing zeros. Et cetera. If we can prove that the input to these casts fits in between one of those ranges *and* has at least the correct number of leading zeros, we can eliminate the casts. LLVM does not currently perform this optimization on this C code:

int floatcast(int x) {
    if (16777217 <= abs(x) && abs(x) <= 33554432) { // abs(x) is definitely between 2**23 and 2**24
        float flt = (float)(x / abs(x) * (abs(x) & (UINT32_MAX ^ 2))); // what's being casted to float definitely has at least one trailing zero in its absolute value
        return (int)flt;
    }
    return x;
}

- CL

Yes, this is all correct. What’s the actual question?

– Steve

I'm saying at the IR level, not the C level. IR makes certain assumptions about the representation of floating point numbers. Nothing to do with C, I only used it as an example.

- CL

We already do this to some extent; see this code in InstCombineCasts:

// fpto{s/u}i({u/s}itofp(X)) → X or zext(X) or sext(X) or trunc(X)
// This is safe if the intermediate type has enough bits in its mantissa to
// accurately represent all values of X. For example, this won’t work with
// i64 → float → i64.
Instruction *InstCombiner::FoldItoFPtoI(Instruction &FI) {

—escha

My understanding is that this checks whether the bit width of the integer type fits in the bit width of the mantissa, not the bit width of the integer value.

  • CL

Is this patch sound? https://ghostbin.com/paste/8wt63
I don’t think it is getting triggered.

  • CL

Patches should generally go to llvm-commits and will need test cases. I didn’t glance at this in detail, but the general approach seems reasonable.