[FP] Constant folding of floating-point operations

[arsenm]

@andykaylor

I think this behavior is implied by the existence of the ‘afn’ fast-math flag. That this flag grants permission to return an approximate result implies that the absence of the flag must require that the result not be approximated. To me that means that for a given input value, all calls to the function will return the same result. You can’t not approximate something that doesn’t return a consistent value.

But all results are approximate, so this isn’t particular clear what it means. The best you can hope for is correctly rounded, which is still an approximation and provided by very few real libm implementations for any of the difficult functions.

@andykaylor

I was somewhat dismayed by the claim in issue 62479 that the C and C++ language standards don’t require that all calls to math library functions will return the same result for a given input. I can’t find any such requirement.

There certainly is no requirement. LLVM and C, C++ barely have defined floating point requirements, much less values for specific functions. OpenCL at least gives ulp tolerances (which also do not need to be consistent)

@andykaylor

my program may produce different numeric results depending on whether or not the compiler is able to deduce that the argument is constant.

I don’t see this as an issue. There’s a broader question of what constitutes the “result” of the operation. In the presence of any kind of folding combine (e.g. any use of the contract or reassoc flag), the apparent result can change depending on the use context beyond constant folding

@andykaylor

During the discussion in issue 62479, there was an emphasis on the impact of constant folding for cross-compilation. However, the problem isn’t necessarily limited to cross-compilation. It can happen any time the program being compiled is linked against a different math library than the compiler itself was linked against.

Kind of the same thing, and I do think this is a problem. It’s been a long standing aspirational issue to have consistent constant folding (which then doesn’t necessarily match the target library behavior), just internally consistent in the compiler. Ideally we would just use MPFR or equivalent for this.

@andykaylor

However, the LLVM constant folder will evaluate this operation as a fused multiply and add even if I’m compiling for a target that doesn’t support fused operations. So, once again, the result can depend on earlier optimization decisions made by the compiler. What’s worse (to me at least) is that if I compile with -ffp-contract=fast instead of -ffp-contract=on, the constant folder doesn’t use fused operations, even if the target does support them.

This is definitely fine. The point is produce whatever is fastest, and constant fold as fma is perfectly fast (and also the better result). “If the target supports them” is meaningless. The compiler could always choose to implement fma in software to implement fmuladd. This is defined operationally and should not differ based on target codegen preference. If the target had some pipelining issue it could resolve by selectively emitting separate fmul+add or an fma, that should also be fine. If you want a specific behavior, you can choose to not use fmuladd.

@andykaylor

My basic proposal here is that the LLVM IR language definition should assume that all floating point operations, including math library functions and math intrinsics, should produce consistent and reproducible results that are independent of compiler optimizations unless fast-math flags permit otherwise.

Practically speaking this proposal is “do no constant folding” which will not go over well. I would love if we had perfect host independent constant folding but that’s a lot of work nobody has seriously thought about undertaking