Motivation
Consider the following transformation from [InstCombine] ninf should not be propagated · Issue #136430 · llvm/llvm-project · GitHub :
Alive2: Compiler Explorer
define half @test_fcmp_select_maxnum(half %x) {
#0:
%cmp2 = fcmp ogt half %x, 0x5bf8
%sel2 = select nnan ninf nsz i1 %cmp2, half %x, half 0x5bf8
ret half %sel2
}
=>
define half @test_fcmp_select_maxnum(half %x) {
#0:
%sel2 = fmax ninf nsz half %x, 0x5bf8
ret half %sel2
}
Transformation doesn't verify!
ERROR: Target is more poisonous than source
Example:
half %x = #xfc00 (-oo)
Source:
i1 %cmp2 = #x0 (0)
half %sel2 = #x5bf8 (255)
Target:
half %sel2 = poison
Source value: #x5bf8 (255)
Target value: poison
Summary:
0 correct transformations
1 incorrect transformations
0 failed-to-prove transformations
0 Alive2 errors
Alive2 and llubi only apply the ninf flag to the result of select instruction. But the current LangRef wording does not specify this behavior. And some optimizations in LLVM assumes that the ninf flag is applied to both the arm of select: [InstCombine] Fix ninf propagation for fcmp+sel -> minmax by dtcxzyw · Pull Request #136433 · llvm/llvm-project · GitHub
For clarity, let’s see the following example:
entry:
br i1 true, label %if.end, label %if.else
if.else:
br label %if.end
if.end:
%x = phi nnan float [ poison, %if.else ], [ %nnan_val, %entry ]
->
%x = select nnan i1 true, float %nnan_val, float poison ->
%x = select nnan i1 true, float %nnan_val, float nan ->
%x = poison (If we apply the nnan flag to both arms)
In some cases, one of the arms of a select may never be selected. If we apply the nnan flag to both the inputs, we cannot preserve the nnan flag in the phi -> select conversion. BTW, it also violates the guarantee that a select doesn’t propagate poison from the value not being selected.
Proposed Change to the LangRef Wording
The LangRef should explicitly state that the poison-generating flags on select instructions are only applied to the final result.
Related comments: [InstCombine] Fix ninf propagation for fcmp+sel -> minmax by dtcxzyw · Pull Request #136433 · llvm/llvm-project · GitHub