[RFC][mlir][Linalg][Vector] Fast-math flags on Linalg and Vector ops

Summary

Linalg and Vector carry fast-math flags in two ways:

  • Linalg has them on the scalar arith and math ops inside a region body (linalg.generic, linalg.map, linalg.reduce). Named ops such as linalg.matmul and linalg.contract have no flag attribute.
  • Vector has them on three ops that each stand for many scalar operations: vector.reduction, vector.multi_reduction and vector.contract.

There is no written rule for how Linalg and Vector transforms treat these flags. Each transform decides on its own. Some lose the flags. Some insert values that the flags do not allow.

There is also an unwritten rule. Today, several Linalg and Vector transforms reorder float reductions and fuse multiply-adds without any flag (section 2.3). LLVM needs reassoc or contract for the same changes. No Linalg or Vector doc says that this is allowed.

This RFC proposes, for Linalg and Vector together:

  1. One meaning for a flag. A flag applies to every scalar operation that its op stands for. For a Linalg body op, that is the op itself, as in arith. For a Vector reduction or contraction, that is every operation of its scalarized form. vector.multi_reduction already says this.
  2. A decision on order and fusion. We recommend writing down what the transforms do today: reductions with an associative combiner may be computed in any order, and contractions may fuse their multiply and add, as LLVM’s llvm.fmuladd may. Section 3.2 compares this with following LLVM.
  3. Five rules for rewrites, the main part of this RFC. They follow the LangRef and LLVM’s code (section 3.3, with code and test references).
  4. Per-op control, as in LLVM IR. A frontend that has per-function or per-translation-unit (per-TU) options writes them as per-op flags, as Clang and Flang do (section 3.4).
  5. Shared helpers and an opt-in guard that carry the flags through splits and rebuilds. Merges, drops and made-up values still need a decision in each pattern (section 3.5).
  6. Fixes for the Linalg and Vector places that break these rules (section 4).

This RFC does not propose to remove the flags, or to replace them with pipeline options.

The tracking issue is #240110, under the Vector meta-issue #235708. It asks three questions. This RFC focuses on the second one: how flags compose when a rewrite splits, merges or drops ops, or makes up a value (section 3.3). That is where the rules are missing today, and where we most need feedback. The other two questions get short answers:

  • Per-op or per-function control: per op (section 3.4, details in appendix C).
  • A mechanism instead of code in every pattern: partly, with shared helpers and an opt-in guard (section 3.5, details in appendix D).

Until this RFC is agreed, only fixes that stop a rewrite from silently changing the meaning of an op should land (#240110). Section 6 lists them.

Background: #236855, #233546, #233548, #234238.

Terms

Term Meaning Example
scalarized form An op written out as a sequence of scalar operations. The LangRef calls this a “scalarized reduction”. vector.reduction <add>, %v, %acc on 4 lanes is (((acc + v0) + v1) + v2) + v3.
combining operation One operation of the scalarized form that joins a value with the running result. The vector.multi_reduction docs use this term. each + above
intermediate result A value in between two combining operations. acc + v0
neutral value A value that a rewrite inserts, and that does not change the result. -0.0 for add
poison A special error value in LLVM IR. Most operations pass it on. Producing poison is not by itself undefined behavior (LangRef).
VP intrinsics LLVM’s vector predication intrinsics, llvm.vp.* (LangRef). They take a mask and a vector length, and masked-off lanes do not take part. Example: llvm.vp.reduce.fadd.

1. What LLVM says and does

All code links point at commit 62e03e74 (main, 2026-11-06). The opt runs used opt at a7117bd3 (main, 2026-11-05).

1.1 The flags

From the LangRef, “Fast-Math Flags”:

Flag Kind Meaning
nnan value-based The operands and the result are not NaN. If one is, the result is poison.
ninf value-based The operands and the result are not ±infinity. If one is, the result is poison.
nsz value-based The sign of a zero operand may flip. -0.0 is not poison.
reassoc rewrite-based Algebraically equivalent rewrites, such as reassociation, are allowed.
contract rewrite-based A separate multiply and add may be fused into a fused multiply-add (FMA).
arcp rewrite-based A division may become a multiply by the reciprocal.
afn rewrite-based A math function may use an approximation.

1.2 Rewrite-based flags

The LangRef, “Rewrite-based flags” says that every instruction of an expression must have the flag, and that “the rewritten instructions will generally have the intersection of the flags present on the input instruction”. Its example:

define double @orig(double %a, double %b, double %c) {
  %t1 = fmul contract reassoc double %a, %b
  %val = fmul contract reassoc arcp double %t1, %c
  ret double %val
}

define double @target(double %a, double %b, double %c) {
  %t1 = fmul contract reassoc double %b, %c
  %val = fmul contract reassoc double %a, %t1
  ret double %val
}
@orig @target
first fmul contract reassoc contract reassoc
second fmul contract reassoc arcp contract reassoc
why The rewrite is allowed because both ops have reassoc. arcp is on one op only, so it is dropped.

LLVM’s passes do not all do exactly this. Two examples:

  • The SelectionDAG forms an FMA only if the fadd and the fmul both have contract, unless the target has an FMAD, a multiply-add that rounds in between and is always safe (DAGCombiner.cpp:18942-19010).
  • The Reassociate pass needs reassoc and nsz for its general floating-point reassociation (Reassociate.cpp:146-152). But it gives every rewritten op the flags of the root of the expression (Reassociate.cpp:785-795), not the intersection:
; input
%m = fmul reassoc nsz float %a, %c
%r = fmul reassoc nsz arcp float %m, %b
; opt -passes=reassociate
%m = fmul reassoc nsz arcp float %b, %a
%r = fmul reassoc nsz arcp float %m, %c

Here arcp reaches both ops, although only one had it.

1.3 Value-based flags

The LangRef says that the intersection rule does not apply to value-based flags: whether one “is present on any or all of the rewritten instructions is based on whether or not it is possible for said instruction to have a NaN input or output, given the original flags”.

InstCombine’s -(X * C) --> X * -C fold shows this (InstCombineAddSub.cpp:3159-3165). From its tests in fneg.ll (:52-60, :1283-1290):

; test 1: @fmul_fneg_fmf
  %m = fmul float %x, 42.0
  %r = fneg reassoc nsz float %m
; becomes
  %r = fmul nsz float %x, -42.0

; test 2: @test_fneg_mul_combine_nnan_ninf_with_vec_const
  %f1 = fmul nnan <2 x float> %a, <float 0.0, float -0.0>
  %f2 = fneg ninf <2 x float> %f1
; becomes
  %f2 = fmul nnan <2 x float> %a, <float -0.0, float 0.0>
Test Old flags New flags Why
1 fmul: none; fneg: reassoc nsz nsz reassoc is rewrite-based and on one op only, so it is dropped. The fold keeps the union of the value-based flags, so nsz stays.
2 fmul: nnan; fneg: ninf nnan nnan stays. This fold keeps ninf only if both ops have it (InstCombineAddSub.cpp:3164).

When a new op can see a value that the old flags did not cover, InstCombine drops the flag. One select fold drops nnan from both new ops because “When X is NaN, the old code evaluated fabs(Y), while the new code evaluates fabs(X)” (InstCombineSelect.cpp:3566-3581).

1.4 Reductions

The flags sit on the call. For llvm.vector.reduce.fadd, reassoc allows any order. Without it, the reduction is sequential: it starts from the start value and goes through the lanes in increasing order. For llvm.vector.reduce.fmax, the order is not fixed; it only matters for signaling NaNs.

When LLVM splits a reduction into scalar operations (ExpandReductions), every new op gets all of the call’s flags (ExpandReductions.cpp:67-76). From expand-experimental-reductions.ll:

; @fadd_f32
  %r = call fast float @llvm.vector.reduce.fadd.f32.v4f32(
           float 0.0, <4 x float> %vec)
; CHECK: %bin.rdx  = fadd fast <4 x float> %vec, %rdx.shuf
; CHECK: %bin.rdx2 = fadd fast <4 x float> %bin.rdx, %rdx.shuf1
; CHECK: %bin.rdx3 = fadd fast float 0.000000e+00, %0

; @fadd_f32_strict
  %r = call float @llvm.vector.reduce.fadd.f32.v4f32(float undef, <4 x float> %vec)
; CHECK: %bin.rdx  = fadd float undef, %0
; CHECK: %bin.rdx1 = fadd float %bin.rdx, %1   ; ... and so on, in lane order
Test Call flags Order of the new ops Flags of the new ops
@fadd_f32 fast tree fast on every op
@fadd_f32_strict none sequential, in lane order none

LLVM uses a tree order only with reassoc for add and mul (ExpandReductions.cpp:91), and only with nnan for fmax and fmin (:161).

LLVM chooses a reduction’s neutral value from its kind and its flags (getReductionIdentity, LoopUtils.cpp:1503-1543). For add, it uses -0.0, or +0.0 with nsz (Constants.cpp:2998-2999). The min and max choices depend on whether the kind propagates NaN; section 3.3 lists them.

1.5 Splitting a vector op

Every piece gets the flags of the op it came from. From the Scalarizer test basic-inseltpoison.ll:760-775 (code: Scalarizer.cpp:560-570):

  %res = fadd fast <2 x float> %x, %y
; CHECK: [[RES_I0:%.*]] = fadd fast float [[X_I0]], [[Y_I0]]
; CHECK: [[RES_I1:%.*]] = fadd fast float [[X_I1]], [[Y_I1]]

The SelectionDAG does the same when it splits a vector op in two (LegalizeVectorTypes.cpp:1838-1850).

1.6 Rebuilding and dropping

Rebuilding. A fold that rebuilds an equivalent op with new operands copies the flags, with BinaryOperator::CreateWithCopiedFlags (InstCombineMulDivRem.cpp:655-660). From fmul.ll:77-86:

  %sub1 = fsub float -0.0, %x
  %sub2 = fsub float -0.0, %y
  %mul = fmul arcp float %sub1, %sub2
; CHECK: [[MUL:%.*]] = fmul arcp float [[X]], [[Y]]

Dropping. Removing flags is the safe choice when a transform cannot keep them. Instruction::dropPoisonGeneratingFlags clears nnan and ninf (Instruction.cpp:427-486).

2. What MLIR does today, in Linalg and Vector

The MLIR examples were run with mlir-opt at 62e03e74. The inputs and outputs are in appendix B.

2.1 Where the flags are

Dialect Ops with flags Ops without
Arith, Math the floating-point ops that declare fast-math flags. The docs give the flag names, but not their meaning (section 3.1).
Linalg the ops in a region body named ops (linalg.matmul, linalg.contract, …). Their bodies are built as arith.mulf and arith.addf without flags (LinalgOps.cpp:4191, :4724).
Vector vector.reduction, vector.multi_reduction, vector.contract vector.outerproduct, vector.fma

2.2 Where the flags are read

Where What changes
Linalg split reduction, partial-reduction tiling, sharding and padding, through arith::getNeutralElement (ArithOps.cpp:3460-3508) The start value of the inserted partial results. In example 4, a maximumf reduction starts from -inf without flags, and from the lowest finite value with nnan,ninf.
Linalg padding (PadTilingInterface.cpp:392-400) A check that the pad value is not NaN under nnan.
Vector MultiReduceToContract (VectorTransforms.cpp:102-109) The contract gets the flags that both ops share (example 2). With allowFastMathFlagLoss = false, the merge is refused when the flags differ.
VectorToLLVM masked reductions (ConvertVectorToLLVM.cpp:794-799) and the XeGPU lowering (#233548) Neutral values.

No Vector folder or canonicalization uses the flags to decide anything.

2.3 What Linalg and Vector do without flags

These are examples, not a complete list.

Transform What it does LLVM would need How we know
Linalg partial-reduction tiling (tile_reduction_using_for) Splits a float sum into partial sums and merges them at the end. reassoc example 5
Linalg split reduction The same, with the partial sums along a new dimension. reassoc example 5
Linalg interchange Can swap two reduction dimensions, which changes the order of a float sum. No flag check (Interchange.cpp:30-72). reassoc code
Linalg sharding Combines the partial results of the shards with shard.all_reduce (ShardingInterfaceImpl.cpp:187). reassoc code
Vector unrolling of vector.reduction and vector.contract Reduces the slices separately, then combines them (VectorUnroll.cpp:572-600, :303-310). reassoc code
XeGPU cross-lane reduction Combines lanes in a tree, with butterfly shuffles (XeGPUUtils.cpp:573-588). reassoc code
linalg.matmul → vectorize → contraction lowering Creates vector.fma (LowerVectorContract.cpp:165-167), which lowers to llvm.intr.fmuladd (ConvertVectorToLLVM.cpp:1305). For 4x8x16: 32 vector.fma, then 32 llvm.intr.fmuladd. contract on the multiply and the add example 6
VectorToLLVM, vector.reduction <add> Keeps the order: it adds reassoc only with the reassociate-fp-reductions option (ConvertVectorToLLVM.cpp:935-942). — code

So, in practice, many MLIR transforms treat reductions as reorderable and multiply-adds as fusable, even when no flag says so. The docs do not say this:

  • The linalg.contract docs define D[H] = (SUM A[I] * B[J]) + C[H] and give no order (LinalgStructuredOps.td:830-836).
  • The Linalg docs say that loop transforms must keep “the lexicographic order of all dependence vectors” (_index.md:250-251).
  • The vector.fma docs say it has llvm.fma semantics and is “guaranteed to lower to the llvm.fma.* intrinsic” (VectorOps.td:806-813). But the lowering emits llvm.fmuladd, for which the LangRef says “it is unspecified whether rounding will be performed between the multiplication and addition steps”.

3. Proposal

3.1 One meaning for a flag

A flag applies to every scalar operation that its op stands for:

  • Linalg body ops: the op itself, as in arith. Nothing changes here.
  • Vector reductions and contractions: every operation of the scalarized form: each combining operation, including the one with the accumulator, and for vector.contract each multiplication. Each of these operations has the flags with their LangRef meaning. For example, with nnan, no operand and no result of any of these operations is NaN. So nnan and ninf also cover the intermediate results.

The second point is the text of the vector.multi_reduction docs today (VectorOps.td:354-359). They say that the flags “have the meaning given in the LLVM Language Reference”, and that every combining operation “behaves like the matching arith op with these flags”. The proposal is to use the same text for vector.reduction and vector.contract. It matches what LLVM’s ExpandReductions does: it puts the call’s flags on every new op (section 1.4).

The Arith dialect itself does not say what the flags mean. Its docs only say that “some floating-point operations may specify rounding modes and/or fast-math flags” (ArithBase.td:33-34), and the flag enum gives only the names (ArithBase.td:116-145). In practice, the meaning is LLVM’s: each flag lowers to the LLVM flag with the same name (AttrToLLVMConverter.cpp:13-29). The proposal is to write this in the Arith docs too.

3.2 Order and fusion

There are two choices:

Choice Benefit Cost
A. Allow reordering and fusion by default Keeps the choices that today’s transforms make (section 2.3). Without flags, an op cannot ask for a fixed reduction order, or for a separate multiply and add.
B. Require LLVM-style flags Keeps the meaning of scalar code without flags through every rewrite. Gives frontends control for each op. Same meaning as LLVM IR. The transforms in section 2.3 need reassoc or contract checks. Named Linalg ops need a way to carry the flags. Frontends that rely on today’s behavior need a way to ask for it.

Recommendation: choice A, with these limits:

  • A Linalg reduction whose combiner is associative (add, mul, min, max), and a Vector reduction or contraction, may be computed in any order.
  • A Linalg contraction (linalg.matmul, linalg.contract, …) and a vector.contract may fuse a multiply with its add, as llvm.fmuladd may. Ops that are fused by definition, like vector.fma and llvm.fma, keep their own meaning.
  • The vector.fma lowering should emit llvm.fma, as the docs say (section 6, step 1). Changing the docs to llvm.fmuladd instead would change the meaning of the op.
  • The value-based flags, and arcp and afn, still come from the ops.

This choice supplies the missing permissions for order and fusion. It does not fix the lost flags or the wrong neutral values in section 4.

We would like to hear from users who need a fixed order or separate rounding in Linalg or Vector. That is the main case for choice B.

3.3 Rules for rewrites

This section is the main part of this RFC (question 2 of #240110). These rules say what to do with the flags after a rewrite that is already legal. They follow the LangRef and the LLVM examples in section 1. LLVM’s passes do not all use the same policy (section 1.2), so where they differ, we chose the safe one.

Rewrite Rule LLVM support MLIR today
Split an op into ops that each do part of the work Each new op gets all of the flags. Scalarizer, SelectionDAG split, ExpandReductions (sections 1.4, 1.5) vector.multi_reduction unrolling copies them (example 1). Others do not (section 4).
Merge ops into one Rewrite-based flags: keep the flags that all merged ops share. Value-based flags: keep a flag only when it holds for the new op. A pattern may refuse to merge, so that no flag is lost. LangRef rule; InstCombine (section 1.3); DAGCombiner FMA check (section 1.2). Reassociate differs: it copies the root’s flags. MultiReduceToContract keeps the shared flags (example 2). There the contract’s operations are the old ops’ operations, so the shared value-based flags still hold.
Rebuild an equivalent op with new operands Copy the flags, and for Vector ops the combining kind. CreateWithCopiedFlags (section 1.6) Fixed for seven contraction rebuilds in #236855 (example 3).
Drop Always correct, but later optimizations are lost. Drop a value-based flag when a new op can see values that the old flags did not cover. dropPoisonGeneratingFlags; the select fold (sections 1.3, 1.6) The Math exp(a) / exp(b) -> exp(a - b) rewrite drops nnan and ninf (example 7).
Insert a value (a neutral, start or pad value) It must not change the result, must fit the type, and must be allowed by the flags. getReductionIdentity, for the kinds it handles (section 1.4) arith::getNeutralElement (example 4), Linalg padding (section 2.2), the VectorToLLVM masked reductions, XeGPU. Gaps in section 4.

Neutral values, for formats with NaN, infinity and both signs of zero (LLVM’s choices, from getReductionIdentity and getBinOpIdentity):

Kind Neutral value
add -0.0. With nsz, +0.0 is also allowed.
mul 1.0
maxnumf, maximumnumf (ignore NaN) NaN. With nnan: -inf. With nnan and ninf: the lowest finite value.
minnumf, minimumnumf (ignore NaN) NaN. With nnan: +inf. With nnan and ninf: the highest finite value.
maximumf (propagates NaN) -inf. With ninf: the lowest finite value.
minimumf (propagates NaN) +inf. With ninf: the highest finite value.

getReductionIdentity does not handle the two *num kinds; the rows for them follow the same scheme. For formats without NaN or infinity, #233471 picks in this order: NaN for a kind that ignores NaN, if the format has NaN and nnan is not set; else the infinity that never wins, if the format has it and ninf is not set; else the lowest (for max) or highest (for min) finite value. In an unsigned format, the lowest value is zero, or the smallest positive value if there is no zero.

The convention in #240110. The issue states it as: “decompose gives each part the flags of the op, combine gives the intersection, drop is always correct, and a made-up value must be one the flags allow”. The rules above keep it, with three changes:

  • Merge. The intersection is right for the rewrite-based flags. For a value-based flag it can be wrong either way: a fold may keep more (InstCombine keeps nsz that only one op had), or must keep less (the select fold drops nnan) (section 1.3). So a value-based flag stays only when it holds for the new op.
  • Rebuild is its own rule. A rebuild that uses the short builder loses the combining kind as well as the flags (#236855).
  • Drop is always correct, and in some rewrites it is required: when a new op can see values that the old flags did not cover (example 7).

Refusing a merge. The merge rule lets a pattern refuse a merge that would lose a flag. MultiReduceToContract already has this as the allowFastMathFlagLoss option (VectorRewritePatterns.h:101-109). We propose to keep it. #236856 exposes it in the transform ops.

3.4 Per-op or per-function control

Proposal: the flags are per op, as in LLVM IR. Linalg and Vector get no function-level or module-level flags. A frontend that has per-function or per-TU options writes them as per-op flags.

Why:

  • LLVM IR keeps the flags on each instruction. It removed the "unsafe-fp-math", "no-infs-fp-math" and "no-nans-fp-math" function attributes between 2025-10 and 2026-03.
  • Clang turns its options and #pragma clang fp into flags on each instruction. Flang does the same in MLIR.
  • A per-function flag can only say what holds for every op in the function. A per-op flag moves with its op when the op is inlined, fused or tiled.

Under choice A (section 3.2), the reductions and contractions listed there may be reordered and fused without flags. That permission comes from the op, not from a flag. Every other flag is per op.

Appendix C has the code references and a Clang example.

3.5 A mechanism to carry the flags

Today each pattern copies the flags by hand, and section 4 lists the places that forget.

Short answer: partly. A mechanism fits two rules of section 3.3, split and rebuild: in both, every new op gets the flags of one source op. Merges, drops and inserted values need a decision in each pattern.

Proposal:

  1. Helpers next to ArithFastMathInterface: a setter, “copy the flags of op A to op B”, and “the rewrite-based flags that ops A and B share”.
  2. An opt-in guard for splits, like LLVM’s FastMathFlagGuard. While it is open, every new op whose flags are none gets the flags of the source op. It works with the greedy driver. Conversion patterns use the helpers.
  3. No driver-wide default. It would be wrong for merges, drops and inserted values.

This is a design sketch. It is not built or tested. Appendix D has what LLVM, Flang and MLIR have today, and how the guard would work.

4. Gaps, in Linalg and Vector

Gap Where Rule
vector.outerproduct and vector.fma have no flag attribute (VectorOps.td:796-804, :1161-1170). The contraction lowering creates vector.fma without the contract’s flags (LowerVectorContract.cpp:165-167). Vector split
Unrolling combines partial reductions without the flags. VectorUnroll.cpp:598-600 split
Warp distribution of vector.reduction passes no flags. VectorDistribute.cpp:2369-2374 split
VectorToLLVM’s float min/max reductions keep the flags on the reduction intrinsic, but not on the operation that combines the result with the accumulator. #233547 fixes this (open). ConvertVectorToLLVM.cpp:739-748 split
VectorToLLVM’s masked add, mul, minnumf, maxnumf, minimumf and maximumf reductions pass no flags to the LLVM VP reductions. ConvertVectorToLLVM.cpp:1046-1138 split
The add neutral value is +0.0 whatever the flags: in arith::getIdentityValueAttr (used by Linalg), in XeGPU, and in VectorToLLVM for reductions without an accumulator. LLVM uses -0.0 unless nsz. With +0.0, a sum of only -0.0 values gives +0.0. ArithOps.cpp:3408-3413, XeGPUUtils.cpp:789-793, ConvertVectorToLLVM.cpp:490-495 neutral value
arith::getIdentityValueAttr builds an infinity without checking that the format has one. For f8E4M3FN, that gives NaN. ArithOps.cpp:3388-3399 neutral value
arith::getNeutralElement does not handle maximumnumf and minimumnumf. ArithOps.cpp:3460-3508 neutral value
The vector.fma docs say llvm.fma; the lowering emits llvm.fmuladd, which may round between the multiply and the add. So the lowering can change the result. section 2.3 meaning
The Arith docs do not say what the fast-math flags mean. ArithBase.td:33-34 docs

Proposed for vector.outerproduct and vector.fma: give both ops an optional fastmath attribute and ArithFastMathInterface. For vector.fma, the flags apply to the fused multiply-add of each lane. For vector.outerproduct, they apply to each multiply and each combining operation, as for vector.contract. The contraction lowering, the unrolling and the lowering to LLVM then keep them. If we only document that the ops have no flags, the contraction lowering always drops the flags.

Fixed already: seven contraction rebuilds (#236855). Six of them used the short builder, which sets the kind to add and drops the flags. The seventh kept the kind but dropped the flags.

5. Open questions

  1. Order and fusion (section 3.2). Choice A (recommended) or choice B?
  2. Linalg named ops. Should they get an optional fastmath attribute, with the meaning of section 3.1? It would let a frontend set the value-based flags on a linalg.matmul, and the rewrite-based ones under choice B.
  3. Merges. Should merges keep more value-based flags when they can show that the flags hold, as some InstCombine folds do?
  4. Infinite intermediate results. Section 3.1 makes ninf cover the intermediate results of the order that a lowering picks. With reassoc, a lowering may pick a tree order with an infinite intermediate result, even when another order ends at a finite value or at NaN. Appendix A has examples. We propose to keep the rule and say this in the docs.
  5. Combining kinds. Which kinds vector.contract allows is discussed in #234238. The rules here do not depend on it.
  6. A pass that sets flags. Should MLIR have a pass that adds flags to every op in a function, for pipelines that have no frontend to set them? It would write per-op flags (section 3.4). The user would name the flags and the ops; the pass would never guess nnan or ninf on its own.
  7. The guard (section 3.5). Is an opt-in guard worth it, or are the helpers enough?

6. Plan

  1. Now, during the pause: the fixes that stop a rewrite from changing the result. In section 4, these are:
    • the +0.0 start for add without nsz;
    • the infinity start in formats that have no infinity;
    • the vector.fma lowering, which should emit llvm.fma, as the docs say. On a target without a fused instruction, llvm.fma must still give the fused result, so it may need a slower software path. llvm.fmuladd may become a separate multiply and add there (LangRef).
  2. Agree here on sections 3.1, 3.2, 3.3, 3.4 and 3.5, and on open question 7.
  3. Document the meaning, the order rule and the rewrite rules, in the Arith, Linalg and Vector docs.
  4. Add the helpers of section 3.5, and the guard if it is agreed.
  5. Fix the other gaps in section 4, each with tests that use flags. Among them: a fastmath attribute on vector.outerproduct and vector.fma, the unrolling, the warp distribution, and VectorToLLVM (#233547). Expose allowFastMathFlagLoss in the transform ops (#236856).
  6. Discuss open questions 2, 3 and 6 separately.

The tasks of #240110 map to this plan: task 1 is this RFC. Tasks 2, 3 and 4 are in step 5. Task 5 is #234238. This RFC does not decide it (open question 5).

Sorry for the long RFC. The part we most need feedback on is section 3.3, 3.2, 3.1. The appendices are there for reference.

@banach-space, @rengolin, @Jianhui-Li, @jpienaar, @FedericoBruzzone


Appendix A: infinite intermediate results

There are two ways to read nnan and ninf on a reduction:

  • Every operation (section 3.1): the flags cover every operation of the order that the lowering picks, so also its intermediate results.
  • Ends only: the flags cover only the original operands and the final result. This is the literal reading of the LangRef for a call.

MAX is the largest finite value of the type. Inputs, run with opt -passes=expand-reductions,instcombine at a7117bd3:

; add, start -0.0, input [MAX, NaN, MAX, 1]
%r = call reassoc ninf float @llvm.vector.reduce.fadd.v4f32(
         float -0.0, <4 x float> %v)

; mul, start 1.0, input [MAX, 0, MAX, 1]
; sequential:
%s = call ninf float @llvm.vector.reduce.fmul.v4f32(float 1.0, <4 x float> %v)
; tree:
%t = call reassoc ninf float @llvm.vector.reduce.fmul.v4f32(
         float 1.0, <4 x float> %v)
Flags Op and input Results of the possible orders Infinite or NaN intermediate result
reassoc ninf add, start -0.0, [MAX, NaN, MAX, 1] NaN in every order, because the NaN input takes part in every order. ninf allows NaN. MAX + MAX = inf in the tree order LLVM picks (lane 0 with lane 2)
reassoc ninf mul, start 1.0, [MAX, 0, MAX, 1] 0 or NaN, never infinity. The sequential order (((1 * MAX) * 0) * MAX) * 1 gives 0. The tree order (MAX * MAX) * (0 * 1) gives inf * 0 = NaN. MAX * MAX = inf in the tree order. The sequential order has none.
nnan vector.contract, kind<maxnumf>, operands [inf, 1] and [0, 1], accumulator 0 1, because maxnumf drops a NaN product the product inf * 0 = NaN

Under “every operation”, the tree orders of the first two rows are poison, and so is the third row. Under “ends only”, none of them is. (The first two rows need reassoc to use a tree order at all.)

The readings agree in these cases:

  • ninf, add, all operands and the accumulator finite. If an intermediate result is infinite, then “that sum first, then the other operands one by one” is an allowed order (with reassoc) that ends at the same infinity. So the input can already give poison under “ends only” too.
  • nnan, add and mul reductions. A NaN intermediate result stays NaN through the remaining operations, so the result of that order is NaN too.

What opt gave: the add example gives NaN, with or without ninf on the new ops. The sequential mul gives +0.0, and the tree mul gives NaN, with or without ninf. These are observations, not proof that the readings are the same: the LangRef allows poison to be replaced by any value.

Appendix B: MLIR examples

Run with mlir-opt at 62e03e74. Outputs are shortened to the relevant lines.

Example 1: split (vector.multi_reduction unrolling)

// input; transform: apply_patterns.vector.multi_reduction_unrolling
//                   lowering_strategy = "innerreduction"
%r = vector.multi_reduction <maxnumf>, %src, %acc fastmath<nnan,ninf> [1]
       : vector<4x8xf32> to vector<4xf32>
// output: one reduction per row, each with the same flags
%2 = vector.reduction <maxnumf>, %0, %1 fastmath<nnan,ninf> : vector<8xf32> into f32
%6 = vector.reduction <maxnumf>, %4, %5 fastmath<nnan,ninf> : vector<8xf32> into f32

Example 2: merge (MultiReduceToContract, -test-vector-reduction-to-contract-patterns)

// input
%m = arith.mulf %a, %b fastmath<nnan,contract> : vector<8x32x16xf32>
%r = vector.multi_reduction <add>, %m, %acc fastmath<nnan,reassoc,contract> [1]
       : vector<8x32x16xf32> to vector<8x16xf32>
// output: the shared flags
%0 = vector.contract {fastmath = #arith.fastmath<nnan,contract>, ...,
                      kind = #vector.kind<add>} %a, %b, %acc

Example 3: rebuild (CombineContractABTranspose, same pass)

// input
%t = vector.transpose %a, [1, 0] : vector<8x4xf32> to vector<4x8xf32>
%r = vector.contract {indexing_maps = [#ma, #mb, #mc],
                      iterator_types = ["parallel", "parallel", "reduction"],
                      kind = #vector.kind<maxnumf>,
                      fastmath = #arith.fastmath<nnan>} %t, %b, %acc
       : vector<4x8xf32>, vector<8x16xf32> into vector<4x16xf32>
// output: the transpose is folded into the maps; the kind and the flags stay
%0 = vector.contract {fastmath = #arith.fastmath<nnan>, ...,
                      kind = #vector.kind<maxnumf>} %a, %b, %acc

Example 4: neutral value (Linalg split reduction)

// input; transform: structured.split_reduction
//                   <split_factor = 4, insert_split_dimension = 1>
%r = linalg.generic {indexing_maps = [affine_map<(i, k) -> (i, k)>,
                                      affine_map<(i, k) -> (i)>],
                     iterator_types = ["parallel", "reduction"]}
     ins(%in : tensor<4x64xf32>) outs(%out : tensor<4xf32>) {
^bb0(%x: f32, %acc: f32):
  %s = arith.maximumf %x, %acc FLAGS : f32
  linalg.yield %s : f32
} -> tensor<4xf32>
FLAGS Start value of the partial results
none arith.constant 0xFF800000 : f32 (-inf)
fastmath<nnan,ninf> arith.constant -3.40282347E+38 : f32 (the lowest finite value)

Example 5: reordering without flags (Linalg)

The same linalg.generic, with %s = arith.addf %x, %acc : f32 and no flags.

Transform Output
structured.tile_reduction_using_for ... by tile_sizes = [0, 8] 8 partial sums start from 0.0 (that is, +0.0). An scf.for adds into them, and a linalg.reduce merges them.
structured.split_reduction <split_factor = 4, insert_split_dimension = 1> 4 partial sums of 16 elements start from 0.0, and a second linalg.generic adds them up.

Both change the order of the sum, and neither checks for reassoc.

Example 6: fusion without flags (Linalg to Vector to LLVM)

func.func @matmul(%a: tensor<4x8xf32>, %b: tensor<8x16xf32>,
                  %c: tensor<4x16xf32>) -> tensor<4x16xf32> {
  %r = linalg.matmul ins(%a, %b : tensor<4x8xf32>, tensor<8x16xf32>)
                     outs(%c : tensor<4x16xf32>) -> tensor<4x16xf32>
  return %r : tensor<4x16xf32>
}

Transforms: structured.vectorize_children_and_apply_patterns, then apply_patterns.vector.lower_contraction lowering_strategy = "outerproduct" and apply_patterns.vector.lower_outerproduct, then -convert-vector-to-llvm.

Stage Output
Vector 32 vector.fma
LLVM dialect 32 llvm.intr.fmuladd

The matmul body is an arith.mulf and an arith.addf without flags. Nothing in this pipeline checks for contract.

Example 7: drop (Math algebraic simplification, -test-math-algebraic-simplification)

In-tree test: algebraic-simplification.mlir:478-490.

// input
%ea = math.exp %a fastmath<fast> : f32
%eb = math.exp %b fastmath<fast> : f32
%q = arith.divf %ea, %eb fastmath<fast> : f32
// output: nnan and ninf are dropped from both new ops
%0 = arith.subf %a, %b fastmath<reassoc,nsz,arcp,contract,afn> : f32
%1 = math.exp %0 fastmath<reassoc,nsz,arcp,contract,afn> : f32

The pattern explains why: “the subtraction consumes the exponents instead of the exponentials. ninf holds for exp(-inf) / exp(0.0), i.e. 0.0 / 1.0, while the subtraction -inf - 0.0 is infinite” (AlgebraicSimplification.cpp:300-310).

Appendix C: per-op or per-function control

Details for section 3.4.

Under choice A, the reductions and contractions of section 3.2 cannot ask for a fixed order, or for a separate multiply and add. Scalar arith and math ops still need reassoc and contract for these changes.

This is what LLVM IR, Clang and Flang do:

Where the flags live Code
LLVM IR On each instruction. LLVM removed the function attributes "unsafe-fp-math" (#164534, 2025-10), "no-infs-fp-math" (#180083, 2026-02) and "no-nans-fp-math" (#186285, 2026-03). Of the function attributes that match a fast-math flag, only "no-signed-zeros-fp-math" is left. Attributes.td:442
Clang The command-line options apply to a TU. #pragma clang fp applies to a scope. When Clang emits IR, it sets them as the IRBuilder’s default flags, for each scope. So they end up as flags on each instruction. CodeGenFunction.cpp:116-126, :140-175
Flang Does the same in MLIR. FirOpBuilder keeps default flags, set from the command-line options. When they are not empty, a builder listener puts them on every new op that has ArithFastMathInterface. Bridge.cpp:6608, FIRBuilder.cpp:982-1007, FIRBuilder.h:713-719, FIRBuilder.cpp:961-970
MLIR pass option VectorToLLVM’s reassociate-fp-reductions adds reassoc to every unmasked float vector.reduction it lowers. It is a switch for the whole pass, but its output is a per-op flag. ConvertVectorToLLVM.cpp:935-942

Clang 22.0.0 (Ubuntu snapshot 856555bf, 2025-10-15, older than the pin), with -O0 -Xclang -disable-O0-optnone -S -emit-llvm, on two small functions: madd returns a * b + c, and sum3 computes t = x + y, then v = t + z under #pragma clang fp reassociate(on). The ops it emits:

Option a * b + c t = x + y v = t + z, under the pragma
-ffp-contract=on (default) llvm.fmuladd fadd fadd reassoc
-ffp-contract=fast fmul contract, fadd contract fadd contract fadd reassoc contract
-ffast-math fmul fast, fadd fast fadd fast fadd fast

Clang records each setting on single instructions. It uses flags for -ffp-contract=fast and -ffast-math. For the default -ffp-contract=on, it uses llvm.fmuladd: on allows fusion inside one expression only (CGExprScalar.cpp:4699-4715), and Clang emits the llvm.fmuladd call with no flag (:4690-4692). The pragma changes only the op in its scope. The llvm.fmuladd case is the same idea as choice A: the op itself carries the permission.

Why per op:

  • A per-function flag can only say what holds for every op in the function. Clang shows this for the one attribute that is left. It keeps "no-signed-zeros-fp-math" true only if every scope in the function has nsz (CodeGenFunction.cpp:167-174).
  • When LLVM inlines a function, the caller keeps "no-signed-zeros-fp-math" only if both functions have it (Attributes.td:502, Attributes.cpp:2665-2669). So inlining one function without it removes the permission for the whole caller. The nsz flags on single instructions stay. Per-op flags move with the op when it is inlined, fused or tiled into other code.
  • The ops that carry flags in Linalg and Vector already have them per op (section 2.1).

Pass options like reassociate-fp-reductions fit this model. They give a permission at one point of the pipeline, and they write it as a per-op flag. We propose to keep them.

Appendix D: a mechanism to carry the flags

Details for section 3.5.

Why a mechanism cannot handle every rule:

  • Merge needs the shared flags of several ops, and a check of the value-based flags.
  • Drop: some rewrites must drop a flag (example 7). A mechanism that copies all the flags would keep it.
  • Insert a value: the new value is usually a constant, which has no flags. But which value is right depends on the flags (example 4).

What exists today:

  • LLVM: FMFSource gives a new instruction the flags of one instruction, or the flags that two share (IRBuilder.h:88-108). FastMathFlagGuard sets default flags for a scope (IRBuilder.h:372-399); ExpandReductions uses it (ExpandReductions.cpp:67-76).
  • Flang: the FirOpBuilder listener (appendix C) puts the builder’s flags on every new op that has ArithFastMathInterface. If the builder has flags, they replace the flags that the op was built with (FIRBuilder.cpp:961-970).
  • MLIR: ArithFastMathInterface has a getter, but no setter (ArithOpsInterfaces.td:18-50). vector.contract, vector.reduction and vector.multi_reduction have it. LLVM dialect ops use another interface, LLVM::FastmathFlagsInterface (LLVMInterfaces.td:19).

How the guard would work.

  • A pattern opens it with the source op. While it is open, every new op that has ArithFastMathInterface and flags none gets the flags of the source op. Ops whose flags are not none keep them. Unlike Flang’s listener, the guard never replaces them. A pattern that must drop a flag on a new op uses the helpers instead.
  • The fastmath attribute defaults to none (ArithOps.td:86-88, VectorOps.td:64-66). So the guard cannot tell “not set” from “set to none”, and it acts on none.
  • With the greedy driver, the guard can be a rewriter listener (Builders.h:288-301). It wraps the current listener with RewriterBase::ForwardingListener (PatternMatch.h:428-431). It sets the flags before it forwards an insertion, skips moved ops, and puts the old listener back when the scope ends.
  • Dialect conversion hides setListener (DialectConversion.h:1039-1040). In its usual mode, it tells its listener about new ops only after the conversion succeeds (DialectConversion.h:1401-1408), so after the guard has closed. Conversion patterns, including the lowering to the LLVM dialect, use the helpers. A guard for them would need a new hook.

Tests. Every pattern that creates float ops gets a test with flags.

Thanks for the RFC, @mshahneo! The level of detail is very helpful. Thanks also @rengolin for the direction on blindly propagation, and @banach-space to rise the question on CIR/Flang vs LLVM approach, in #236855.

I’ll leave my preliminary comments below, with the aim of guiding this dense RFC towards a narrower scope :smiley:

Per-op flags (point 4)

I agree with per-op flags. The flags come, eventually, from the frontend (e.g., Flang/CIR/IREE/etc.), and we don’t want to recover that information - see the Traceability principle - i.e., linalg and vector only receive them from above.
So these layers should preserve what the frontend inferred, and I’d write the split, merge and rebuild rules in 3.x as a preservation contract.
Dropping a flag is always valid, but the drop should be documented as the exception, from my POV.

Question 2

Today a named op has no way to carry a flag. My proposal (which follows principles similar to those I outlined in [RFC] Unify masking in the Vector dialect) is that the flag lives where whoever writes the IR can write it, and in one place per op:

Linalg op Who writes the body Where the flag lives
generic, map, reduce The user Only on the ops in the body, no attribute on the op. Eventually, we could consider a mutually exclusive thing - i.e., either in the op or in the body (?).
Named ops (matmul, contract, elementwise, batch_matmul, …) Derived via regionBuilder fastmath attribute on the op. regionBuilder applies it to every generated op that implements ArithFastMathInterface
softmax, winograd_* No body fastmath attribute on the op, propagated by the decompositions.

The attribute follows your point 1: it applies to every scalar operation the op stands for.
This means a named op cannot give the mulf and the addf different flags, and anyone who needs that can use a linalg.generic.
Since the two levels never exist on the same op, there is no conflict to resolve.

Two concerns:

  1. generalizeNamedOp would copy the attribute onto each generated body op. specializeGenericOp is the hard direction. If the body ops have different flags, there is no single attribute to represent them. Should it refuse, or apply the merge rule from 3.x?
  2. A named op can still get a hand-written body through the generic form, e.g., "linalg.matmul" ... <hand-written-body>. Should a verifier reject a body that disagrees with the attribute, or is that simply unsupported?

Propagation (3.x)

I agree with propagating by default and blocking only when the semantics would break.
IIUC, this is a conservative evolution of the blind propagation approach by Renato.
While we want to stay LLVM agnostic, the LangRef already says what break means, and I suggest we use it for the helpers in point 5:

Flags Rule in a rewrite
reassoc, contract, arcp, afn The rewrite is legal only if all the involved ops have the flag. The new ops get the intersection. A split copies all the flags.
nnan, ninf, nsz The intersection does not apply. A new op keeps the flag only if, given the original flags, it cannot see a value that the flag excludes.

A shared helper like getValidFlags(newOp, replacedOps) could apply the first row exactly and the second one conservatively (?)
An operand is accepted if (i) it is an operand of a replaced op, (ii) a result of an already accepted new op, or (iii) a constant that does not violate the flag. Otherwise the flag is dropped.
I think that this is sound, but it may drop more than necessary. Right?
Of course, we could extend this reasoning as we like.

Now, is a conservative helper acceptable for now, or should the guard rely on a real value analysis?

In addition to @banach-space, @rengolin, and @Jianhui-Li, I believe the perspective of @gysit and @matthias-springer could be really useful.

Agreed.

It should refuse to specialize, otherwise the roundtrip would not work.

We discussed this during the morphism RFC and the agreement is that named ops should only have their nominal bodies. Or rather, even if you can create a matmul with a funny body, and then generalize, when you try to specialize again, it will fail.

It will, mostly because of the intersection rule. But if you’re rewriting multiple ops into other multiple ops, it’s hard to tell what’s the relationship between them individually. My impression is that most rewrites are one-to-one or one-to-many, and rarely many-to-one or many-to-many, so the concern may not be a practical problem. @matthias-springer may know more.

If today we drop most, a conservative helper would be a net improvement anyway.

Thank you for starting this discussion and for your very comprehensive summary! :folded_hands:

While there are a lot of interesting details in your post, given its size it is hard to identify what to focus on. IMO, there are a lot of elements that deserve a dedicated RFC.

Specifically, you lean towards one narrow item:

However, where specifically do we look for your proposal regarding that point? There are so many sections (I counted 31) :sweat_smile: .


I would like for us to take a huge step back and decide what to discuss and what to park. Let me start with the items from the GitHub issue that you referred to.


  • Where do Linalg and Vector sit between per-op control (the LLVM model) and per-function or per-TU control (the Clang model)?

Having thought about it, I feel that we need to consider both.

At a “dialect” level, we wouldn’t concern ourselves with functions, modules or TUs. However, neither Linalg nor Vector Ops exist in isolation. In practice, everything will be located in some “function”.

While it’s tempting to dismiss “function-level” concerns as something irrelevant to Linalg and Vector (a matter for Func dialect?), I feel that because of how Linalg/Vector are used in practice, we should include this in a discussion. I will get back to this later in my reply.

In the meantime, I wanted to say that the approach from Flang makes a lot of sense to me, thank you for that example.

  • How do flags compose when a rewrite decomposes, combines or drops ops, or makes up a value (pad, neutral, init)?

Normally, I would say “lets just follow LLVM”, but it sounds like LLVM is not very consistent here? In which case, I’d opt for being very conservative and wait for efforts like Starting a community Floating-Point Working Group to help us navigate this.

  • Can the flags be propagated by a mechanism instead of by hand in every pattern?

Right now a lot of our infra just drops the flags. If we are to maintain them, are we to update everything by hand?

I feel that for this item, the answer will depend heavily on:

  • What exactly we want fastmath flags for?

I don’t think that the answer is that obvious, see my question below.


Key thing

As you note in your post, both Linalg and Vector will happily carry out transformations for which LLVM requires fastmath flags. I refer to that as implicit fastmath.

Below is your proposal for that aspect:

To me, this is key and that’s where we should start:

  • Why bother with fastmath flags if MLIR’s default behaviour will be to ignore them?

If implicit fastmath is the preferred behaviour (*), why not just apply fastmath explicitly at the function level (**)? As in:

  • Assume that we want fast-math-like behaviour everywhere.
    • The current situation is inconsistent with LLVM - in the absence of explicit fastmath we assume fastmath everywhere, i.e. implicit fastmath.
    • IMHO, that’s very bad and I would just advocate for adding fastmath everywhere explicitly instead.
  • If we do start adding fastmath explicitly everywhere, surely that should happen at a function level and that’s something that we need to discuss (hence why per-function control should be part of this discussion).

My view on fastmath

  • Align with LLVM - fastmath is too complex and too nuanced for us to invent something new. Implicit fastmath is a huge issue, IMO.
  • From your post: " Linalg and Vector carry fast-math flags in two ways:* - this inconsistency should be fixed.
  • Per-op vs per-function control - both. However, lets start with the use cases that we want to support (e.g. to make sure that the LLVM backend receives Ops with fastmath flags set).

-Andrzej

(*) To be perfectly honest, it sounds like this is exactly what everyone is happy with today.
(**) This is one reason why it’s important that we discuss function-level control.

One downside to fast-math is that analysis functions such as computeKnownFPClass cannot model fast-math flags apart from nnan, ninf, nsz, and maybe arcp (we can only model what Alive2 is able to model). Then again, computeKnownFPClass is mainly used for “as-if” transformations, which will be much more weaker and limited compared to the rewrite power of afn, assoc, contract, and arcp.

Thanks for the detailed RFC, and sorry for joining the discussion late. I’d like to share my perspective, mainly on Linalg.

I think Linalg and Vector should treat fast-math flags differently (in particular, whether reordering and fusion need reassoc/contract), since they serve different use cases and are reached from different frontends.

Linalg models tensor math, the way GEMM in BLAS/cuBLAS/oneDNN does: a reduction iterator has no defined evaluation order and may fuse multiply-add, by definition. I wouldn’t call this “implicit fastmath”; it’s what the op means, and it’s what makes tiling along K, split-K and parallel reductions legal. Matrix hardware (DPAS, tensor cores, AMX) can’t evaluate these ops in sequential IEEE order anyway. Everything else in the body still follows the body ops’ flags, as in LLVM.

Vector is a SIMD-level dialect that can also be produced from scalar loop code (e.g. the affine vectorizer), so I think it should follow LLVM: reordering needs reassoc and fusion needs contract. When vectorizing Linalg ops, the vectorizer sets these flags on the resulting Vector ops, so from Vector down, everything is explicit and LLVM-aligned.

For the other flags (nnan/ninf/nsz, arcp, afn), I’d be conservative about adding a fastmath attribute to Linalg named ops; it should be driven by a clear use case. That use case should also determine the right form: a scope attribute, or a per-op attribute. I’m not convinced the min/max reduction case requires exposing nnan/nsz at this level: frontends that don’t need NaN propagation can choose maxnumf, which also doesn’t order -0.0 and +0.0.

That’s a very interesting perspective, thank you for bringing this up! In addition, from A Primer on “Structured” Linalg Operations (extra emphasis by me):

Note that the order in which the region is executed for various tuples of elements read from the buffers is not specified, and the write to the out buffer is written as a whole at the end of the operation.

It sounds like we should make the permissions for reordering Linalg reductions (reassoc) more prominent in the documentation, and clarify whether Linalg contractions also permit multiply-add fusion (contract). I am not sure about explicit vs implicit signposting, though. The latter would mean less noise.

BLAS/cuBLAS/oneDNN and other libraries seem to implement their own meaning of “fast math” and “strict” math, which does not map directly to LLVM flags. I wasn’t able to find any documentation re Op fusion (contract) or order of evaluation (reassoc) - to me that confirms your assessment (*).

One concern that I have about treating Linalg and Vector differently is a possible lack of consistency. Also, if we add reassoc + contract to Linalg Ops, we should consider whether there are points in the pipeline where we deliberately stop propagating those permissions (e.g. during vectorization, once an implementation has been selected).


(extra emphasis by me)

This highlights another source of concern for me. So far I have only seen patches to either add or pass fastmath flags around. Usually those flags come from some unspecified “frontend”, I am still missing a concrete example showing why the proposed level of control is needed.

This means that we are implementing something, but it’s unclear to me what that thing is.


In general, it sounds like we should split the discussion into:

  • Linalg vs Vector behaviour.
    • I would opt for uniform behaviour today. We could diverge over time, but that would be another RFC, IMO.
  • “Value-based” vs “rewrite-based” flags.
  • “Scope” vs “per-op” behaviour (I still feel that we need both).

In addition, I am still missing a concrete motivating use case for the proposed control. Especially for nnan/ninf/nsz, arcp, afn that @Jianhui-Li mentioned.

-Andrzej

(*) The only exception is oneMKL - Data Parallel C++ Developer Reference. Please share if you have more examples.

P.S. Thank you for all the discussion so far, it’s been very helpful in organising all of this in my head.