Summary
Linalg and Vector carry fast-math flags in two ways:
- Linalg has them on the scalar
arithandmathops inside a region body (linalg.generic,linalg.map,linalg.reduce). Named ops such aslinalg.matmulandlinalg.contracthave no flag attribute. - Vector has them on three ops that each stand for many scalar operations:
vector.reduction,vector.multi_reductionandvector.contract.
There is no written rule for how Linalg and Vector transforms treat these flags. Each transform decides on its own. Some lose the flags. Some insert values that the flags do not allow.
There is also an unwritten rule. Today, several Linalg and Vector transforms reorder float reductions and fuse multiply-adds without any flag (section 2.3). LLVM needs reassoc or contract for the same changes. No Linalg or Vector doc says that this is allowed.
This RFC proposes, for Linalg and Vector together:
- One meaning for a flag. A flag applies to every scalar operation that its op stands for. For a Linalg body op, that is the op itself, as in
arith. For a Vector reduction or contraction, that is every operation of its scalarized form.vector.multi_reductionalready says this. - A decision on order and fusion. We recommend writing down what the transforms do today: reductions with an associative combiner may be computed in any order, and contractions may fuse their multiply and add, as LLVM’s
llvm.fmuladdmay. Section 3.2 compares this with following LLVM. - Five rules for rewrites, the main part of this RFC. They follow the LangRef and LLVM’s code (section 3.3, with code and test references).
- Per-op control, as in LLVM IR. A frontend that has per-function or per-translation-unit (per-TU) options writes them as per-op flags, as Clang and Flang do (section 3.4).
- Shared helpers and an opt-in guard that carry the flags through splits and rebuilds. Merges, drops and made-up values still need a decision in each pattern (section 3.5).
- Fixes for the Linalg and Vector places that break these rules (section 4).
This RFC does not propose to remove the flags, or to replace them with pipeline options.
The tracking issue is #240110, under the Vector meta-issue #235708. It asks three questions. This RFC focuses on the second one: how flags compose when a rewrite splits, merges or drops ops, or makes up a value (section 3.3). That is where the rules are missing today, and where we most need feedback. The other two questions get short answers:
- Per-op or per-function control: per op (section 3.4, details in appendix C).
- A mechanism instead of code in every pattern: partly, with shared helpers and an opt-in guard (section 3.5, details in appendix D).
Until this RFC is agreed, only fixes that stop a rewrite from silently changing the meaning of an op should land (#240110). Section 6 lists them.
Background: #236855, #233546, #233548, #234238.
Terms
| Term | Meaning | Example |
|---|---|---|
| scalarized form | An op written out as a sequence of scalar operations. The LangRef calls this a “scalarized reduction”. | vector.reduction <add>, %v, %acc on 4 lanes is (((acc + v0) + v1) + v2) + v3. |
| combining operation | One operation of the scalarized form that joins a value with the running result. The vector.multi_reduction docs use this term. |
each + above |
| intermediate result | A value in between two combining operations. | acc + v0 |
| neutral value | A value that a rewrite inserts, and that does not change the result. | -0.0 for add |
| poison | A special error value in LLVM IR. Most operations pass it on. Producing poison is not by itself undefined behavior (LangRef). | |
| VP intrinsics | LLVM’s vector predication intrinsics, llvm.vp.* (LangRef). They take a mask and a vector length, and masked-off lanes do not take part. Example: llvm.vp.reduce.fadd. |
1. What LLVM says and does
All code links point at commit 62e03e74 (main, 2026-11-06). The opt runs used opt at a7117bd3 (main, 2026-11-05).
1.1 The flags
From the LangRef, “Fast-Math Flags”:
| Flag | Kind | Meaning |
|---|---|---|
nnan |
value-based | The operands and the result are not NaN. If one is, the result is poison. |
ninf |
value-based | The operands and the result are not ±infinity. If one is, the result is poison. |
nsz |
value-based | The sign of a zero operand may flip. -0.0 is not poison. |
reassoc |
rewrite-based | Algebraically equivalent rewrites, such as reassociation, are allowed. |
contract |
rewrite-based | A separate multiply and add may be fused into a fused multiply-add (FMA). |
arcp |
rewrite-based | A division may become a multiply by the reciprocal. |
afn |
rewrite-based | A math function may use an approximation. |
1.2 Rewrite-based flags
The LangRef, “Rewrite-based flags” says that every instruction of an expression must have the flag, and that “the rewritten instructions will generally have the intersection of the flags present on the input instruction”. Its example:
define double @orig(double %a, double %b, double %c) {
%t1 = fmul contract reassoc double %a, %b
%val = fmul contract reassoc arcp double %t1, %c
ret double %val
}
define double @target(double %a, double %b, double %c) {
%t1 = fmul contract reassoc double %b, %c
%val = fmul contract reassoc double %a, %t1
ret double %val
}
@orig |
@target |
|
|---|---|---|
first fmul |
contract reassoc |
contract reassoc |
second fmul |
contract reassoc arcp |
contract reassoc |
| why | The rewrite is allowed because both ops have reassoc. arcp is on one op only, so it is dropped. |
LLVM’s passes do not all do exactly this. Two examples:
- The SelectionDAG forms an FMA only if the
faddand thefmulboth havecontract, unless the target has an FMAD, a multiply-add that rounds in between and is always safe (DAGCombiner.cpp:18942-19010). - The Reassociate pass needs
reassocandnszfor its general floating-point reassociation (Reassociate.cpp:146-152). But it gives every rewritten op the flags of the root of the expression (Reassociate.cpp:785-795), not the intersection:
; input
%m = fmul reassoc nsz float %a, %c
%r = fmul reassoc nsz arcp float %m, %b
; opt -passes=reassociate
%m = fmul reassoc nsz arcp float %b, %a
%r = fmul reassoc nsz arcp float %m, %c
Here arcp reaches both ops, although only one had it.
1.3 Value-based flags
The LangRef says that the intersection rule does not apply to value-based flags: whether one “is present on any or all of the rewritten instructions is based on whether or not it is possible for said instruction to have a NaN input or output, given the original flags”.
InstCombine’s -(X * C) --> X * -C fold shows this (InstCombineAddSub.cpp:3159-3165). From its tests in fneg.ll (:52-60, :1283-1290):
; test 1: @fmul_fneg_fmf
%m = fmul float %x, 42.0
%r = fneg reassoc nsz float %m
; becomes
%r = fmul nsz float %x, -42.0
; test 2: @test_fneg_mul_combine_nnan_ninf_with_vec_const
%f1 = fmul nnan <2 x float> %a, <float 0.0, float -0.0>
%f2 = fneg ninf <2 x float> %f1
; becomes
%f2 = fmul nnan <2 x float> %a, <float -0.0, float 0.0>
| Test | Old flags | New flags | Why |
|---|---|---|---|
| 1 | fmul: none; fneg: reassoc nsz |
nsz |
reassoc is rewrite-based and on one op only, so it is dropped. The fold keeps the union of the value-based flags, so nsz stays. |
| 2 | fmul: nnan; fneg: ninf |
nnan |
nnan stays. This fold keeps ninf only if both ops have it (InstCombineAddSub.cpp:3164). |
When a new op can see a value that the old flags did not cover, InstCombine drops the flag. One select fold drops nnan from both new ops because “When X is NaN, the old code evaluated fabs(Y), while the new code evaluates fabs(X)” (InstCombineSelect.cpp:3566-3581).
1.4 Reductions
The flags sit on the call. For llvm.vector.reduce.fadd, reassoc allows any order. Without it, the reduction is sequential: it starts from the start value and goes through the lanes in increasing order. For llvm.vector.reduce.fmax, the order is not fixed; it only matters for signaling NaNs.
When LLVM splits a reduction into scalar operations (ExpandReductions), every new op gets all of the call’s flags (ExpandReductions.cpp:67-76). From expand-experimental-reductions.ll:
; @fadd_f32
%r = call fast float @llvm.vector.reduce.fadd.f32.v4f32(
float 0.0, <4 x float> %vec)
; CHECK: %bin.rdx = fadd fast <4 x float> %vec, %rdx.shuf
; CHECK: %bin.rdx2 = fadd fast <4 x float> %bin.rdx, %rdx.shuf1
; CHECK: %bin.rdx3 = fadd fast float 0.000000e+00, %0
; @fadd_f32_strict
%r = call float @llvm.vector.reduce.fadd.f32.v4f32(float undef, <4 x float> %vec)
; CHECK: %bin.rdx = fadd float undef, %0
; CHECK: %bin.rdx1 = fadd float %bin.rdx, %1 ; ... and so on, in lane order
| Test | Call flags | Order of the new ops | Flags of the new ops |
|---|---|---|---|
@fadd_f32 |
fast |
tree | fast on every op |
@fadd_f32_strict |
none | sequential, in lane order | none |
LLVM uses a tree order only with reassoc for add and mul (ExpandReductions.cpp:91), and only with nnan for fmax and fmin (:161).
LLVM chooses a reduction’s neutral value from its kind and its flags (getReductionIdentity, LoopUtils.cpp:1503-1543). For add, it uses -0.0, or +0.0 with nsz (Constants.cpp:2998-2999). The min and max choices depend on whether the kind propagates NaN; section 3.3 lists them.
1.5 Splitting a vector op
Every piece gets the flags of the op it came from. From the Scalarizer test basic-inseltpoison.ll:760-775 (code: Scalarizer.cpp:560-570):
%res = fadd fast <2 x float> %x, %y
; CHECK: [[RES_I0:%.*]] = fadd fast float [[X_I0]], [[Y_I0]]
; CHECK: [[RES_I1:%.*]] = fadd fast float [[X_I1]], [[Y_I1]]
The SelectionDAG does the same when it splits a vector op in two (LegalizeVectorTypes.cpp:1838-1850).
1.6 Rebuilding and dropping
Rebuilding. A fold that rebuilds an equivalent op with new operands copies the flags, with BinaryOperator::CreateWithCopiedFlags (InstCombineMulDivRem.cpp:655-660). From fmul.ll:77-86:
%sub1 = fsub float -0.0, %x
%sub2 = fsub float -0.0, %y
%mul = fmul arcp float %sub1, %sub2
; CHECK: [[MUL:%.*]] = fmul arcp float [[X]], [[Y]]
Dropping. Removing flags is the safe choice when a transform cannot keep them. Instruction::dropPoisonGeneratingFlags clears nnan and ninf (Instruction.cpp:427-486).
2. What MLIR does today, in Linalg and Vector
The MLIR examples were run with mlir-opt at 62e03e74. The inputs and outputs are in appendix B.
2.1 Where the flags are
| Dialect | Ops with flags | Ops without |
|---|---|---|
| Arith, Math | the floating-point ops that declare fast-math flags. The docs give the flag names, but not their meaning (section 3.1). | |
| Linalg | the ops in a region body | named ops (linalg.matmul, linalg.contract, …). Their bodies are built as arith.mulf and arith.addf without flags (LinalgOps.cpp:4191, :4724). |
| Vector | vector.reduction, vector.multi_reduction, vector.contract |
vector.outerproduct, vector.fma |
2.2 Where the flags are read
| Where | What changes |
|---|---|
Linalg split reduction, partial-reduction tiling, sharding and padding, through arith::getNeutralElement (ArithOps.cpp:3460-3508) |
The start value of the inserted partial results. In example 4, a maximumf reduction starts from -inf without flags, and from the lowest finite value with nnan,ninf. |
| Linalg padding (PadTilingInterface.cpp:392-400) | A check that the pad value is not NaN under nnan. |
Vector MultiReduceToContract (VectorTransforms.cpp:102-109) |
The contract gets the flags that both ops share (example 2). With allowFastMathFlagLoss = false, the merge is refused when the flags differ. |
| VectorToLLVM masked reductions (ConvertVectorToLLVM.cpp:794-799) and the XeGPU lowering (#233548) | Neutral values. |
No Vector folder or canonicalization uses the flags to decide anything.
2.3 What Linalg and Vector do without flags
These are examples, not a complete list.
| Transform | What it does | LLVM would need | How we know |
|---|---|---|---|
Linalg partial-reduction tiling (tile_reduction_using_for) |
Splits a float sum into partial sums and merges them at the end. | reassoc |
example 5 |
| Linalg split reduction | The same, with the partial sums along a new dimension. | reassoc |
example 5 |
| Linalg interchange | Can swap two reduction dimensions, which changes the order of a float sum. No flag check (Interchange.cpp:30-72). | reassoc |
code |
| Linalg sharding | Combines the partial results of the shards with shard.all_reduce (ShardingInterfaceImpl.cpp:187). |
reassoc |
code |
Vector unrolling of vector.reduction and vector.contract |
Reduces the slices separately, then combines them (VectorUnroll.cpp:572-600, :303-310). | reassoc |
code |
| XeGPU cross-lane reduction | Combines lanes in a tree, with butterfly shuffles (XeGPUUtils.cpp:573-588). | reassoc |
code |
linalg.matmul → vectorize → contraction lowering |
Creates vector.fma (LowerVectorContract.cpp:165-167), which lowers to llvm.intr.fmuladd (ConvertVectorToLLVM.cpp:1305). For 4x8x16: 32 vector.fma, then 32 llvm.intr.fmuladd. |
contract on the multiply and the add |
example 6 |
VectorToLLVM, vector.reduction <add> |
Keeps the order: it adds reassoc only with the reassociate-fp-reductions option (ConvertVectorToLLVM.cpp:935-942). |
— | code |
So, in practice, many MLIR transforms treat reductions as reorderable and multiply-adds as fusable, even when no flag says so. The docs do not say this:
- The
linalg.contractdocs defineD[H] = (SUM A[I] * B[J]) + C[H]and give no order (LinalgStructuredOps.td:830-836). - The Linalg docs say that loop transforms must keep “the lexicographic order of all dependence vectors” (_index.md:250-251).
- The
vector.fmadocs say it hasllvm.fmasemantics and is “guaranteed to lower to thellvm.fma.*intrinsic” (VectorOps.td:806-813). But the lowering emitsllvm.fmuladd, for which the LangRef says “it is unspecified whether rounding will be performed between the multiplication and addition steps”.
3. Proposal
3.1 One meaning for a flag
A flag applies to every scalar operation that its op stands for:
- Linalg body ops: the op itself, as in
arith. Nothing changes here. - Vector reductions and contractions: every operation of the scalarized form: each combining operation, including the one with the accumulator, and for
vector.contracteach multiplication. Each of these operations has the flags with their LangRef meaning. For example, withnnan, no operand and no result of any of these operations is NaN. Sonnanandninfalso cover the intermediate results.
The second point is the text of the vector.multi_reduction docs today (VectorOps.td:354-359). They say that the flags “have the meaning given in the LLVM Language Reference”, and that every combining operation “behaves like the matching arith op with these flags”. The proposal is to use the same text for vector.reduction and vector.contract. It matches what LLVM’s ExpandReductions does: it puts the call’s flags on every new op (section 1.4).
The Arith dialect itself does not say what the flags mean. Its docs only say that “some floating-point operations may specify rounding modes and/or fast-math flags” (ArithBase.td:33-34), and the flag enum gives only the names (ArithBase.td:116-145). In practice, the meaning is LLVM’s: each flag lowers to the LLVM flag with the same name (AttrToLLVMConverter.cpp:13-29). The proposal is to write this in the Arith docs too.
3.2 Order and fusion
There are two choices:
| Choice | Benefit | Cost |
|---|---|---|
| A. Allow reordering and fusion by default | Keeps the choices that today’s transforms make (section 2.3). | Without flags, an op cannot ask for a fixed reduction order, or for a separate multiply and add. |
| B. Require LLVM-style flags | Keeps the meaning of scalar code without flags through every rewrite. Gives frontends control for each op. Same meaning as LLVM IR. | The transforms in section 2.3 need reassoc or contract checks. Named Linalg ops need a way to carry the flags. Frontends that rely on today’s behavior need a way to ask for it. |
Recommendation: choice A, with these limits:
- A Linalg reduction whose combiner is associative (add, mul, min, max), and a Vector reduction or contraction, may be computed in any order.
- A Linalg contraction (
linalg.matmul,linalg.contract, …) and avector.contractmay fuse a multiply with its add, asllvm.fmuladdmay. Ops that are fused by definition, likevector.fmaandllvm.fma, keep their own meaning. - The
vector.fmalowering should emitllvm.fma, as the docs say (section 6, step 1). Changing the docs tollvm.fmuladdinstead would change the meaning of the op. - The value-based flags, and
arcpandafn, still come from the ops.
This choice supplies the missing permissions for order and fusion. It does not fix the lost flags or the wrong neutral values in section 4.
We would like to hear from users who need a fixed order or separate rounding in Linalg or Vector. That is the main case for choice B.
3.3 Rules for rewrites
This section is the main part of this RFC (question 2 of #240110). These rules say what to do with the flags after a rewrite that is already legal. They follow the LangRef and the LLVM examples in section 1. LLVM’s passes do not all use the same policy (section 1.2), so where they differ, we chose the safe one.
| Rewrite | Rule | LLVM support | MLIR today |
|---|---|---|---|
| Split an op into ops that each do part of the work | Each new op gets all of the flags. | Scalarizer, SelectionDAG split, ExpandReductions (sections 1.4, 1.5) |
vector.multi_reduction unrolling copies them (example 1). Others do not (section 4). |
| Merge ops into one | Rewrite-based flags: keep the flags that all merged ops share. Value-based flags: keep a flag only when it holds for the new op. A pattern may refuse to merge, so that no flag is lost. | LangRef rule; InstCombine (section 1.3); DAGCombiner FMA check (section 1.2). Reassociate differs: it copies the root’s flags. | MultiReduceToContract keeps the shared flags (example 2). There the contract’s operations are the old ops’ operations, so the shared value-based flags still hold. |
| Rebuild an equivalent op with new operands | Copy the flags, and for Vector ops the combining kind. | CreateWithCopiedFlags (section 1.6) |
Fixed for seven contraction rebuilds in #236855 (example 3). |
| Drop | Always correct, but later optimizations are lost. Drop a value-based flag when a new op can see values that the old flags did not cover. | dropPoisonGeneratingFlags; the select fold (sections 1.3, 1.6) |
The Math exp(a) / exp(b) -> exp(a - b) rewrite drops nnan and ninf (example 7). |
| Insert a value (a neutral, start or pad value) | It must not change the result, must fit the type, and must be allowed by the flags. | getReductionIdentity, for the kinds it handles (section 1.4) |
arith::getNeutralElement (example 4), Linalg padding (section 2.2), the VectorToLLVM masked reductions, XeGPU. Gaps in section 4. |
Neutral values, for formats with NaN, infinity and both signs of zero (LLVM’s choices, from getReductionIdentity and getBinOpIdentity):
| Kind | Neutral value |
|---|---|
add |
-0.0. With nsz, +0.0 is also allowed. |
mul |
1.0 |
maxnumf, maximumnumf (ignore NaN) |
NaN. With nnan: -inf. With nnan and ninf: the lowest finite value. |
minnumf, minimumnumf (ignore NaN) |
NaN. With nnan: +inf. With nnan and ninf: the highest finite value. |
maximumf (propagates NaN) |
-inf. With ninf: the lowest finite value. |
minimumf (propagates NaN) |
+inf. With ninf: the highest finite value. |
getReductionIdentity does not handle the two *num kinds; the rows for them follow the same scheme. For formats without NaN or infinity, #233471 picks in this order: NaN for a kind that ignores NaN, if the format has NaN and nnan is not set; else the infinity that never wins, if the format has it and ninf is not set; else the lowest (for max) or highest (for min) finite value. In an unsigned format, the lowest value is zero, or the smallest positive value if there is no zero.
The convention in #240110. The issue states it as: “decompose gives each part the flags of the op, combine gives the intersection, drop is always correct, and a made-up value must be one the flags allow”. The rules above keep it, with three changes:
- Merge. The intersection is right for the rewrite-based flags. For a value-based flag it can be wrong either way: a fold may keep more (InstCombine keeps
nszthat only one op had), or must keep less (the select fold dropsnnan) (section 1.3). So a value-based flag stays only when it holds for the new op. - Rebuild is its own rule. A rebuild that uses the short builder loses the combining kind as well as the flags (#236855).
- Drop is always correct, and in some rewrites it is required: when a new op can see values that the old flags did not cover (example 7).
Refusing a merge. The merge rule lets a pattern refuse a merge that would lose a flag. MultiReduceToContract already has this as the allowFastMathFlagLoss option (VectorRewritePatterns.h:101-109). We propose to keep it. #236856 exposes it in the transform ops.
3.4 Per-op or per-function control
Proposal: the flags are per op, as in LLVM IR. Linalg and Vector get no function-level or module-level flags. A frontend that has per-function or per-TU options writes them as per-op flags.
Why:
- LLVM IR keeps the flags on each instruction. It removed the
"unsafe-fp-math","no-infs-fp-math"and"no-nans-fp-math"function attributes between 2025-10 and 2026-03. - Clang turns its options and
#pragma clang fpinto flags on each instruction. Flang does the same in MLIR. - A per-function flag can only say what holds for every op in the function. A per-op flag moves with its op when the op is inlined, fused or tiled.
Under choice A (section 3.2), the reductions and contractions listed there may be reordered and fused without flags. That permission comes from the op, not from a flag. Every other flag is per op.
Appendix C has the code references and a Clang example.
3.5 A mechanism to carry the flags
Today each pattern copies the flags by hand, and section 4 lists the places that forget.
Short answer: partly. A mechanism fits two rules of section 3.3, split and rebuild: in both, every new op gets the flags of one source op. Merges, drops and inserted values need a decision in each pattern.
Proposal:
- Helpers next to
ArithFastMathInterface: a setter, “copy the flags of op A to op B”, and “the rewrite-based flags that ops A and B share”. - An opt-in guard for splits, like LLVM’s
FastMathFlagGuard. While it is open, every new op whose flags arenonegets the flags of the source op. It works with the greedy driver. Conversion patterns use the helpers. - No driver-wide default. It would be wrong for merges, drops and inserted values.
This is a design sketch. It is not built or tested. Appendix D has what LLVM, Flang and MLIR have today, and how the guard would work.
4. Gaps, in Linalg and Vector
| Gap | Where | Rule |
|---|---|---|
vector.outerproduct and vector.fma have no flag attribute (VectorOps.td:796-804, :1161-1170). The contraction lowering creates vector.fma without the contract’s flags (LowerVectorContract.cpp:165-167). |
Vector | split |
| Unrolling combines partial reductions without the flags. | VectorUnroll.cpp:598-600 | split |
Warp distribution of vector.reduction passes no flags. |
VectorDistribute.cpp:2369-2374 | split |
| VectorToLLVM’s float min/max reductions keep the flags on the reduction intrinsic, but not on the operation that combines the result with the accumulator. #233547 fixes this (open). | ConvertVectorToLLVM.cpp:739-748 | split |
VectorToLLVM’s masked add, mul, minnumf, maxnumf, minimumf and maximumf reductions pass no flags to the LLVM VP reductions. |
ConvertVectorToLLVM.cpp:1046-1138 | split |
The add neutral value is +0.0 whatever the flags: in arith::getIdentityValueAttr (used by Linalg), in XeGPU, and in VectorToLLVM for reductions without an accumulator. LLVM uses -0.0 unless nsz. With +0.0, a sum of only -0.0 values gives +0.0. |
ArithOps.cpp:3408-3413, XeGPUUtils.cpp:789-793, ConvertVectorToLLVM.cpp:490-495 | neutral value |
arith::getIdentityValueAttr builds an infinity without checking that the format has one. For f8E4M3FN, that gives NaN. |
ArithOps.cpp:3388-3399 | neutral value |
arith::getNeutralElement does not handle maximumnumf and minimumnumf. |
ArithOps.cpp:3460-3508 | neutral value |
The vector.fma docs say llvm.fma; the lowering emits llvm.fmuladd, which may round between the multiply and the add. So the lowering can change the result. |
section 2.3 | meaning |
| The Arith docs do not say what the fast-math flags mean. | ArithBase.td:33-34 | docs |
Proposed for vector.outerproduct and vector.fma: give both ops an optional fastmath attribute and ArithFastMathInterface. For vector.fma, the flags apply to the fused multiply-add of each lane. For vector.outerproduct, they apply to each multiply and each combining operation, as for vector.contract. The contraction lowering, the unrolling and the lowering to LLVM then keep them. If we only document that the ops have no flags, the contraction lowering always drops the flags.
Fixed already: seven contraction rebuilds (#236855). Six of them used the short builder, which sets the kind to add and drops the flags. The seventh kept the kind but dropped the flags.
5. Open questions
- Order and fusion (section 3.2). Choice A (recommended) or choice B?
- Linalg named ops. Should they get an optional
fastmathattribute, with the meaning of section 3.1? It would let a frontend set the value-based flags on alinalg.matmul, and the rewrite-based ones under choice B. - Merges. Should merges keep more value-based flags when they can show that the flags hold, as some InstCombine folds do?
- Infinite intermediate results. Section 3.1 makes
ninfcover the intermediate results of the order that a lowering picks. Withreassoc, a lowering may pick a tree order with an infinite intermediate result, even when another order ends at a finite value or at NaN. Appendix A has examples. We propose to keep the rule and say this in the docs. - Combining kinds. Which kinds
vector.contractallows is discussed in #234238. The rules here do not depend on it. - A pass that sets flags. Should MLIR have a pass that adds flags to every op in a function, for pipelines that have no frontend to set them? It would write per-op flags (section 3.4). The user would name the flags and the ops; the pass would never guess
nnanorninfon its own. - The guard (section 3.5). Is an opt-in guard worth it, or are the helpers enough?
6. Plan
- Now, during the pause: the fixes that stop a rewrite from changing the result. In section 4, these are:
- the
+0.0start for add withoutnsz; - the infinity start in formats that have no infinity;
- the
vector.fmalowering, which should emitllvm.fma, as the docs say. On a target without a fused instruction,llvm.fmamust still give the fused result, so it may need a slower software path.llvm.fmuladdmay become a separate multiply and add there (LangRef).
- the
- Agree here on sections 3.1, 3.2, 3.3, 3.4 and 3.5, and on open question 7.
- Document the meaning, the order rule and the rewrite rules, in the Arith, Linalg and Vector docs.
- Add the helpers of section 3.5, and the guard if it is agreed.
- Fix the other gaps in section 4, each with tests that use flags. Among them: a
fastmathattribute onvector.outerproductandvector.fma, the unrolling, the warp distribution, and VectorToLLVM (#233547). ExposeallowFastMathFlagLossin the transform ops (#236856). - Discuss open questions 2, 3 and 6 separately.
The tasks of #240110 map to this plan: task 1 is this RFC. Tasks 2, 3 and 4 are in step 5. Task 5 is #234238. This RFC does not decide it (open question 5).
Sorry for the long RFC. The part we most need feedback on is section 3.3, 3.2, 3.1. The appendices are there for reference.
@banach-space, @rengolin, @Jianhui-Li, @jpienaar, @FedericoBruzzone
Appendix A: infinite intermediate results
There are two ways to read nnan and ninf on a reduction:
- Every operation (section 3.1): the flags cover every operation of the order that the lowering picks, so also its intermediate results.
- Ends only: the flags cover only the original operands and the final result. This is the literal reading of the LangRef for a call.
MAX is the largest finite value of the type. Inputs, run with opt -passes=expand-reductions,instcombine at a7117bd3:
; add, start -0.0, input [MAX, NaN, MAX, 1]
%r = call reassoc ninf float @llvm.vector.reduce.fadd.v4f32(
float -0.0, <4 x float> %v)
; mul, start 1.0, input [MAX, 0, MAX, 1]
; sequential:
%s = call ninf float @llvm.vector.reduce.fmul.v4f32(float 1.0, <4 x float> %v)
; tree:
%t = call reassoc ninf float @llvm.vector.reduce.fmul.v4f32(
float 1.0, <4 x float> %v)
| Flags | Op and input | Results of the possible orders | Infinite or NaN intermediate result |
|---|---|---|---|
reassoc ninf |
add, start -0.0, [MAX, NaN, MAX, 1] |
NaN in every order, because the NaN input takes part in every order. ninf allows NaN. |
MAX + MAX = inf in the tree order LLVM picks (lane 0 with lane 2) |
reassoc ninf |
mul, start 1.0, [MAX, 0, MAX, 1] |
0 or NaN, never infinity. The sequential order (((1 * MAX) * 0) * MAX) * 1 gives 0. The tree order (MAX * MAX) * (0 * 1) gives inf * 0 = NaN. |
MAX * MAX = inf in the tree order. The sequential order has none. |
nnan |
vector.contract, kind<maxnumf>, operands [inf, 1] and [0, 1], accumulator 0 |
1, because maxnumf drops a NaN product |
the product inf * 0 = NaN |
Under “every operation”, the tree orders of the first two rows are poison, and so is the third row. Under “ends only”, none of them is. (The first two rows need reassoc to use a tree order at all.)
The readings agree in these cases:
ninf, add, all operands and the accumulator finite. If an intermediate result is infinite, then “that sum first, then the other operands one by one” is an allowed order (withreassoc) that ends at the same infinity. So the input can already give poison under “ends only” too.nnan, add and mul reductions. A NaN intermediate result stays NaN through the remaining operations, so the result of that order is NaN too.
What opt gave: the add example gives NaN, with or without ninf on the new ops. The sequential mul gives +0.0, and the tree mul gives NaN, with or without ninf. These are observations, not proof that the readings are the same: the LangRef allows poison to be replaced by any value.
Appendix B: MLIR examples
Run with mlir-opt at 62e03e74. Outputs are shortened to the relevant lines.
Example 1: split (vector.multi_reduction unrolling)
// input; transform: apply_patterns.vector.multi_reduction_unrolling
// lowering_strategy = "innerreduction"
%r = vector.multi_reduction <maxnumf>, %src, %acc fastmath<nnan,ninf> [1]
: vector<4x8xf32> to vector<4xf32>
// output: one reduction per row, each with the same flags
%2 = vector.reduction <maxnumf>, %0, %1 fastmath<nnan,ninf> : vector<8xf32> into f32
%6 = vector.reduction <maxnumf>, %4, %5 fastmath<nnan,ninf> : vector<8xf32> into f32
Example 2: merge (MultiReduceToContract, -test-vector-reduction-to-contract-patterns)
// input
%m = arith.mulf %a, %b fastmath<nnan,contract> : vector<8x32x16xf32>
%r = vector.multi_reduction <add>, %m, %acc fastmath<nnan,reassoc,contract> [1]
: vector<8x32x16xf32> to vector<8x16xf32>
// output: the shared flags
%0 = vector.contract {fastmath = #arith.fastmath<nnan,contract>, ...,
kind = #vector.kind<add>} %a, %b, %acc
Example 3: rebuild (CombineContractABTranspose, same pass)
// input
%t = vector.transpose %a, [1, 0] : vector<8x4xf32> to vector<4x8xf32>
%r = vector.contract {indexing_maps = [#ma, #mb, #mc],
iterator_types = ["parallel", "parallel", "reduction"],
kind = #vector.kind<maxnumf>,
fastmath = #arith.fastmath<nnan>} %t, %b, %acc
: vector<4x8xf32>, vector<8x16xf32> into vector<4x16xf32>
// output: the transpose is folded into the maps; the kind and the flags stay
%0 = vector.contract {fastmath = #arith.fastmath<nnan>, ...,
kind = #vector.kind<maxnumf>} %a, %b, %acc
Example 4: neutral value (Linalg split reduction)
// input; transform: structured.split_reduction
// <split_factor = 4, insert_split_dimension = 1>
%r = linalg.generic {indexing_maps = [affine_map<(i, k) -> (i, k)>,
affine_map<(i, k) -> (i)>],
iterator_types = ["parallel", "reduction"]}
ins(%in : tensor<4x64xf32>) outs(%out : tensor<4xf32>) {
^bb0(%x: f32, %acc: f32):
%s = arith.maximumf %x, %acc FLAGS : f32
linalg.yield %s : f32
} -> tensor<4xf32>
FLAGS |
Start value of the partial results |
|---|---|
| none | arith.constant 0xFF800000 : f32 (-inf) |
fastmath<nnan,ninf> |
arith.constant -3.40282347E+38 : f32 (the lowest finite value) |
Example 5: reordering without flags (Linalg)
The same linalg.generic, with %s = arith.addf %x, %acc : f32 and no flags.
| Transform | Output |
|---|---|
structured.tile_reduction_using_for ... by tile_sizes = [0, 8] |
8 partial sums start from 0.0 (that is, +0.0). An scf.for adds into them, and a linalg.reduce merges them. |
structured.split_reduction <split_factor = 4, insert_split_dimension = 1> |
4 partial sums of 16 elements start from 0.0, and a second linalg.generic adds them up. |
Both change the order of the sum, and neither checks for reassoc.
Example 6: fusion without flags (Linalg to Vector to LLVM)
func.func @matmul(%a: tensor<4x8xf32>, %b: tensor<8x16xf32>,
%c: tensor<4x16xf32>) -> tensor<4x16xf32> {
%r = linalg.matmul ins(%a, %b : tensor<4x8xf32>, tensor<8x16xf32>)
outs(%c : tensor<4x16xf32>) -> tensor<4x16xf32>
return %r : tensor<4x16xf32>
}
Transforms: structured.vectorize_children_and_apply_patterns, then apply_patterns.vector.lower_contraction lowering_strategy = "outerproduct" and apply_patterns.vector.lower_outerproduct, then -convert-vector-to-llvm.
| Stage | Output |
|---|---|
| Vector | 32 vector.fma |
| LLVM dialect | 32 llvm.intr.fmuladd |
The matmul body is an arith.mulf and an arith.addf without flags. Nothing in this pipeline checks for contract.
Example 7: drop (Math algebraic simplification, -test-math-algebraic-simplification)
In-tree test: algebraic-simplification.mlir:478-490.
// input
%ea = math.exp %a fastmath<fast> : f32
%eb = math.exp %b fastmath<fast> : f32
%q = arith.divf %ea, %eb fastmath<fast> : f32
// output: nnan and ninf are dropped from both new ops
%0 = arith.subf %a, %b fastmath<reassoc,nsz,arcp,contract,afn> : f32
%1 = math.exp %0 fastmath<reassoc,nsz,arcp,contract,afn> : f32
The pattern explains why: “the subtraction consumes the exponents instead of the exponentials. ninf holds for exp(-inf) / exp(0.0), i.e. 0.0 / 1.0, while the subtraction -inf - 0.0 is infinite” (AlgebraicSimplification.cpp:300-310).
Appendix C: per-op or per-function control
Details for section 3.4.
Under choice A, the reductions and contractions of section 3.2 cannot ask for a fixed order, or for a separate multiply and add. Scalar arith and math ops still need reassoc and contract for these changes.
This is what LLVM IR, Clang and Flang do:
| Where the flags live | Code | |
|---|---|---|
| LLVM IR | On each instruction. LLVM removed the function attributes "unsafe-fp-math" (#164534, 2025-10), "no-infs-fp-math" (#180083, 2026-02) and "no-nans-fp-math" (#186285, 2026-03). Of the function attributes that match a fast-math flag, only "no-signed-zeros-fp-math" is left. |
Attributes.td:442 |
| Clang | The command-line options apply to a TU. #pragma clang fp applies to a scope. When Clang emits IR, it sets them as the IRBuilder’s default flags, for each scope. So they end up as flags on each instruction. |
CodeGenFunction.cpp:116-126, :140-175 |
| Flang | Does the same in MLIR. FirOpBuilder keeps default flags, set from the command-line options. When they are not empty, a builder listener puts them on every new op that has ArithFastMathInterface. |
Bridge.cpp:6608, FIRBuilder.cpp:982-1007, FIRBuilder.h:713-719, FIRBuilder.cpp:961-970 |
| MLIR pass option | VectorToLLVM’s reassociate-fp-reductions adds reassoc to every unmasked float vector.reduction it lowers. It is a switch for the whole pass, but its output is a per-op flag. |
ConvertVectorToLLVM.cpp:935-942 |
Clang 22.0.0 (Ubuntu snapshot 856555bf, 2025-10-15, older than the pin), with -O0 -Xclang -disable-O0-optnone -S -emit-llvm, on two small functions: madd returns a * b + c, and sum3 computes t = x + y, then v = t + z under #pragma clang fp reassociate(on). The ops it emits:
| Option | a * b + c |
t = x + y |
v = t + z, under the pragma |
|---|---|---|---|
-ffp-contract=on (default) |
llvm.fmuladd |
fadd |
fadd reassoc |
-ffp-contract=fast |
fmul contract, fadd contract |
fadd contract |
fadd reassoc contract |
-ffast-math |
fmul fast, fadd fast |
fadd fast |
fadd fast |
Clang records each setting on single instructions. It uses flags for -ffp-contract=fast and -ffast-math. For the default -ffp-contract=on, it uses llvm.fmuladd: on allows fusion inside one expression only (CGExprScalar.cpp:4699-4715), and Clang emits the llvm.fmuladd call with no flag (:4690-4692). The pragma changes only the op in its scope. The llvm.fmuladd case is the same idea as choice A: the op itself carries the permission.
Why per op:
- A per-function flag can only say what holds for every op in the function. Clang shows this for the one attribute that is left. It keeps
"no-signed-zeros-fp-math"true only if every scope in the function hasnsz(CodeGenFunction.cpp:167-174). - When LLVM inlines a function, the caller keeps
"no-signed-zeros-fp-math"only if both functions have it (Attributes.td:502, Attributes.cpp:2665-2669). So inlining one function without it removes the permission for the whole caller. Thenszflags on single instructions stay. Per-op flags move with the op when it is inlined, fused or tiled into other code. - The ops that carry flags in Linalg and Vector already have them per op (section 2.1).
Pass options like reassociate-fp-reductions fit this model. They give a permission at one point of the pipeline, and they write it as a per-op flag. We propose to keep them.
Appendix D: a mechanism to carry the flags
Details for section 3.5.
Why a mechanism cannot handle every rule:
- Merge needs the shared flags of several ops, and a check of the value-based flags.
- Drop: some rewrites must drop a flag (example 7). A mechanism that copies all the flags would keep it.
- Insert a value: the new value is usually a constant, which has no flags. But which value is right depends on the flags (example 4).
What exists today:
- LLVM:
FMFSourcegives a new instruction the flags of one instruction, or the flags that two share (IRBuilder.h:88-108).FastMathFlagGuardsets default flags for a scope (IRBuilder.h:372-399);ExpandReductionsuses it (ExpandReductions.cpp:67-76). - Flang: the
FirOpBuilderlistener (appendix C) puts the builder’s flags on every new op that hasArithFastMathInterface. If the builder has flags, they replace the flags that the op was built with (FIRBuilder.cpp:961-970). - MLIR:
ArithFastMathInterfacehas a getter, but no setter (ArithOpsInterfaces.td:18-50).vector.contract,vector.reductionandvector.multi_reductionhave it. LLVM dialect ops use another interface,LLVM::FastmathFlagsInterface(LLVMInterfaces.td:19).
How the guard would work.
- A pattern opens it with the source op. While it is open, every new op that has
ArithFastMathInterfaceand flagsnonegets the flags of the source op. Ops whose flags are notnonekeep them. Unlike Flang’s listener, the guard never replaces them. A pattern that must drop a flag on a new op uses the helpers instead. - The
fastmathattribute defaults tonone(ArithOps.td:86-88, VectorOps.td:64-66). So the guard cannot tell “not set” from “set tonone”, and it acts onnone. - With the greedy driver, the guard can be a rewriter listener (Builders.h:288-301). It wraps the current listener with
RewriterBase::ForwardingListener(PatternMatch.h:428-431). It sets the flags before it forwards an insertion, skips moved ops, and puts the old listener back when the scope ends. - Dialect conversion hides
setListener(DialectConversion.h:1039-1040). In its usual mode, it tells its listener about new ops only after the conversion succeeds (DialectConversion.h:1401-1408), so after the guard has closed. Conversion patterns, including the lowering to the LLVM dialect, use the helpers. A guard for them would need a new hook.
Tests. Every pattern that creates float ops gets a test with flags.