I’d like to propose two things, in order. This is an early RFC. A prototype of both exists, nothing is proposed for landing yet, and what I’m after is a judgment on whether the shape is viable.
First, declare LLVM’s optimization remark kinds in TableGen and generate one typed builder per kind, so that a remark’s name, outcome, message and arguments are stated once and every channel renders them from that declaration.
Second, add a metadata kind, !remark, that attaches such a remark to the instruction it describes. The plumbing carries it through the middle end and instruction selection onto MachineInstr, where a backend can write it out next to the code. Part 1 stands on its own; part 2 depends on it. Both build on mechanisms the compiler already has, the TableGen declarations Clang uses for its diagnostics, the middle end’s metadata attachments and their merge rules, and the !pcsections carry through instruction selection, and put them to work for remarks.
Motivation
Engineers read optimization remarks to learn what the compiler did to their code and what it declined to do, through -Rpass on the console or opt-viewer over the side file. A second reader is arriving, and we are evaluating it: AI agents and other tools that take the records as input and act on the code or on the compiler’s parameters. A record written for a person to skim is not enough for a tool. It needs each remark to have a declared meaning it can rely on, and a tie to the instruction the remark describes, so that a profile can be joined to it. The producers are widening too: front ends and DSL compilers that lower to LLVM IR have decisions of their own to report, and no channel through LLVM for them. None of this exists.
Background
Today a pass emits a remark by building an OptimizationRemark inside a lambda. It passes a pass name, a remark name typed by hand, a location, and streamed pieces of text and ore::NV pairs. The loop unroller’s success remark reads:
ORE->emit([&]() {
return OptimizationRemark(DEBUG_TYPE, "FullyUnrolled", L->getStartLoc(),
L->getHeader())
<< "completely unrolled " + LoopKind.str() + "loop with "
<< NV("UnrollCount", ULO.Count) << " iterations";
});
About 500 sites under llvm/lib emit a remark this way. The record that reaches the side file has structure, a pass, a name, a location and key-value arguments, but nothing declares what may go in it. The name and the keys are string literals typed at each site, so nothing checks them; two passes already use UnableToFindTarget at the same outcome for different facts. The values are strings, and the message is text fragments around them rather than a template. So no tool can list what the compiler is able to say, and consumers such as opt-viewer and llvm-remarkutil key on names and keys that hold only by convention.
The remark also exists only between the pass that emits it and the handler that prints it. What survives is a console line or a side file keyed on a source location: a file, a line and a column, with no inlining chain. That names a place in the source, not an instruction. Inlined and unrolled copies share one key, and a loop’s remark sits on the loop’s first line while the hot instruction is somewhere in the body. -remarks-section marks the limit: on MachO it embeds the side file’s path, not the remarks. So a profiler can show a hot instruction and say nothing about why it is there. Where LLVM’s output feeds another compiler, as with PTX, SPIR-V, DXIL and WebAssembly, the compiler that schedules and allocates registers never sees LLVM’s reasoning either.
We want a middle-end fact to reach the instruction it is about and leave the backend with it. That needs a well-defined schema first, and part 1 is that schema.
Part 1: declare remark kinds in TableGen
Clang shows what a declared diagnostic buys. Every Clang diagnostic is a TableGen def with a stable identity, a message template with typed holes and a generated reference page. LLVM’s remarks get none of that. Clang declares them only as an envelope, one diagnostic per outcome in clang/include/clang/Basic/DiagnosticFrontendKinds.td with a single %0 hole for the already-rendered string. The remarks library’s author listed TableGen-structured remarks and per-remark documentation of possible actions and reasons as open work at the 2019 developers’ meeting, and the library still carries a FIXME asking for typed values. Part 1 is that work, on the LLVM side where the remarks are made.
A kind is one def in llvm/include/llvm/IR/OptRemarks.td: a stable local name, an outcome, a message template with named holes, and the typed argument list:
def LoopUnrollFullyUnrolled
: OptRemark<"FullyUnrolled", Passed,
"completely unrolled %{LoopKind}loop with %{UnrollCount} "
"iterations",
[RemarkArg<"LoopKind", "str">, RemarkArg<"UnrollCount", "u32">]> {
let Scope = Region;
}
A new TableGen backend, -gen-opt-remarks, emits the tables and one typed builder per kind, here optremarks::annotateLoopUnrollFullyUnrolled. The outcome is one of passed, missed, failed and analysis, the four the diagnostic classes already distinguish. Argument types are integers, floats, strings, integer arrays and IR values. The generator prefixes llvm/ to every name. The def is the only place the name, the keys, the types and the wording exist: TableGen rejects a duplicate name or a hole without an argument, the C++ compiler rejects a call with the wrong arguments, and every channel renders from the same template.
This is a region kind, so the unroller’s site passes the loop’s region:
ORE->emit(DEBUG_TYPE, optremarks::annotateLoopUnrollFullyUnrolled(
loopRemarkRegion(*L), LoopKind, ULO.Count));
An instruction kind takes the instruction itself; its def leaves Scope out, since Instr is the default. AtomicExpand’s remark for an atomic it lowered to a compare-and-swap loop, with the atomicrmw as the anchor, becomes:
ORE.emit(DEBUG_TYPE, optremarks::annotateAtomicExpandedToCmpXchgLoop(
AI, AI->getOperationName(AI->getOperation()), MemScope));
The Scope field is what separates the two. An Instr kind reports its instruction’s own location and block. A Region kind, such as every loop remark, takes a small struct with the location and block the site wants reported, a loop’s start and header or a function’s line and entry block. Converting a pass therefore does not move where its remarks point.
OptimizationRemarkEmitter gains one overload that takes the builder’s return value and turns it into the ordinary OptimizationRemark, OptimizationRemarkMissed or OptimizationRemarkAnalysis. The hotness threshold, the pass-name filters, the YAML and bitstream serializers and the diagnostic handler therefore see exactly what they see today. MachineOptimizationRemarkEmitter gets the same overload for machine passes, and every kind’s builder is generated twice, once per anchor type. Most kinds need only one of the pair: AtomicExpand’s remark has no machine anchor and GlobalISel’s has no IR one. A field on the def naming the layer could restrict the pair to the side the pass runs on, which the prototype leaves for later. The whole API surface a pass sees is the builder pair and the two overloads:
// generated into OptRemarks.inc, one pair per kind
DeclaredOptimizationRemark
annotateAtomicExpandedToCmpXchgLoop(Instruction *I, StringRef Arg0, StringRef Arg1);
DeclaredMachineOptimizationRemark
annotateAtomicExpandedToCmpXchgLoop(MachineInstr *MI, StringRef Arg0, StringRef Arg1);
DeclaredOptimizationRemark
annotateLoopUnrollFullyUnrolled(const RemarkRegion &R, StringRef Arg0, uint32_t Arg1);
DeclaredMachineOptimizationRemark
annotateLoopUnrollFullyUnrolled(const MachineRemarkRegion &R, StringRef Arg0, uint32_t Arg1);
// llvm/Analysis/OptimizationRemarkEmitter.h
void emit(const char *PassName, optremarks::DeclaredOptimizationRemark &&R);
// llvm/CodeGen/MachineOptimizationRemarkEmitter.h
void emit(const char *PassName, optremarks::DeclaredMachineOptimizationRemark &&R);
What consumers see
The console text does not change for a converted site whose template is its old wording. Two things always change in the YAML and bitstream records. Name: becomes the namespaced kind, 'llvm/FullyUnrolled', and every hole of the template becomes a named argument where the hand-written remark had a run of text. -Rpass, -pass-remarks-output, opt-viewer and llvm-remarkutil keep working unchanged. A script that matches on the old bare names needs the prefix, a mechanical fix, and gains argument keys it can read instead of parsing a sentence. Where one hand-written name covered several facts we split it, with a family segment in the name. The greedy allocator’s LoopSpillReloadCopies becomes llvm/LoopSpillReloadCopies/Spills, /Reloads and so on, one record per statistic.
Why not keep the streaming interface?
Because the << form with its ore::NV pairs has no schema to check against and no identity beyond the string. A converted site is no longer than the lambda it replaces. The declaration behind it is what makes the rest possible: a verifier that knows a kind’s argument types, a side-file reader that knows them too, and a metadata node that carries them typed. A catalog is also one place to read. Today the compiler’s user-facing remark text is scattered over hundreds of sites in dozens of files, so nobody reviews it as a whole. With every kind in one file, wording and naming can be made consistent in one review, and a reference page generated as Clang’s is. A downstream that carries local changes can audit everything its compiler says in one place, and sees its divergence from upstream’s remarks as the diff of one file.
Costs
The declared form does more work than the lambda when remarks are off. The hand-written ORE.emit([&] { ... }) never runs its lambda then; the declared form packs the arguments and builds the event, and only then learns that nobody is listening. We measured that difference against the pre-conversion compiler with remarks off. On passes built to fire 100,000 remarks and do nothing else it is a few microseconds per firing site, 2 to 8 percent of the pass, and no measurable memory. On a real compile it is noise: APInt.cpp at O2 fires about 1,500 declared remarks in a 3.5-second compile, at most 6 ms.
Converting a pass is a visible change for that pass’s consumers. Each pass therefore converts in its own commit with its tests re-baselined. The conversion itself is quick: an LLM agent working from a reviewed plan converted the passes listed under where this stands, plus fifteen target passes of our own, in one day: twenty-seven commits, a few minutes per pass once the infrastructure was in. What takes time is a human reviewing each pass’s visible change.
Part 2: carry remarks as !remark metadata
A remark becomes a node in a slot of its own, MD_remark. The slot holds a tuple of remark nodes, so one instruction may carry several. Each node begins with the namespaced kind name and the outcome, then the typed arguments:
br i1 %cmp.1, label %loop, label %exit, !llvm.loop !5, !remark !0
!0 = !{!1}
!1 = !{!"llvm/PartialUnrolled", !"passed", !"", i32 2, !""}
For a declared kind the template comes from the generated table, so operand 2 is already the first argument: here the loop-kind prefix, empty for a plain loop, then the factor, then the run-time suffix, also empty here. A kind LLVM does not declare puts its template inline and its arguments after it. That is how a front end that feeds LLVM IR carries its own vocabulary:
!1 = !{!"myfe/loop.tiled", !"passed", !"loop tiled %{m}x%{n}", i32 4, i32 4}
Any reader of the node is LLVM code with the catalog in hand. It resolves the name first and reads operand 2 accordingly; an undeclared kind carries its template, so it needs no catalog. The namespace decides nothing about the shape; it only lets a consumer tell, if it cares, whether a remark came from LLVM’s own passes or from a front end that lowered into LLVM IR.
The verifier accepts only argument shapes that can always be persisted, and rejects a reference to a Value above all. A remark can therefore never point at an instruction a later pass deletes. The node round-trips through bitcode and textual IR.
Attachment is behind one global switch, -structured-remarks, default off. With it off the builders of part 1 only stream, so an unflagged compile’s IR is byte-identical to today’s. Finer selection, per kind or per pass, can be added on the same gate later.
What carries it
The middle end needed one change. combineMetadata in Local.cpp, which every CSE, GVN, hoist, sink and SimplifyCFG merge goes through, drops unknown kinds. Remarks union there instead, because a remark asserts nothing about the survivor and both records stay true of the instructions it replaced. Hanging the record off the instruction’s DILocation would tie it to debug info: without -g there is no location to hang it on, and merged locations keep what they share, which is right for a coordinate and lossy for a record. Cloning is free, so inlining and unrolling preserve remarks; an instruction folded to a constant or deleted as dead takes its remarks with it. Under LTO, module linking keeps the attachments of the definitions it keeps and drops those of the copies it discards, as it does their debug info; a discarded copy’s remarks describe code that no longer exists, and the LTO backend attaches its own to the code that survives.
Instruction selection reuses what !pcsections built: SelectionDAG’s per-node side table, and GlobalISel’s builder state seeded by the IR translator. Both land in a slot on MachineInstr, which MIR spells as remark !0.
Anchors
A fact about a loop has no instruction it is about. We anchor loop remarks on the single latch terminator, where !llvm.loop already lives, and keep reporting the loop’s start location and header block as the hand-written remark did. A Region builder attaches only when there is an anchor.
What a backend does with it
That is the backend’s decision; this RFC fixes only that the fact arrives with the instruction. A target can write the remarks into the binary in a section format of its own. It can stream them as a side channel with the extra information only the backend has, the final instruction and its offset. Or it can hand them to analytical passes late in its own pipeline.
Costs
Instructions without remarks stay free. An annotated one pays a little at every DAG replacement and in its MachineInstr’s storage, and the module pays once for each distinct remark, where two remarks are the same when their kind, outcome and argument values all match and are then stored as one node. Two shapes bound the cost:
- Shared remark. The arguments repeat across sites, an opcode name or a handful of unroll counts, so any number of sites share a few remarks. Measured on LICM, whose hoists all carry the one
Hoistedremark, and on synthetic modules with one remark. - Unique remark. The arguments differ at every site, a callee name, a cost, a trip count, so every site has its own. Measured on the inliner, whose refusals each carry their own callee and caller names, and on synthetic modules with a fresh integer or string per remark.
Each cell below is the growth over the same run with attachment off; the 100,000 column also gives the size without remarks.
Bitcode size
| remarks attached | 1,000 | 10,000 | 100,000 |
|---|---|---|---|
| shared remark (LICM) | +16 KB | +160 KB | +1.6 MB on 17 MB |
| unique remark (inliner) | +70 KB | +0.7 MB | +7.2 MB on 19 MB |
| shared remark (synthetic) | +16 KB | +160 KB | +1.6 MB on 8.5 MB |
| unique remark (synthetic) | +47 to 97 KB | +0.5 to 1.0 MB | +5.2 to 9.7 MB on 8.5 MB |
Peak resident memory
| remarks attached | 1,000 | 10,000 | 100,000 |
|---|---|---|---|
| producing compile, shared remark (LICM) | within noise | +0.2 MB | +1.6 MB on 3.7 GB |
| producing compile, unique remark (inliner) | +0.4 MB | +2.4 MB | +48 MB on 0.8 GB |
| reader loading the module, shared remark (synthetic) | +0.3 MB | +0.3 MB | +3 MB on 217 MB |
| reader loading the module, unique remark (synthetic) | up to +0.4 MB | +3.5 to 5.4 MB | +32 to 57 MB on 217 MB |
Pass time is unchanged below 100,000 remarks and rises under one percent at 100,000 with shared remarks, a tenth of the inliner’s pass with unique ones. Those modules are small for their remark counts, so the relative growth overstates a real module’s: LLVM’s own APInt.cpp at O2 attached 1,388 remarks, 322 of them distinct, and grew 3 percent, with no measurable change in peak memory or wall time. In MIR there is no choke point for a union, so a merging pass keeps one side’s attachment. That loses entries and never adds a false one, since the attachment is a log of what passes decided.
Where this stands
Both parts exist as a prototype, with lit and unit tests. We converted the remark sites of the passes our pipelines run, 134 sites: the unroller family, the inliner core, LICM, GVN’s load PRE, tail recursion elimination, loop-idiom, loop deletion, AtomicExpand, GlobalISel’s failure remarks and the greedy allocator’s per-loop spill summary. The catalog holds 116 kinds, and the carry runs end to end through one backend. If the shape holds up, part 1 can land with no output change and part 2 behind its flag, each pass conversion as a patch of its own.
Feedback
I’m asking for a judgment on the direction, not a review of these patches. For part 1, I’d like to hear from people who own or consume the remark infrastructure (opt-viewer, llvm-remarkutil, the YAML readers) whether a declared catalog with namespaced names and typed arguments is something they would want. I’d also like to hear from pass owners whether they would accept their remarks being converted. For part 2, I’d like to know whether a metadata carry is acceptable in principle, with the combineMetadata union as the point most worth pushing on, and from backend owners whether a target of theirs would use the fact once it arrives. If you maintain a front end with its own diagnostics, I’d like to know whether the undeclared-node shape is one you could target. If the answer to any of these is no, I’d rather learn it now.
