[CSSPGO] Unblocking pseudo-probe-safe optimizations

A pseudo probe is implemented as a dedicated intrinsic call with the IntrInaccessibleMemOnly attribute which intentionally blocks optimizations which merges code for better counts quality but i’ve noticed it also accidentally blocks few seemingly not so harmful optimizations like Loop Vectorization of loops with early exits and LoopChain. I wanted to ask the following :
→ Is it worthwhile to systematically find such accidentally-blocked optimizations and unblock them?
→ I understand transformations that merge or delete blocks degrade profile quality – are there other transformation classes that degrade pseudo-probe profile quality?
→ What’s the recommended way to measure the impact of such a change – are there preferred benchmarks?

Test Case :

; RUN: opt -S < %s -p pseudo-probe,loop-vectorize -debug-only=loop-vectorize -force-vector-width=4 -disable-output 2>&1 | FileCheck %s

declare void @init_mem(ptr, i64);

define i64 @loop_contains_safe_div() {
; CHECK-LABEL: LV: Checking a loop in ‘loop_contains_safe_div’
; CHECK: LV: Found an early exit loop with symbolic max backedge taken count: 63
; CHECK-NEXT: LV: We can vectorize this loop!
entry: %p1 = alloca [1024 x i8]
%p2 = alloca [1024 x i8]
call void @init_mem(ptr %p1, i64 1024)
call void @init_mem(ptr %p2, i64 1024)
br label %loop

loop:
%index = phi i64 [ %index.next, %loop.inc ], [ 3, %entry ]
%arrayidx = getelementptr inbounds i32, ptr %p1, i64 %index
%ld1 = load i32, ptr %arrayidx, align 1
%div = udiv i32 %ld1, 20000
%cmp = icmp eq i32 %div, 1
br i1 %cmp, label %loop.inc, label %loop.end

loop.inc:
%index.next = add i64 %index, 1
%exitcond = icmp ne i64 %index.next, 67
br i1 %exitcond, label %loop, label %loop.end

loop.end:
%retval = phi i64 [ %index, %loop ], [ 67, %loop.inc ]
ret i64 %retval
}

Output
LV: Checking a loop in ‘loop_contains_safe_div’ from
LV: Loop hints: force=? width=4 interleave=0
LV: Found a loop: loop
LV: Found an induction variable.
LV: Did not find one integer induction var.
LV: Not vectorizing: Writes to memory unsupported in early exit loops.
LV: Interleaving disabled by the pass manager
remark: :0:0: loop not vectorized: Cannot vectorize early exit loop with writes to memory
LV: Not vectorizing: Cannot prove legality.

Same loop vectorizes if we remove pseudo-probe optimization in the run line i.e.
opt -S < %s -p loop-vectorize -debug-only=loop-vectorize -force-vector-width=4 -disable-output 2>&1 | FileCheck %s

Tagging @WenleiHe @modimo @wlei @apolloww

Hello, and thank you all for the excellent work on CSSPGO and pseudo probes. We’ve observed some nice performance gains on several internal AMD workloads after enabling CSSPGO.

As part of our downstream validation effort, we’ve recently started evaluating a number of amd-specific optimizations under CSSPGO and noticed that some of them are currently blocked due to pseudo probes. While looking into this, I also noticed the optimizations discussed in this RFC appear to have been similarly restricted in upstream LLVM.

I was hoping to get your perspective on whether these optimizations should be unblocked. My current understanding is that these transformations do not merge or eliminate probe-carrying blocks and therefore should not significantly impact profile quality or profile reconstruction. However, I’m not deeply familiar with all of the design assumptions behind pseudo probes and CSSPGO, so it’s entirely possible I’m overlooking something important.

Thank you again for your work on this infrastructure and for any guidance you can provide.

Great to hear that you saw perf gains with CSSPGO.

You are right that merge is the main transformation we want to block with pseudo-probe and vectorization and unrolling should not blocked. The litmus test is to see whether a transformation impacts profile count to unoptimized IR correlation/attribution in an unrecoverable way. If not, we should not block; otherwise, it’s desirable to block unless the optimization’s own benefit overweigh the benefit of improved profile quality. There are tricky cases like if-convert too where the answer isn’t clear cut, and we need to evaluate case by case.

If you find important optimizations that are blocked, but it shouldn’t because it doesn’t hurt profile quality, patches are welcomed!

Hi @chs_chandan thanks for the effort in identifying CodeGen differences.

Is it worthwhile to systematically find such accidentally-blocked optimizations and unblock them?

It is definitely worth the effort to identify differences, as block important optimization like vectorization can result upto 30% perf regressions on key workloads.

In terms of benchmark, in llvm-test-suite, SPEC 2017 and 2026, there aren’t any remaining large performance regression with pseudoprobes enabled on AArch64. But there are still a few cases where pseudoprobes are blocking optimizations. I have a few patches that I haven’t gotten around to validating quite yet, for example [loop-vectorize][PseudoProbe] Unblock pseudo probe in predicated block by mustartt · Pull Request #197588 · llvm/llvm-project · GitHub would require dropping probes to unblock predicated vectorization. Case like this would require a more careful analysis of the profile quality impact vs performance progression.
But there are still a handful of cases, where teaching optimization results in no profile quality. Those should be resolved if possible.