RFC Basic Sample Support in llvm-profgen

,

Summary

Extend llvm-profgen to generate LLVM sample profiles directly from basic perf events that contain sampled instruction pointers (IPs).

Background

The LLVM profiling ecosystem currently relies on the out-of-tree AutoFDO create_llvm_prof tool to generate sample profiles from sampled-IP perf data without branch stacks. This leaves users dependent on a separate tool and profile-generation pipeline that may not remain in step with upstream LLVM.

llvm-profgen initially generated profiles from x86 Last Branch Record (LBR) branch stacks. More recently, support for decoding Arm Embedded Trace Macrocell (ETM) traces into LLVM sample profiles was merged. This establishes llvm-profgen as infrastructure for more than one profile source.

Problem

llvm-profgen supports branch-stack and ETM inputs, but it cannot directly consume basic sampled-IP perf events. This gap has also motivated community demand for upstream support in llvm-profgen.

Proposal

Add an explicit --basic-events mode, with -ba as an alias, for perf input containing sampled instruction pointers.

In this mode, llvm-profgen should accept either raw perf.data or perfscript. Generating the source profile also requires the profiled binary and its debug information. Perfscript input should contain sampled IPs, process IDs and memory-mapping records, with task records describing user-space process-lifetime changes. The mode should be architecture-neutral: any architecture that can provide input in this form should use the same path.

The result should be a normal, flat, line-number LLVM sample profile, combining samples across runtime callers in each function’s profile. It should provide function and source-location information through the existing sample-profile path and inform inlining, hot/cold classification, and block or function weighting.

Branch-stack-based perf samples contain recently taken branches from which llvm-profgen derives weighted ranges, branch edges, and call targets. Basic-event profiles can preserve the nesting of functions inlined into their callers, using the binary’s debug information.

--skip-symbolization should save intermediate instruction-address counts. --unsymbolized-profile should reload them for conversion to function and source-location samples.

When the implementation has been validated, the user manual’s section on using sampling
profilers
should be updated.

Suggested Implementation

Add PerfContent::BasicEvent to the existing content enum and introduce BasicEventPerfReader, selected by --basic-events. Derive it from PerfScriptReader to reuse the perfscript event loop. For perf.data, extend the existing perf script invocation to request sampled IPs, process IDs, mappings and task events.

AutoFDO’s sampled-IP reader uses per-process mappings to resolve samples before combining counts for the target binary. Basic-event mode should provide this support to cover the same workflows, including recordings with multiple processes, relocated binaries and forked children.

Add an ExecutableMapping structure to record runtime address ranges and their translation into addresses in the profiled binary. Store mappings per process and add task state and a resolveMappedAddress() helper to PerfScriptReader, which owns mmap-event processing. The new reader overrides parseSample() to filter by PID, resolve instruction addresses
and accumulate counts. Track process lifetime and inherited mappings through perf task events. Kernel samples use mappings shared across processes.

The existing SampleCounter structure holds the address-based counts produced by a reader, including counts for instruction ranges and branch edges. Extend it with a BasicSampleCounter member using the new BasicSample address-to-count map type.

Use the existing ContextSampleCounterMap to pass the reader’s counters to profile generation. Each entry holds a SampleCounter for one calling context. The Basic reader should provide a single entry containing all instruction-address counts for the binary.

The existing ProfileGenerator maps binary-address counts to source functions and locations using debug information. Extend it so Basic counts share LBR’s per-instruction symbolisation and counting logic. LBR reaches this logic by walking instructions within its ranges; Basic supplies individual sampled addresses and counts. Use the line-number path for Basic input, including binaries containing pseudo-probe profiling metadata. The existing sample-profile writer then emits the LLVM profile.

Extend the existing writeUnsymbolizedProfile() and UnsymbolizedProfileReader to write and restore Basic counts. Add a header marker identifying the file as Basic input, so the reader selects the address-to-count layout. Preserve markerless LBR profiles.

Pull Request.

Open Questions

What terminology should this mode use? This RFC uses basic-events for sampled-IP perf events without branch stacks, but upstream may prefer another mode, flag, or perf-content name.

The command-line interface is already crowded. Should it use a sample-content selector, such as --perf-content=basic, instead of adding a dedicated --basic-events flag? Future profiling mechanisms, such as Arm Statistical Profiling Extension (SPE) on AArch64, may also require new input modes, so a multi-valued selector may be preferable.

Should the unsymbolised input identify Basic content with an in-band marker, or should the content type be selected explicitly on the command line?

@snehasish @SergeyShch01 @aaupov please have a look

Is this referring to SPE support in create_llvm_prof?

Characterizing the linked issue as “demand” is not accurate in my opinion. It is motivated by loose wording in the documentation and has no concrete example of what non-LBR support means.

What is the expected usage model of these basic events? For branch profiling, the existing format should suffice regardless of where it is derived from as demonstrated by ETM. For novel usages of the sample based information, such as software prefetching we would want to use to be demonstrated and merged upstream. I don’t think we want to carry the complexity of profile formats unless there is a use for it in upstream LLVM.

Thank you for taking the time to review the RFC.

Is this referring to SPE support in create_llvm_prof?

This is for PMU instruction-pointer samples without branch stacks. We also aim to support SPE, but that is separate from this RFC.

“demand” is not accurate in my opinion.

Apologies, I should have made clear that I was referring to the comments in the issue. zamazan4ik’s comment describes recurring problems building AutoFDO against newer LLVM versions and suggestes an in-tree alternative would help.

What is the expected usage model of these basic events?

The aim is to generate profiles for Clang’s existing sample PGO. I am not proposing changes to the main LLVM sample-profile format. The format change is only to llvm-profgen’s unsymbolised intermediate files, so the existing --skip-symbolization and --unsymbolized-profile path also works with basic input.

Thanks for putting this together. This support looks useful and would make LLVM’s profiling toolchain more self-contained by reducing its dependency on out-of-tree conversion tools.

A sample-content selector also seems reasonable, although it may be cleaner to introduce it in a separate preparatory commit. For terminology, I wonder whether ip-events would be more descriptive than basic-events.

One architectural consideration: could the proposed per-process mapping support be structured as common infrastructure? The existing LBR/BRBE-oriented path could potentially benefit from it as well and thereby cover additional scenarios. It need not be generalized to LBR/BRBE immediately, but designing it so that such an extension remains straightforward could help avoid parallel mapping mechanisms.

--basic-event sounds a bit too generic.. And also curious: what would be the use case for ip-only samples? Modern x86 CPUs almost all support branch sampling.. and for ARM why do we want SPE where we have ETM already supported?

There is now a separate community contribution adding Arm SPE branch-profile support to llvm-profgen: [llvm-profgen] Add Arm SPE branch profile support by SergeyShch01 · Pull Request #223237 · llvm/llvm-project · GitHub It looks related to the SPE support mentioned here while remaining separate from the basic-sample support proposed in this RFC.
BTW, ip-events looks good to me, though I wonder if ip-samples might be even clearer since this is about sampled IPs.

This is for PMU instruction-pointer samples without branch stacks.

Thanks for the clarification. We moved away from this approach since it doesn’t scale beyond simple CFGs often leading to performance regressions. Have you tried this on any large workloads recently? I think modern PGO in LLVM relies heavily on profile accuracy, particularly things like late stage function splitting can cause severe degradation if the profile is not accurate.

describes recurring problems building AutoFDO against newer LLVM versions and suggestes an in-tree alternative would help.

The build issues have been resolved, we now pin the repo to a specific LLVM version. It is in maintenence mode mostly as we plan to transition to upstream LLVM.

and for ARM why do we want SPE where we have ETM already supported?

ETM has higher overhead and I believe requires elevated privileges.

My understanding is that the basic ip samples support is essentially bringing the existing AutoFDO --use_lbr=false style workflow into the LLVM tree. SPE would then build on top of that by providing richer branch information, improving SamplePGO profile quality and providing the edges counters information Propeller needs for high-quality code layout on platforms where BRBE is unavailable, while BRBE could serve as the native AArch64 branch stack source for Propeller. Please correct me if I’m missing something or if you see the SPE and BRBE usage differently.

That’s great to hear. Would you be interested in bringing the profiling functionality from create_llvm_prof into llvm-profgen as part of that transition?

My understanding is that the basic ip samples support is essentially bringing the existing AutoFDO --use_lbr=false style workflow into the LLVM tree.

Thanks, I follow that. I’m trying to point out that it’s best left behind. For the hardware and intended usage my understanding is with basic ip samples we only get the block address (e.g precise pc on inst sampling), with SPE we get 1 entry with a branch from → to address pair. PBT adds another entry to the SPE format while BRBE is the full branch stack.

The basic ip sample approach didn’t work well in practice and supporting it in llvm-profgen (which is distributed widely) doesn’t sound appealing to me because of a couple of reasons:

  • Users trying out the basic functionality are let down by poor performance and are turned off from using PGO altogether.
  • Vendors skip SPE / BRBE support but hope that they support hardware assisted PGO since basic samples are supported.

@davidxl can share more historical context on this.

So I think we can proceed directly to the SPE proposed by @SergeyShch01.

Would you be interested in bringing the profiling functionality from create_llvm_prof into llvm-profgen as part of that transition?

Not for the basic samples support as noted above.

Google (Android specifically) is interested in using llvm-profgen; we’re kicking the tires to see what works and what we might be able to help with, but just wanted to signal that some folks at Google are interested in using llvm-profgen.

Somewhat curious about it’s relationship to llvm-bolt for data processing. Perhaps worth a conversation in person at the LLVM dev meeting if you plan to attend?

To @snehasish 's point, it is probably not worth adding support for basic IP sample (retired instruction). IIRC, we managed to get some improvement with SPEC, but it did not work well with real world workloads. What works well is LBR like branch stack sampling, or SPE like branch sampling, or ETM. By the way, there is a plan to upstream create_llvm_prof tool. @tmsriram @dhoekwater

Yes, I’ve chatted with @pirama regarding this recently. The out-of-tree create_llvm_prof tool is definitely a source of friction for Android partners. So our plan is roughly to consolidate usage around llvm-profgen and upstream the necessary pieces from create_llvm_prof. This does not include the non-lbr basic ip feature since it’s unmaintained at this point.

Thank you for the response. I do understand the concerns about user experience and vendor incentives; however, we have CPUs where only PMU sampling is available, and we’ve seen meaningful speedups:

Program AutoFDO This Patch
MongoDB 1.298× 1.288×
Clang 1.087× 1.070×
PostgreSQL 1.241× 1.240×
Python 1.084× 1.120×
MariaDB 1.181× 1.166×

Thanks for collecting the data! It’s quite counter to our intuition though I suspect that the improvement (at least in part) since profi is now enabled by default. Could you collect the numbers for this patch with profile disabled (-fno-sample-profile-use-profi)?

Also I would appreciate some details on the baseline. For example, is it using SPE? What were the collection parameters and the platform used? For the workloads, what was the training run and testing runs? Sharing the scripts to reproduce these numbers would be appreciated too.

Overall, I am much more positive on adding this support to llvm-profgen after seeing the numbers.

What processor is this? The sample attribution may be more precise.

Another factor is the improved debug information in the compiler.

Program AutoFDO Basic llvm-profgen Basic AutoFDO SPE
MongoDB 1.2981x / 1.2234x 1.2924x / 1.2131x 1.3387x / 1.3094x
Clang 1.0841x / 1.0494x 1.0685x / 1.0380x 1.1459x / 1.1273x
PostgreSQL 1.2067x / 1.1914x 1.2091x / 1.2093x 1.0317x / 1.0239x
Python 1.0835x / 1.1229x 1.1200x / 1.1467x 1.2070x / 1.1934x
MariaDB 1.1699x / 1.1449x 1.1639x / 1.1589x 1.2623x / 1.2617x

Profi Enabled / Profi Disabled

The baseline is the performance when profile guide optimisation is done. I used the same benchmarks for both training and testing: database read/write workloads, Python computation and data handling tests and LLVM/Clang builds. These workloads are part of our internal testing infrastructure, so it is difficult to produce reproduction scripts unfortunately.

I’d like to share two ideas:

  1. For basic samples, we might try to improve PMU profile accuracy by retaining more of the collected sample counts when estimating block weights, rather than using only the maximum per-instruction sample count in each block. One approach would be to sample a retired-branch event such as BR_RETIRED and sum the sample counts attributed to instructions within each block instead of taking their maximum. This would retain all sample contributions while avoiding the direct block-length dependency introduced by summing instruction-event samples. Skid would still redistribute samples across blocks, so the benefit would need evaluation, with and without Profi.

SPE precisely identifies individual sampled branch outcomes, including both taken and not-taken branches. We could preserve these edge weights directly and infer block weights through flow propagation. Converting edge counts into block weights and subsequently reconstructing edges can lose information and produce inaccurate estimates in some CFGs.

For edge-weight estimation, SPE may even be more accurate than BRBE. BRBE retains the last N taken branches, while reconstructed not-taken branches consume no buffer entries. Inclusion therefore depends on subsequent taken-branch density; correlations between outcomes and that density can bias the resulting counts.

I’ve opened a separate PR for the shared mapping support. It adds per-process mappings to LBR/BRBE, and the basic-sample reader would also be able to reuse the same code.

1 Like

I’ve rerun the benchmarks with rebased LLVM builds and the proposed SPE support. Worker samples are also now included.

Preliminary speedups over an unprofiled baseline, shown as Profi on / off:

Program AutoFDO Basic llvm-profgen Basic AutoFDO SPE llvm-profgen SPE
MongoDB 1.365x / 1.328x 1.342x / 1.349x 1.507x / 1.504x 1.539x / 1.504x
Clang 1.169x / 1.130x 1.158x / 1.120x 1.266x / 1.238x 1.267x / 1.240x
PostgreSQL 1.288x / 1.236x 1.297x / 1.268x 1.440x / 1.417x 1.418x / 1.422x
Python 1.184x / 1.150x 1.180x / 1.150x 1.218x / 1.210x 1.230x / 1.206x
MariaDB 1.187x / 1.153x 1.197x / 1.144x 1.280x / 1.253x 1.282x / 1.249x