[RFC] Faster Sample Profile Loading

IR & Optimizations
4

As I’ve highlighted above, this project has two components – speeding up SecNameTable and SecFuncOffsetTable. The former is being addressed as:

I’d like to revise the proposal for the latter – SecFuncOffsetTable – as I’ve learned more details.

[RFC] Faster Sample Profile Loading (Update)

We continue to use the on-disk hash table for function offsets, but we won’t integrate SecLBRProfile.

Design & Compatibility

  • Format Version v104: To support the on-disk hash table without breaking backward compatibility for older compilers, we are introducing a new profile format version, v104.
  • Separation of Index and Data: We do not integrate the SecLBRProfile (the actual profile data) into the hash table. The hash table acts strictly as a lightweight index, mapping a 64-bit function name GUID to a fixed 32-bit byte offset pointing into the SecLBRProfile section. This avoids the complexity of dealing with the variable-length serialization of FunctionSamples inside the hash table payload.

Storage & Memory Impact

  • Storage Cost: The index structure introduces a minor storage overhead. For a 1.2GB profile (containing ~600k total symbols, with ~96k written to the flat offset table), the total file size increases by roughly 10MB (~0.8%).
  • Heap Savings: In the older format, the reader must parse the entire offset table at startup to construct a DenseMap in memory. For a profile with ~600k symbols, this map alone consumes 16MB of heap. For a flat profile of this size, v104 completely eliminates this 16MB allocation. For split CS profiles, we save the heap for the flat section index (~2MB for the ~96k flat symbols).
  • Startup Speedup: Based on our measurements, parsing the offset table takes up to 8.88% of the total compilation time for shared profiles (XFDO) and 1.64% for non-shared profiles. By querying the on-disk hash table directly, we expect to recover almost all of this overhead during compiler startup.

Version Handling

  • Default to v103: The writer will continue to default to v103 to ensure no impact on existing workflows unless explicitly requested.
  • Opt-in to v104: Generation of the new format is opt-in via a new hidden command-line flag (-sample-profile-format-version=104).
  • Transparent Reader: The reader supports both v103 and v104 transparently.

Compression Restriction (for Mmap Efficiency)

To preserve zero-copy mmap loading, we disable compression on the hash table section (even if global compression is enabled). The reader will reject compressed hash tables as malformed. MD5 GUIDs are incompressible, and compression defeats zero-copy mmap. The SecLBRProfile section itself remains compressible.

Phased Patch & Rollout Plan

To ensure a safe deployment and keep code reviews manageable, we plan to split the work into three sequential phases:

  • Phase 1: Hash Table Infrastructure (Off by Default)
    Introduce the serialization helper classes (traits) for the on-disk chained hash table, verified with isolated unit tests (~100 lines of library changes). This is a pure library addition with no functional changes to FDO yet.
  • Phase 2: Integration & Production Verification (Opt-in)
    Integrate the helpers into SampleProfReader and SampleProfWriter, implement compression restrictions, and enable the v104 format under a hidden flag (-sample-profile-format-version=104) (~150 lines). This allows testing and verifying the new format in large-scale production environments on an opt-in basis before enabling it globally.
  • Phase 3: Flip Default (Default On)
    Flip the default format version in SampleProfWriter.cpp from 103 to 104 (~5 lines), making the new format the default for all users after successful production verification.