When building a project with both thinLTO and FDO, I noticed the same FDO sample profile was read twice, once in SampleProfileLoader pass that deals with inlining for the current module, and again in MIRProfileLoader pass. This does not seem to be very efficient since it is the same file being read twice. Even if these two passes need to load different functions from samples, it still takes non-trivial amount of time to load the profile file, construct the function offset table and read profile symbol list. I am looking into preserving the sample profile reader instance across passes. What is the standard way in LLVM to keep a large data structure across passes?
once in SampleProfileLoader pass that deals with inlining for the current module, and again in MIRProfileLoader pass
This sounds like FSAFDO to me. I’m not familiar with its internals to tell if SampleProfileLoader pass result could be directly used (or re-computed from the original SPGO profile) by MIRProfileLoader pass though (as in, as transformations goes on, MIRProfileLoader means to make profile annotation more accurate for passes after it)
I am looking into preserving the sample profile reader instance across passes. What is the standard way in LLVM to keep a large data structure across passes?
I think the standard way to preserve result across passes is an analysis pass. The New Pass Manager - The LLVM Project Blog describes about how an analysis pass works (e.g. caching and invalidating results).
Yes, keeping the results as an analysis is the way to go. In this case, make sure it will not get invalidated by making it a stateless analysis. For example, see GlobalsAA’s invalidate().