PM: High-level review of the new Pass Manager (so far)

Hi Chandler,

This is a high-level review of the new WIP `PassManager` infrastructure.

For those that haven't dug into Chandler's commits, here's a very
high-level overview (assuming IIUC):

  - The driver supports simple declarative syntax for specifying passes
    to run. E.g., `module(a,b,function(c,d),e)` runs module passes `a`
    and `b`, then function passes `c` and `d` for each function, and
    then module pass `e`.

  - There is a new concept of an AnalysisManager (AM), which runs
    analyses on-demand and caches the results.

  - Analysis passes are split conceptually from transformation passes.
        - Analysis: run: (IRUnit*, AM*) -> Result
        - Transformation: run: (IRUnit*, AM*) -> PreservedAnalyses

    Note that neither of these matches the old interface, which was
    runOnIRUnit: IRUnit* -> bool.

  - PassManagers interoperate via adaptors. E.g.,
    ModuleToFunctionPassAdaptor is a module transformation pass that
    contains a FunctionPassManager (with some set of function passes).

  - AnalysisManagers interoperate via proxies. E.g.,
    FunctionAnalysisManagerModuleProxy is a module analysis pass that
    forwards to a FunctionAnalysisManager.

  - LazyCallGraph and ModuleToPostOrderCGSCCPassAdaptor collude to visit
    SCCs in post-order, including API for updating the SCC-graph
    on-the-fly without invalidating the traversal.

I think what's done generally looks great. In particular, the
declarative syntax is awesome and it's a huge step to have an
AnalysisManager that can reason about analyses separately from the pass
pipeline. I haven't analysed the algorithms in LazyCallGraph carefully,
but I think it's great that they keep the traversal valid amid a
changing call graph.

Questions and concerns:

1. PassConcept requires a `name()` for passes. Why don't
    AnalysisPassConcept and AnalysisResultConcept?

2. AnalysisManagerBase defines two versions of `invalidate()` -- one of
    which takes a `Module*` (instead of `IRUnitT`). It's not clear to
    me why the analysis managers (other than ModuleAnalysisManager) need
    this. What am I missing?

3. The number of lines of code to enable transformation pass managers
    (and analysis managers) to interoperate seems to scale quadratically
    in the number of types of IRUnit. Am I reading this wrong? Is
    there a plan to manage that, or is it an open problem? (So far the
    infrastructure only supports Module, SCC, and Function. What
    happens (e.g.) when we add BasicBlock passes?)

4. Previously, (e.g.) function analysis passes would typically hold
    results only for one function -- when the next function was
    analyzed, the previous function's results would be invalidated.
    Now, the FunctionAnalysisManager will happily store results for
    every function. (Looking forward, BasicBlockAnalysisManager might be
    able to store results for every BasicBlock.)

    This is powerful, but costs memory. What's the plan for keeping
    memory usage in check (especially for large modules)?

5. In the old infrastructure, `CGPassManager::runOnModule()` (in
    `CallGraphSCCPass.cpp`) has a do-while loop that re-runs all passes
    on a given SCC as long something new gets devirtualized, up to a
    maximum number of iterations. I don't see the equivalent loop in
    CGSCCPassManager, and I don't see an obvious way to accomplish the
    same optimization cycle.

      - How will we re-implement this loop? (Or did I miss it?)
      - There's a potential infinite loop between devirtualization and
        inlining on recursive virtual functions. How will we avoid it?

6. There aren't any machine-level pass managers yet. Will they look
    about the same? (Are there open problems to solve there?)

7. Andy asked in a separate thread [1] (my words): "Does this
    infrastructure allow for invalidation of classes of analyses?"
    (E.g., invalidate all analyses whose results depend on the
    control-flow graph.) Your response there [2] was (my words): "Not
    directly, but `AnalysisManager` will be a good place for that API."

    I thought a little about this, and it's not clear to me how to fit
    this API into AnalysisManager cleanly.

    One side seems straightforward. Assuming you have a transformation
    pass that modifies the CFG, tell the AnalysisManager that you've
    modified the CFG, and return that all analyses have been preserved
    (since everything else has). This side is easy to make opt-in.
    
    The other side is not opt-in, though. *All* analyses that depend on
    the CFG must respond, or the invalidation mechanism doesn't work.

8. `CGPassManager` was one place where the old return value for a
    transformation pass (`bool` representing "did-change") was useful.
    Are there any others? Is the API using `PreservedAnalyses::all()`
    as a stand-in for "did-not-change"? If so, does that conflict with
    adding more nuanced API for on-the-fly invalidation of analyses?

-- dpnes

[1]: http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-June/073767.html
[2]: http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-June/073785.html

First off, sorry for not circling back to this sooner. I think it managed to get caught in some part of the weird mail bounces from one address and I had to find it in my other account.

Also, more improtantly, I got distracted. But I’m back to hacking on this now and wanted to close the loop here.

Hi Chandler,

This is a high-level review of the new WIP PassManager infrastructure.

For those that haven’t dug into Chandler’s commits, here’s a very
high-level overview (assuming IIUC):

  • The driver supports simple declarative syntax for specifying passes
    to run. E.g., module(a,b,function(c,d),e) runs module passes a
    and b, then function passes c and d for each function, and
    then module pass e.

  • There is a new concept of an AnalysisManager (AM), which runs
    analyses on-demand and caches the results.

  • Analysis passes are split conceptually from transformation passes.

  • Analysis: run: (IRUnit*, AM*) → Result

  • Transformation: run: (IRUnit*, AM*) → PreservedAnalyses

Note that neither of these matches the old interface, which was
runOnIRUnit: IRUnit* → bool.

Minor note for others: the PreservedAnalyses can be computed conservatively from the bool by selecting between all (false in the old interface) and none (true in the old interface)

  • PassManagers interoperate via adaptors. E.g.,
    ModuleToFunctionPassAdaptor is a module transformation pass that
    contains a FunctionPassManager (with some set of function passes).

  • AnalysisManagers interoperate via proxies. E.g.,
    FunctionAnalysisManagerModuleProxy is a module analysis pass that
    forwards to a FunctionAnalysisManager.

  • LazyCallGraph and ModuleToPostOrderCGSCCPassAdaptor collude to visit
    SCCs in post-order, including API for updating the SCC-graph
    on-the-fly without invalidating the traversal.

I think what’s done generally looks great. In particular, the
declarative syntax is awesome and it’s a huge step to have an
AnalysisManager that can reason about analyses separately from the pass
pipeline. I haven’t analysed the algorithms in LazyCallGraph carefully,
but I think it’s great that they keep the traversal valid amid a
changing call graph.

Questions and concerns:

  1. PassConcept requires a name() for passes. Why don’t
    AnalysisPassConcept and AnalysisResultConcept?

This was just me implementing things as I needed them. I hadn’t done the analysis debug logging, and so hadn’t needed a name yet. I’ve added that at this point for analysis passes (along with the name) but not for results. I’m not sure results need a name, the pass having a name seems sufficient to me.

  1. AnalysisManagerBase defines two versions of invalidate() – one of
    which takes a Module* (instead of IRUnitT). It’s not clear to
    me why the analysis managers (other than ModuleAnalysisManager) need
    this. What am I missing?

There are two ways invalidation happens, and they require different interfaces.

a) A transformation pass returns some set of preserved analyses, and all others are invalidated by the pass manager. The pass manager can’t enumerate the analysis passes and so just hands the preserved set and the IR unit to the analysis manager.

b) A transformation pass directly invalidates a specific pass for its IR unit, potentially to re-compute a fresh version while running. This can be particularly important for cross-domain analyses. As a toy example, you could imagine having the inliner invalidate and recompute the domtree for the caller post inlining if it used the domtree over the caller to analyze the call sites.

This may be a bit more clear now that the “invalidate” utility passes are in place, but if not, I should add some comments to this effect.

  1. The number of lines of code to enable transformation pass managers
    (and analysis managers) to interoperate seems to scale quadratically
    in the number of types of IRUnit. Am I reading this wrong? Is
    there a plan to manage that, or is it an open problem? (So far the
    infrastructure only supports Module, SCC, and Function. What
    happens (e.g.) when we add BasicBlock passes?)

You’re not wrong. I see several ways of managing it, but so far I’ve just left it open because the overhead of the abstractions needed to manage it seem larger than the problem. That may change, and we’d need to revisit this if so.

However, I’m not sure the number of IR units is likely to grow that much. Regarding basic block passes, I would like to just do away with them completely. I think the complexity they bring is greater than the utility they provide.

Notably, I’m not sure it will ever make sense to have analysis management at a finer granularity than function, because I don’t think there are any more narrow domains which can compute analysis results in conjunction with transformations independently on two units. Maybe I’m wrong?

I’m somewhat expecting the only other significant addition to be a loop pass management layer, and I suspect that one will not provide the analyses at all.

I’m curious what you and others think here. I do consider this area still “open” in that I don’t think we’re painted into a corner (yet).

  1. Previously, (e.g.) function analysis passes would typically hold
    results only for one function – when the next function was
    analyzed, the previous function’s results would be invalidated.
    Now, the FunctionAnalysisManager will happily store results for
    every function. (Looking forward, BasicBlockAnalysisManager might be
    able to store results for every BasicBlock.)

This is powerful, but costs memory. What’s the plan for keeping
memory usage in check (especially for large modules)?

The is both the benefit and challenge of the new system. The “plan” (such as it is) is to teach the pass managers to GC analysis results. I’m not focused on that until the system is, in essence, working because its hard to know what the right balance is here without being able to run a big pipeline of passes over a big pile of IR. I have to imagine that the module → SCC adaptor layer is going to do some amount of GC’ing of analysis results for SCCs that are quiesced, the question is what the right signal is for quiesced. We have a lot of options here while we’re using the caching approach, and I have no way to predict which will be the best.

Ultimately, this is allowing us to consciously make a time/space tradeoff. While it means we have to think about that tradeoff at least we get to make it now! =]

  1. In the old infrastructure, CGPassManager::runOnModule() (in
    CallGraphSCCPass.cpp) has a do-while loop that re-runs all passes
    on a given SCC as long something new gets devirtualized, up to a
    maximum number of iterations. I don’t see the equivalent loop in
    CGSCCPassManager, and I don’t see an obvious way to accomplish the
    same optimization cycle.
  • How will we re-implement this loop? (Or did I miss it?)

I’m planning to have a iterate utility that can add a limited amount of iteration based on specific conditions of any pass, including a CGSCC pass manager.

  • There’s a potential infinite loop between devirtualization and
    inlining on recursive virtual functions. How will we avoid it?

I don’t follow? We don’t inline recursive functions. Eventually, we either run out of call sites or see direct recursion and stop inlining. Am I missing a case where this behaves differently during devirtualization?

(Arguably, we shouldn’t just stop, we should do something more intelligent, but that’s orthogonal to the pass manager. That’s just a weakness of the inliner. And naturally, solving it includes setting limits.)

  1. There aren’t any machine-level pass managers yet. Will they look
    about the same? (Are there open problems to solve there?)

I have NO IDEA! =[ This is a big open area.

My current plan is to start off the way the current pass manager works: there are currently two independent module pass managers. At first, there will still be, and only one will be the new pass manager.

My next planned step is to try to encapsulate codegen within a single function pass, and the MC streaming within a single module pass. Then, internally, codegen can continue using MachineFunctionPass and other things. I don’t know how well this will work with analyses though.

Failing that, there will be a new RFC thread about how to move CodeGen into the new pass manager world because it’ll be a substantial chunk of work, and largely independent work. This is an area where I would love help of course. =D I think the only thing it would depend on is having most of the IR-level analysis up and running. From that point on, it should be completely independent.

  1. Andy asked in a separate thread [1] (my words): “Does this
    infrastructure allow for invalidation of classes of analyses?”
    (E.g., invalidate all analyses whose results depend on the
    control-flow graph.) Your response there [2] was (my words): “Not
    directly, but AnalysisManager will be a good place for that API.”

I thought a little about this, and it’s not clear to me how to fit
this API into AnalysisManager cleanly.

One side seems straightforward. Assuming you have a transformation
pass that modifies the CFG, tell the AnalysisManager that you’ve
modified the CFG, and return that all analyses have been preserved
(since everything else has). This side is easy to make opt-in.

The other side is not opt-in, though. All analyses that depend on
the CFG must respond, or the invalidation mechanism doesn’t work.

My very hand-wavy idea has been to use something along the lines of the preserved set for this. Essentially, analyses which rely on the CFG can test for whether that has been preserved, and if not, become invalid. Then we can just “try” to invalidate all the analysis results, and only those that are safe will remain.

There are several related options with different API tradeoffs. I’d need to sit down and implement it to see which pattern was really the best. Open to suggestions here.

Another option I’ve toyed with, but not yet found a way that I like is to have some shared IDs in addition to the unique IDs, and use those to form category sets.

  1. CGPassManager was one place where the old return value for a
    transformation pass (bool representing “did-change”) was useful.
    Are there any others?

Not to my knowledge. =[

Is the API using PreservedAnalyses::all()
as a stand-in for “did-not-change”? If so, does that conflict with
adding more nuanced API for on-the-fly invalidation of analyses?

Currently, yes, it is using preserving all analyses as a stand-in for “did no work”. This seems fine for invalidation purposes, but not so good for the iteration thing I mentioned above. I think we’ll need a better way to indicate “no work done”, but not sure what the best way is. Suggestions very welcome here as well.

A few random comments scattered throughout.

Strongly agreed. And having a BBPassBase which is a Function pass with a pure runOnBB function - with no special knowledge anywhere else - seems to solve any remaining benefit they do have. In particular, I’d like to remove the can’t modify any other block requirement since I’ve yet to meet a pass which actually obeys this. The only one I question is a loop analysis. What happens if a loop transform modifies one small subloop out of a large function? (I should have read further before typing…) I’ll be interested in your thoughts on this when you get to it. This might be another one to talk about in person. :slight_smile: Something like a victim cache might be one option, but let’s talk about this more once you get there. :slight_smile: Having this be parametrizable will be nice. :slight_smile: Alternatively, we could add a callback/option/flag on analyses to indicate they need to be invalidated on CFG change and ‘dummy analysis’ for each grouping. The analysis manager could then loop through all registered analyses and call invalidate iff the flag is set. It might be a good idea to segregate passes into hard categories for “manually invalidated/preserved” and “inferred from groups”. Attempting to use an analysis from one group with the other mechanism should result in clear and obvious error messages. (Maybe? Not sure here.) Honestly, I’m a bit queasy with the idea of having two ways to express invalidation/preservation though. If we do want to support it, it should be unified as early in the process as possible with good logging about what’s happening. I’ve got some out of tree code that uses the return code when running sub-passes. And yes, I know that constructing an inner pass manager, or even just a single pass, and running it inside another pass is utterly unsupported. It is often useful though. :slight_smile: