PassManager again...

Project Infrastructure LLVM Dev List Archives
1

Hi,

I have trouble making my ProfileInfo-Analysis available in the backend, I have changed llc so it loads the ProfileInfo if requested, the PassManger gives this output:

> Target Data Layout
> Create Garbage Collector Module Metadata
> Basic Alias Analysis (default AA impl)
> DWARF Information Writer
> No Profile Information
> Module Information
> ModulePass Manager
> Profiling information loader
> FunctionPass Manager
> Preliminary module verification
> Dominator Tree Construction
> Module Verifier
> Natural Loop Information
> Loop Pass Manager
> Canonicalize natural loops
> Scalar Evolution Analysis
> Loop Pass Manager
> Induction Variable Users
> Canonicalize natural loops
> Induction Variable Users
> Loop Strength Reduction
> Dominance Frontier Construction
> Exception handling preparation
> Lower Garbage Collection Instructions
> Remove unreachable blocks from the CFG
> Optimize for code generation
> Insert stack protectors
> Machine Function Analysis
> X86 DAG->DAG Instruction Selection
> X86 FP_REG_KILL inserter
...

Okay, so the ProfileInfoLoader is working, but when I examine the executions more closely I see that the ProfileInfo generated by the ProfileInfoLoader is immediately discarded, when the SelectionDAGISel kicks in the "No Profile Info"-Implementation is used:

> 0x1c1a740 Executing Pass 'Profiling information loader' on Module 'main.c-O0.ll.bc'...
> -*- 'Profiling information loader' is the last user of following pass instances. Free these instances
> 0x1c1a740 Freeing Pass 'Profiling information loader' on Module 'main.c-O0.ll.bc'...
> 0x1c1a740 Executing Pass 'Function Pass Manager' on Module 'main.c-O0.ll.bc'...
> 0x1c301a0 Executing Pass 'Preliminary module verification' on Function 'main'...
> 0x1c301a0 Executing Pass 'Dominator Tree Construction' on Function 'main'...
> 0x1c301a0 Executing Pass 'Module Verifier' on Function 'main'...
> 0x1c1ae50 Required Analyses: Preliminary module verification, Dominator Tree Construction
> -*- 'Module Verifier' is the last user of following pass instances. Free these instances
> 0x1c301a0 Freeing Pass 'Module Verifier' on Function 'main'...
> 0x1c301a0 Freeing Pass 'Preliminary module verification' on Function 'main'...
> 0x1c301a0 Executing Pass 'Natural Loop Information' on Function 'main'...
> 0x1c33000 Required Analyses: Dominator Tree Construction
> 0x1c301a0 Executing Pass 'Loop Pass Manager' on Function 'main'...
> 0x1c33460 Required Analyses: Natural Loop Information
> 0x1c301a0 Executing Pass 'Scalar Evolution Analysis' on Function 'main'...
> 0x1c33cb0 Required Analyses: Natural Loop Information
> 0x1c301a0 Executing Pass 'Loop Pass Manager' on Function 'main'...
> 0x1c359b0 Required Analyses: Natural Loop Information
> -*- 'Loop Pass Manager' is the last user of following pass instances. Free these instances
> 0x1c301a0 Freeing Pass 'Scalar Evolution Analysis' on Function 'main'...
> 0x1c301a0 Freeing Pass 'Natural Loop Information' on Function 'main'...
> 0x1c301a0 Executing Pass 'Dominance Frontier Construction' on Function 'main'...
> 0x1c370b0 Required Analyses: Dominator Tree Construction
> 0x1c301a0 Executing Pass 'Exception handling preparation' on Function 'main'...
> 0x1c36c70 Required Analyses: Dominator Tree Construction, Dominance Frontier Construction
> 0x1c36c70 Preserved Analyses: Dominator Tree Construction, Dominance Frontier Construction, Load profile information from llvmprof.out
> -*- 'Exception handling preparation' is the last user of following pass instances. Free these instances
> 0x1c301a0 Freeing Pass 'Exception handling preparation' on Function 'main'...
> 0x1c301a0 Freeing Pass 'Dominance Frontier Construction' on Function 'main'...
> 0x1c301a0 Freeing Pass 'Dominator Tree Construction' on Function 'main'...
> 0x1c301a0 Executing Pass 'Lower Garbage Collection Instructions' on Function 'main'...
> 0x1c1cec0 Required Analyses: Create Garbage Collector Module Metadata
> 0x1c1cec0 Preserved Analyses: Load profile information from llvmprof.out
> -*- 'Lower Garbage Collection Instructions' is the last user of following pass instances. Free these instances
> 0x1c301a0 Freeing Pass 'Lower Garbage Collection Instructions' on Function 'main'...
> 0x1c301a0 Executing Pass 'Remove unreachable blocks from the CFG' on Function 'main'...
> 0x1c36c00 Preserved Analyses: Load profile information from llvmprof.out
> -*- 'Remove unreachable blocks from the CFG' is the last user of following pass instances. Free these instances
> 0x1c301a0 Freeing Pass 'Remove unreachable blocks from the CFG' on Function 'main'...
> 0x1c301a0 Executing Pass 'Optimize for code generation' on Function 'main'...
> 0x1c38430 Preserved Analyses: Load profile information from llvmprof.out
> -*- 'Optimize for code generation' is the last user of following pass instances. Free these instances
> 0x1c301a0 Freeing Pass 'Optimize for code generation' on Function 'main'...
> 0x1c301a0 Executing Pass 'Insert stack protectors' on Function 'main'...
> 0x1c325a0 Preserved Analyses: Load profile information from llvmprof.out
> -*- 'Insert stack protectors' is the last user of following pass instances. Free these instances
> 0x1c301a0 Freeing Pass 'Insert stack protectors' on Function 'main'...
> 0x1c301a0 Executing Pass 'Machine Function Analysis' on Function 'main'...
> 0x1c301a0 Executing Pass 'X86 DAG->DAG Instruction Selection' on Function 'main'...
> 0x1c390d0 Required Analyses: No Alias Analysis (always returns 'may' alias), Create Garbage Collector Module Metadata, DWARF Information Writer, Load profile information from llvmprof.out, Machine Function Analysis
...

I was under the impression that it is possible to use the results form an Module-wide analysis in the following FunctionPasses, but in this case this is not working.

Also, I have modified all passes that run between the ProfileInfoLoader and the SelectionDAGISel to preserve the ProfileInfo, each pass reports this properly.

What really puzzles me is that the SelectionDAGISel reports the ProfileInfoLoader as required analysis (last line), so why is the PassManager assuming that the ProfileInfoLoader analysis is not used anymore?

I have tried to make the ProfileInfoLoader the default implementation (instead of "No ProfileInfo") but this gives an assertion while creating the passes:

> llc:.../include/llvm/PassSupport.h:111: llvm::Pass* llvm::PassInfo::createPass() const: Assertion `NormalCtor && "Cannot call createPass on PassInfo without default ctor!"' failed.

Can someone enlighten me please?

Andi

2

...

Did you update all these function passes to preserve profile info ?

3

Hi,

Devang Patel wrote:

4

Hi,

5

Hi.

Hi,

Devang Patel wrote:

Okay, so the ProfileInfoLoader is working, but when I examine the executions more closely I see that the ProfileInfo generated by the ProfileInfoLoader is immediately discarded, when the SelectionDAGISel kicks in the "No Profile Info"-Implementation is used:

  > 0x1c1a740 Executing Pass 'Profiling information loader' on Module 'main.c-O0.ll.bc'...
  > -*- 'Profiling information loader' is the last user of following pass instances. Free these instances
  > 0x1c1a740 Freeing Pass 'Profiling information loader' on Module 'main.c-O0.ll.bc'...
  > 0x1c1a740 Executing Pass 'Function Pass Manager' on Module 'main.c-O0.ll.bc'...
  > 0x1c301a0 Executing Pass 'Preliminary module verification' on Function 'main'...
  > 0x1c301a0 Executing Pass 'Dominator Tree Construction' on Function 'main'...

...

Did you update all these function passes to preserve profile info ?

Ja, they are all either PreserveAll or I have updated them to
AU.setPreserved<ProfileInfo>().

So I have tried this for some days now, I don't get it:

If I use AU.addRequired<ProfileInfo>() in SelectionDAGISel.cpp the wrong ProfileInfo is used. It uses the "No ProfileInfo" implementation if ProfileInfo but not the one from ProfileInfoLoaderPass. (Which is immediately discarded after creation.)

When I use AU.addRequiredID(ProfileLoaderPassID) I get assertions that a normal constructor is not available. Interesting enough its not the ProfileInfoLoader constructor thats misssing but the one from MachineFunctionAnalysis, which indeed has none.

This raises the question why different passes are loaded (in a different order) when I use addRequiredID instead of addRequired<>.

To get this clear: I try to use the ProfileInfo form a ModulePass in a subsequent FunctionPass, can this be a problem?

Andi

6

Hello, Andreas

When I use AU.addRequiredID(ProfileLoaderPassID) I get assertions that a
normal constructor is not available. Interesting enough its not the
ProfileInfoLoader constructor thats misssing but the one from
MachineFunctionAnalysis, which indeed has none.

Right. You cannot schedule any pass IR-level pass after
MachineFunctionAnalysis. I have no idea whether this is intentional or
not.

7

You need to debug pass manager yourself in a debugger to understand
why pass manager is discarding ProfileInfoLoaderPass. Usually it is
because

1) The pass manager does not know of any user of your pass - If that's
not the case then find out why? This may be a bug.
2) Someone is not preserving this pass. If that's the case then who is
it? Is it explicitly claiming to preserve your pass ?

8

Hi all!

>

If I use AU.addRequired<ProfileInfo>() in SelectionDAGISel.cpp the
wrong ProfileInfo is used. It uses the "No ProfileInfo" implementation
if ProfileInfo but not the one from ProfileInfoLoaderPass. (Which is
immediately discarded after creation.)

You need to debug pass manager yourself in a debugger to understand
why pass manager is discarding ProfileInfoLoaderPass. Usually it is
because

1) The pass manager does not know of any user of your pass - If that's
not the case then find out why? This may be a bug.
2) Someone is not preserving this pass. If that's the case then who is
it? Is it explicitly claiming to preserve your pass ?

Okay, got it, did cost me 4 days of debugging tough.

Why so long? PassManager is quite ingeniosly crafted, making it hard to
debug. Thats why there is massive CLI-Support for debugging passes (think "-debug-pass=Details"). Problem is that the debug output does not show _all_ the passes that are invalidated but some _only some_ of them.

Since _some_ invalidated passes are show, I never bothered to to check if my Pass was _actually really_ preserved because it did not show up in
the list of invalidated passes.

Attached patch adds debug output for _all_ invalidated passes. I will
commit this in the next days if there are no objections.

>
> Andreas Neustifter wrote:
>>
>> When I use AU.addRequiredID(ProfileLoaderPassID) I get assertions
>> that a normal constructor is not available. Interesting enough its
>> not the ProfileInfoLoader constructor thats misssing but the one from
>> MachineFunctionAnalysis, which indeed has none.
>
> Right. You cannot schedule any pass IR-level pass after
> MachineFunctionAnalysis. I have no idea whether this is intentional or
> not.

I found out about that one too. As mentioned in MachineFunctionPass.h:42 a MachineFunctionPass that overrides getAnalysisUsage() _has_ to call MachineFunctionPass::getAnalysisUsage() because there MachineFunctionAnalysis is required and preserved.

The problem mentioned by me was in a newly created BranchFolderPass::getAnalysisUsage() method which did not call MachineFunctionPass::getAnalysisUsage().

Andi

llvm-r90454.passmanager.patch (801 Bytes)

9

OK. Thanks!