I am trying to get some starting points for working on following new pass manager related bugs:
https://bugs.llvm.org/show_bug.cgi?id=28322 [PM] Remove use of old PM in the middle-end.
https://bugs.llvm.org/show_bug.cgi?id=28323 [PM] Use new PM in production for Clang, LLD, libLTO, etc. middle-end
https://bugs.llvm.org/show_bug.cgi?id=28321 [PM] Remove use of old PM in the backend
I read related code but did not get a good starting point.
Can someone guide me through this? Can we add more details to these bugs? Or can we further divide these bugs to smaller workable items?
Any help will be appreciated.
Have you read the mailing list threads on this topic? I don’t believe we’re quite ready to make the switch yet. There was a discussion last October about what was left to be done. I’m sure it has been discussed since then too. Here’s a link to the start of the October discussion.
If you’d like to get involved, one possible area you could contribute is adding optbisect/optnone support as mentioned in this bug:
If that looks like something you’re interested in I can offer some guidance with it.
Have you read the mailing list threads on this topic? I don’t believe
we’re quite ready to make the switch yet. There was a discussion last
October about what was left to be done. I’m sure it has been discussed
since then too. Here’s a link to the start of the October discussion.
Yes I have gone through that mail chain. One thing mentioned in that was
Code Generation does not use new PM so I wanted to start working in that
If you’d like to get involved, one possible area you could contribute is
adding optbisect/optnone support as mentioned in this bug:
If that looks like something you’re interested in I can offer some
guidance with it.
Sure I am happy to work on it. Could you please update the bug with your
thoughts on how that needs to be done?
The opt-bisect facility is pretty simple. Essentially, you just need an interface function that gets call from a pass and returns true or false based on a counter that gets incremented whenever the function is called.
In the current implementation (with the legacy pass manager) there is an OptBisector object owned by the LLVMContext that does this. Each of the legacy pass base classes has a skip routine (skipFunction, skipLoop, skipBasicBlock, etc.) which provides a default implementation of the code to get a reference to the OptBisector and call one of its shouldRunPass() routines.
An important aspect of the current implementation is that the skip functions for any IR unit at the function level or below also checks for the presence of the “optnone” function attribute if the OptBisector does not indicate that the pass run should be skipped.
For the new pass manager, we’ll probably want to do something similar. I don’t have a polished design in mind, but just off the cuff I’m imagining something like a template shouldRunPass that takes an IR unit argument and uses it to get the LLVM context and check for the “optnone” attribute if applicable. This could possibly live in the Pass.
The OptPassGate interface used by the OptBisector requires an IR unit and a Pass pointer, but it only uses these to construct a description of the pass run for output. That can be changed if necessary. My initial prototype of the OptBisect function (which you can find here https://reviews.llvm.org/D18576) had support for the new pass manager. The pass manager driven approach in that patch was abandoned, but the OptBisect interface itself was substantially similar to what we ended up with. The patch is almost certainly stale, but it could provide some useful ideas.
With regard to the code generation passes, that’s a pretty big project and not one that I would recommend attempting if you aren’t sure where to start. Until that work has been done, whatever happens with the opt bisect support for the new pass manager should be designed to work in conjunction with the opt bisect support for the legacy pass manager so that the same counter is used for both.
bisect and optnone are certainly low hanging fruit in terms of implementation. On the other hand they need a cleaner design than they have now. E.g., OptBisect today is a managed static, which we absolutely should get rid of. Instead, bisect functionality can be much more cleanly implemented on top of the debug counters!
While the function call for optnone doesn’t strike me as similarly bad, there is another angle there. I think optnone should be handled by the passes, and not the manager. Considering running the pass outside of a manager, you’d probably expect it to respect optnone.
In summary, while easy to implement, these things need reconsidering and a solid RFC. So if you want to work on this, you should draft such a design document and post it to the list to collect comments and requests from the community.
Ok I will design the solution first.
As a point of clarification, optnone is already being handled by the pass itself in the legacy implementation. The skip[IR unit] functions are provided by the pass base classes, and the attribute is checked there. This happens any time the legacy wrapper is run, no matter how it is run.
Regarding the opt-bisect design, I’m not particularly fond of the managed static either, but I do want to mention a couple of things that I don’t want to lose about the current solution. First, it is important that we continue to print out information about the passes and the IR units as they are run or skipped. Our QA team uses this information as a first step in identifying the source of failures. Second, a change was recently introduced to generalizing the opt bisect interface (as obtained through the LLVMContext) so that clients can plug in other mechanism that use other criteria for stopping compilation.
After reading OptBisect and DebugCounter related code and playing bit around it I have following simple design:
- Add a debug counter for opt-bisect. Initilize it against option -opt-bisect-limit=.
- DebugCounter is a singleton class so can be accessed by both new and legacy passmanager.
We may need few more static method like getCounterIdForName(std::string &Name) etc.
- Use it to decide if this pass is required to be executed or not.
- For new passmaager just before executing run() for a pass we can check this counter.
- For legacy pass manager we can directly use this debug counter in skipFunction()/skipModule() etc method.
- There is already FIXME: added for moving getDescription() from OptBisect class to respective
IR units like Loop, Region etc. So that new pass manager can also use those methods.
- However to support feature added in this https://reviews.llvm.org/D44464 we may need to add a callback function
pointer that can be set and if present use it instead for normal counter value check to decide if a pass should
be executed or not. Or some other mechanism to provide this feature.
- I am not completely sure but if we are able provide custom callback feature then we may be able to remove OptBiset and OptPassGate
Anything missed here?
Please share your thoughts.
can you elaborate why you’re looking for a one-size-fits-all solution? What is the noteworthy benefit over adding a new-pm specific implementation?
Several changes you mention are purely for the benefit of supporting the legacy PM (which already has a working, tried, and tested solution). E.g.
getCounterIdForName, the FIXMEs you mention, and the callbacks. All of these are heavyweight changes, but I don’t see an upside of going this direction.
can you elaborate why you're looking for a one-size-fits-all solution?
What is the noteworthy benefit over adding a new-pm specific implementation?
I am happy with new-pm specific implementation but for example with
experimental pass manager switch clang have to use legacy pm for code
generation so in such case if user want to use opt-bisect to investigate
code generation passes then we need to add code which will initialize
OptBisect class with remaining pass execution count. Or we need to inform
that opt-bisect for new pass manager will work for passes that are ported
Several changes you mention are purely for the benefit of supporting the
legacy PM (which already has a working, tried, and tested solution). E.g.
`getCounterIdForName`, the FIXMEs you mention, and the callbacks. All of
these are heavyweight changes, but I don't see an upside of going this
I agreed that above mentioned changes are sufficiently big and need
thorough testing. But I have a simpler alternative to address above
mentioned issue. We can use debug counter added for new-PM in legacy-pm
code when ever it is available and OptBisect limit is not set.