I just wanted to give everyone a head up. I’m starting work on the pass manager based on numerous list discussions and some IRC discussions.
The first steps will be marking the existing code as “legacy” and clearing a path to build new facilities here. From there I’ll start building the new facilities without enabling them.
Some explicit legacy support goals:
I’m going to build a bridge so that an existing Pass can be inserted into the new pass manager with some adaptors and everything will just work. This may require touching the code that sets up the pass manager, but not the code that defines a pass. This will work even once the new pass manager bits are enabled if at all possible.
If you have code that includes the current PassManager headers, nothing should break right away, but when the new manager infrastructure is enabled, I’ll likely remove the old PassManager headers, breaking this code. My goal is to only break code that directly interacts with the management layer.
I’m going to play namespace games so that we don’t end up with PassManagerV2 and other silly names. The legacy headers will paper over this to keep legacy code compiling without change.
I'm planning to essentially try to build a new pass management layer that
doesn't suffer from the problems the existing one does. I could describe
every interface I plan in prose and email it out, but I'm not sure that's
really the most effective way to do it. My plan has been to work on
implementing the new system in-tree, and let folks code review it and pick
it apart as usual. When folks are happy with it, and it works, then we can
look at switching over.
Which RFCs cover this? Will the new system allow for passes to be inserted dynamically after startup? I have some delicate things that I did to make this ability to switch between mips 32 and mips16 ( they are different instruction sets) that relies on the pass manager working the way it does now. I’m not sure what you are planning to do and how it might affect what I did. It was very tricky to get that to work and a lot of things depend on it working now. I don’t think that we need to see how this will be implemented but I think that it would be helpful to list what you are trying to accomplish at a high level and what new features will be there and how the behavior will change from now.
The primary one was 'Pass Manager Redux', but over the past year the
planned design has shifted some. But not in ways that I think will impact
you at all.
Will the new system allow for passes to be inserted dynamically after
startup?
Yes.
I have some delicate things that I did to make this ability to switch
between mips 32 and mips16 ( they are different instruction sets) that
relies on the pass manager working the way it does now. I'm not sure what
you are planning to do and how it might affect what I did. It was very
tricky to get that to work and a lot of things depend on it working now.
As long as this is tested in the tree, it won't break.
I don't think that we need to see how this will be implemented but I think
that it would be helpful to list what you are trying to accomplish at a
high level and what new features will be there and how the behavior will
change from now.
Sure. Some of this is covered in my RFC. Others will be covered with the
new code and documentation that explain the micro-level differences. Most
of the things that are different from the RFC are the obvious consequence
of me and others looking at how it would be implemented. That's why I want
to implement it before committing to a lot more details.
The flip side is that I'd like code review of my implementation in a
reasonable incremental manner, and so it is likely to go directly into the
tree, and merely be disabled. I think that's a reasonable path forward, and
it will still leave lots of time for discussion about the exact design and
implementation before we make any switch.
I think it would be nice just to centralize the results of all discussions
thus far in a google doc or whatever.
I don't recall ever seeing in one place a listing of the primary issues
that the rewrite is meant to address. I've definitely seen various
discussions here and there from time to time, but nothing that a person
could look at and get a big picture view of what's happening and why.
Do'h, just saw you mention downthread of the "Pass Manager Redux" you
posted like a year ago ("redux" was apparently the word that was missing
from all my mail searches). (It's been a year, can you blame me for not
remembering? )
I have a question about the design of the new pass manager.
In the RFC on "codifying the optimization levels in clang and LLVM" ( http://lists.cs.uiuc.edu/pipermail/llvmdev/2013-January/058112.html ) the
idea was expressed of having function attributes to allow specific
functions to override the command line optimization level. If at some
point we decide to go in that direction and we introduce function
attributes for the remaining optimization levels (I am currently working
on the 'optnone' attribute which ideally could match optlevel -O0) how
would this affect the designing of the new pass manager?
I was wondering if you plan to design the new pass manager to
simplify/help achieving this goal?
Do you plan to address the problems raised only by "Pass Manager Redux" ( http://lists.cs.uiuc.edu/pipermail/llvmdev/2012-July/051638.html)
or do you also plan to investigate the optimization levels idea which I
expressed here?
Thanks,
Andrea Di Biagio
From: Sean Silva <silvas@purdue.edu>
To: Chandler Carruth <chandlerc@google.com>,
Cc: Reed Kotler <rkotler@mips.com>, LLVM Developers Mailing List
<llvmdev@cs.uiuc.edu>, Andrea_DiBiagio@sn.scee.net
Date: 16/09/2013 01:15
Subject: Re: [LLVMdev] Heads up: Pass Manager changes will be starting
I'm planning to essentially try to build a new pass management layer that doesn't suffer from the problems the existing one does. I could describe every interface I plan in prose and email it out, but I'm not sure that's really the most effective way to do it. My plan has been to work on implementing the new system in-tree, and let folks code review it and pick it apart as usual. When folks are happy with it, and it works, then we can look at switching over.
I think it would be nice just to centralize the results of all discussions thus far in a google doc or whatever.
Why not make a blog post? That is where we have put similar things before (e.g. type system changes, MC)
I just wanted to give everyone a head up. I'm starting work on the pass manager based on numerous list discussions and some IRC discussions.
Great! Aside from mailing list discussions, do you have a whitepaper that describes the end-result that you're shooting for? It doesn't need to be overly formal, but something along the lines of Random LLVM Notes would be useful.
The first steps will be marking the existing code as "legacy" and clearing a path to build new facilities here. From there I'll start building the new facilities without enabling them.
Some explicit legacy support goals:
1) I'm going to build a bridge so that an existing Pass can be inserted into the new pass manager with some adaptors and everything will just work. This may require touching the code that sets up the pass manager, but not the code that defines a pass. This will work even once the new pass manager bits are enabled if at all possible.
2) If you have code that includes the current PassManager headers, nothing should break right away, but when the new manager infrastructure is enabled, I'll likely remove the old PassManager headers, breaking this code. My goal is to only break code that directly interacts with the management layer.
3) I'm going to play namespace games so that we don't end up with PassManagerV2 and other silly names. The legacy headers will paper over this to keep legacy code compiling without change.
Sounds great to me, incremental is good. I think it's perfectly fine to do something like:
I’ve read pass manager redux, but you also mentioned IRC discussions. For the benefit of everyone who wasn’t on IRC, can you please give a summary of what (if anything) has changed from pass manager redux as a result of those discussions. Even better would be a new proposal, but if the changes are small then i’m not pushing for that.
…
I don’t think post commit review is appropriate for such an important change to the compiler. In your original RFC you said
'If folks generally like where this is going, I’ll get to work writing up initial code. The first thing I would do is provide an example collection of interfaces for the passes and pass managers to make sure we’re all on the same page. By then I should have a decent idea about the best strategy for cutting this into the actual codebase.”
which to me meant that you were going to work up some initial code and email that for review so that “we’re all on the same page”. I think that would be a better way to go here, at least for the initial code.
Also, i’m curious as to how you’re planning on testing this before its enabled by default. Is it going to require a significant number of unit tests, or do you have another plan? The reason I ask is that its not clear to me how easy it would be to test existing passes in your new framework without changes to the existing passes.
)