status of CodeGen in new Pass Manager

Project Infrastructure LLVM Dev List Archives
1

Hi,
This is a newbie question around CodeGen related passes and the current status in new Pass Manager.

From https://llvm.org/docs/NewPassManager.html#status-of-the-new-and-legacy-pass-managers, there are ongoing efforts to make the codegen pipeline work in the new Pass Manager (which is great!). Searching in the bug list (https://bugs.llvm.org/buglist.cgi?component=opt&list_id=226453&product=tools&query_format=advanced&resolution=—&short_desc=codegen&short_desc_type=allwordssubstr) gives no result.

I’m wondering if anyone has more information on the current status of CodeGen in the new Pass Manager (a tracking bug or other pointers)?

The context is that, I’m using opt CLI (by default new PM is used), and surprised that codegenprepare pass doesn’t run, so dig down and having more questions :slight_smile:

Any related information will be appreciated!

2

Unless I’m missing something, llc runs codegen, opt runs the optimization passes up to machine lowering. In other words - try llc instead of opt

3

Thanks Mircea! Yeah I get it that target dependent code is generated in the backend (llc)

(I should have highlighted this in the original post)

The CodeGenPrepare pass works in the legacy PM, but is not a part of the new PM. From this line (and the surrounding function) this is intended.

4

The part of https://llvm.org/docs/NewPassManager.html#status-of-the-new-and-legacy-pass-managers that’s relevant:

Some IR passes are considered part of the backend codegen pipeline even if they are LLVM IR passes (whereas all MIR passes are codegen passes). This includes anything added via TargetPassConfig hooks, e.g. TargetPassConfig::addCodeGenPrepare(). As mentioned before, passes added in TargetMachine::adjustPassManager() are part of the optimization pipeline, and should have a corresponding line in TargetMachine::registerPassBuilderCallbacks().

codegenprepare is in the list of passes that aren’t part of the LLVM IR optimization pipeline, but rather part of the codegen pipeline.

5

Hi Mingming,

About the status of using the new pass manager for the codegen pipeline, the RFC was here ([llvm-dev] [RFC] Introducing classes for the codegen driven by new pass manager) but there was no Bugzilla ticket for it, sorry! I've just created one 52493 – Use the new pass manager for codegen with updates for anyone who might be interested. I haven't been able to follow up on it for a while but a few in-flight patches are still relevant and in good shape (check PR52493). I'll see if I could push them forward in the near future.

About codegen-prepare, I don't have much to add other than Arthur's answer, except that D85168 would enable the use case, although it has some dependencies so it's not like that it could be landed soon.

HTH,
- Yuanfang

6

Thank you so much Arthur and Yuanfang! These pointers are very educational.

Now I realize there are two questions

  1. Use NPM for machine passes; this is the desired state RFC and D85168 tries to push forward.
  2. Whether CodeGenPrepare should be enabled by default (e.g., user of opt CLI specifies an IR with sufficient target information, but doesn’t enable CodeGenPrepare explicitly).

From https://llvm.org/docs/NewPassManager.html#status-of-the-new-and-legacy-pass-managers, the preferred option is to not run CodeGenPrepare in the default settings (although users can still run it via specifying -passes=codegenprepare).

I could make sense of the pointers, and understood the rationales better now.

I’m curious if there were proposals to turn on CodeGenPrepare by default (if IR has sufficient target information). (didn’t find one with this search query)
The good thing is that, when someone (e.g., like me when ramping up on the llvm infra) pipes the opt CLI and llc CLI together, the machine assembly is closer to the machine assembly of Clang (in cpp to assembly mode).

7

opt is concerned about the optimization pipeline and llc is concerned about the codegen pipeline. codegenprepare is part of the codegen pipeline, not the optimization pipeline. We happen to be able to use opt to run codegenprepare on its own because of how legacy PM passes are structured and llc is not well suited to run individual IR passes. This wouldn’t change even if we used the NPM for the codegen pipeline.

8

opt is concerned about the optimization pipeline and llc is concerned about the codegen pipeline. codegenprepare is part of the codegen pipeline, not the optimization pipeline. We happen to be able to use opt to run codegenprepare on its own because of how legacy PM passes are structured and llc is not well suited to run individual IR passes.

These all make sense to me.

(The following idea side-tracks from the original topic, but just brainstorming how to make the tools more friendly).

If it (piping opt and llc misses CodeGenPrepare and causes surprises) becomes a common question, llc tool might be enhanced by emitting a warning/hint to CLI users that the IR probably needs CodeGenPrepare pass (if input IR has metadata to record which middle-end passes ran)

This wouldn’t change even if we used the NPM for the codegen pipeline.

I get the point that CodeGenPrepare could be supported in opt (w/ NPM) since opt does IR to IR transformations.

9

llc should always run codegenprepare on IR before isel.

10

I used “llc -print-after-all -O3 <file.ll>” on this IR gives an assembly (https://godbolt.org/z/K6cszrPPf), and codegenprepare indeed runs in llc (from print-after-all output)

The source of my confusion is:

  1. Running the same IR by opt -O3 -codegenprepare gives a more optimized IR (https://godbolt.org/z/fdqTGsqG4)
  2. Piping the IR of step 1 (https://godbolt.org/z/544GMqaco) to llc -O3 gives a better assembly (tail call generated).
    I’m missing something here…
11

llc -O3 does not run the optimization pipeline on the IR, so IR-level optimizations aren’t being run unless you use opt -O3.
opt -O3 optimizes the IR.
llc -O3 (mostly) enables MIR optimizations and better isel. But if the input IR isn’t optimized then you lose most optimization opportunities.
So a typical clang -O3 would be somewhat equivalent to running Clang’s output IR through opt -O3 then llc -O3.

12

llc -O3 does not run the optimization pipeline on the IR, so IR-level optimizations aren’t being run unless you use opt -O3.
opt -O3 optimizes the IR.
llc -O3 (mostly) enables MIR optimizations and better isel. But if the input IR isn’t optimized then you lose most optimization opportunities.
So a typical clang -O3 would be somewhat equivalent to running Clang’s output IR through opt -O3 then llc -O3.

Thanks for pointing this out!

I built a local clang binary (to include a patch WIP) and ran clang -v -x ir -S -O3 <file.ll>. It indeed worked equivalently to a three-step operation (opt CLI with a patch | opt CLI to run CodeGenPrepare | llc)