How to get from "opt -O3" to "opt <a-single-pass>" with the new PM?

Hi,

I just wrote a PR about a crash that I only see with the new PM:
49185 – opt -O3 crashes with Assertion `NewL->getParentLoop() == ParentL && "All of the new loops must be siblings of the current loop!"' failed. with new PM

Normally (with the old PM) when I get a crash with e.g. "opt -O3" I use
-debug-pass=Arguments to get the list of passes run, and then I usually
try to reduce the amount of passes with the goal to reach a simpler opt
command line just running one pass to get a small and stable
reproducer.

With the new PM I have no idea how to do this. -debug-pass=Arguments
doesn't do anything, it doesn't give an error either, it's just silent.

What is the intended workflow to reach a proper nice and small
reproducer that has a minumum of dependencies towards different passes
with the new PM?

Thanks,
Mikael

Unfortunately there’s nothing similar to bugpoint’s crashing pass reduction feature that works for the new pass manager at the moment. I am also not aware of anyone working on this.

For the new PM, there’s -debug-pass-manager, which prints the passes that run, but not in a format suitable to feed back to `opt`.

Arthur, Alina, do you know if there’s a way to get a string of passes that corresponds to a all passes in a given pipeline, which can be passed directly to `opt`? With that, it might also be easier to build a llvm-reduce-based pass reduction tool as well.

Cheers,
Florian

There’s no way to get a list of passes. Currently the name used to specify a pass, e.g. “instcombine” in -passes=instcombine is mostly a one way street to creating the pass, it’s hard to go backwards from a pass to the original name due to the implementation. This could be fixed, but it’d be a fairly large change.

Personally I’d prefer an addition to https://reviews.llvm.org/D86657 where we dump the IR on a crash to a file. Then with -debug-pass-manager, we can see which pass caused the crash. Then use llvm-reduce to reduce the IR, running only that one pass.

There's no way to get a list of passes. Currently the name used to
specify a pass, e.g. "instcombine" in -passes=instcombine is mostly a
one way street to creating the pass, it's hard to go backwards from a
pass to the original name due to the implementation. This could be
fixed, but it'd be a fairly large change.

Personally I'd prefer an addition to https://reviews.llvm.org/D86657
where we dump the IR on a crash to a file. Then with -debug-pass-manager, we can see which pass caused the crash. Then use llvm-reduce to reduce the IR, running only that one pass.

Ok. So more or less the tool we have is -print-before-all -debug-pass-
manager and copy/paste the last input and rerun with the last
interesting pass. -print-on-crash makes it a little bit more convenient
when a pass crashes. I suppose this will work ok in many cases.

Another case is when a number of passes is needed to make a certain
pass expose something interesting.

Now and then I run into cases where
opt -pass1 ... -passN-1 -passN
is needed to trigger something and splitting the opt command to
opt -pass1 ... -passN-1 | opt passN
doesn't expose the problem. And the way I then find the needed passes
is to get the passlist with -debug-pass=Arguments and reduce the list
as far as I can. This is quite is rare though.

Thanks,
Mikael

Unfortunately this won’t work when the problem is caused by state not captured in LLVM IR. E.g. a pass could optimize the MemorySSA for a function and then the optimized MemorySSA is used by a subsequent pass, which crashes because the optimized MemorySSA enables an optimization. If you just use the IR before the crash, the pass will use unoptimized MemorySSA and may not crash. There are other analysis results that may expose similar effects.

I think we need a way to reduce the passes involved in such scenarios.

Cheers,
Florian

>
> > There's no way to get a list of passes. Currently the name used
> > to
> > specify a pass, e.g. "instcombine" in -passes=instcombine is
> > mostly a
> > one way street to creating the pass, it's hard to go backwards
> > from a
> > pass to the original name due to the implementation. This could
> > be
> > fixed, but it'd be a fairly large change.
> >
> > Personally I'd prefer an addition to
> > https://protect2.fireeye.com/v1/url?k=b28a4060-ed117a79-b28a00fb-8682aaa22bc0-2fe31fd4ca0dd9ee&q=1&e=71cf8e51-b719-4c90-b79a-9aad4f1804c5&u=https%3A%2F%2Freviews.llvm.org%2FD86657
> > where we dump the IR on a crash to a file. Then with -debug-pass-
> > manager, we can see which pass caused the crash. Then use llvm-
> > reduce to reduce the IR, running only that one pass.
> >
>
> Ok. So more or less the tool we have is -print-before-all -debug-
> pass-
> manager and copy/paste the last input and rerun with the last
> interesting pass. -print-on-crash makes it a little bit more
> convenient
> when a pass crashes. I suppose this will work ok in many cases.

Unfortunately this won’t work when the problem is caused by state not
captured in LLVM IR. E.g. a pass could optimize the MemorySSA for a
function and then the optimized MemorySSA is used by a subsequent
pass, which crashes because the optimized MemorySSA enables an
optimization. If you just use the IR before the crash, the pass will
use unoptimized MemorySSA and may not crash. There are other analysis
results that may expose similar effects.

I think we need a way to reduce the passes involved in such
scenarios.

+1

This is what I tried to describe in the

"Another case is when a number of passes is needed to make a certain
pass expose something interesting."

case.

Thanks,
Mikael

Alright, I will try to come up with a nice way of reducing the list of passes.

That’s great, thanks!

If we were to do something like -debug-pass=Arguments, https://reviews.llvm.org/D97722 is the first step.

Otherwise we could do some sort of pass instrumentation that skips random passes until we get to a minimal set of passes required to repro an issue. Sort of like opt-bisect but more granular.