Making a pass available to llc?

Hi - I wanted to make a pass available for use with llc, and found
that even though there is a PassInfo::LLC value, it isn't currently
used.

If you compare analyze.cpp and opt.cpp to llc.cpp, the first two find
appropriate pass names with a FilteredPassNameParser and create them,
but llc does not - it just gets all the passes it wants through the
TargetMachine.

Is there a particular reason not to create general Passes from llc?

Thanks,
mike

I think that the answer is that llc is intended to be the final compiler
of executable output. Running "general" passes in llc was never part of
the design. It just runs machine passes, assuming that its bytecode
input is already optimized. What you're looking for is a program that is
the combination of opt and llc. Eventually, the llvmc program intends to
be that program. It currently runs most of the passes run by gccld and
also provides a -load option to allow extension of pass sets. However,
llvmc is an unfinished work and a bit more cumbersome to use.

Why not just create your pass as a shared object and:

opt -load mypass.so -mypass | llc

?
Reid.

I think that the answer is that llc is intended to be the final compiler
of executable output. Running "general" passes in llc was never part of
the design. It just runs machine passes, assuming that its bytecode
input is already optimized.

OK, that makes sense.

What you're looking for is a program that is
the combination of opt and llc. Eventually, the llvmc program intends to
be that program. It currently runs most of the passes run by gccld and
also provides a -load option to allow extension of pass sets. However,
llvmc is an unfinished work and a bit more cumbersome to use.

Oh, I was a little confused, then - I thought llc actually was llvmc.

Why not just create your pass as a shared object and:

opt -load mypass.so -mypass | llc

My pass is an implementation of an analysis group that I wanted to
make available to machine passes as well as optimization passes. So I
really do want it in llc, not just its output.

I could easily change my llc to just load my pass, I just wanted to
check if there was a better way first. Specifically, should I dive
into llvmc and help flush out bugs? (how unfinished is it?)

Thanks,
-mike

> Why not just create your pass as a shared object and:
>
> opt -load mypass.so -mypass | llc

My pass is an implementation of an analysis group that I wanted to
make available to machine passes as well as optimization passes. So I
really do want it in llc, not just its output.

I could easily change my llc to just load my pass, I just wanted to
check if there was a better way first.

That sounds like the simplest thing to do. Loadable analyses in llc
would be a good feature to add if you wanted to contribute that back.
That way any machine level analyses that are needed could be loaded
(presumably via llvmc).

Specifically, should I dive
into llvmc and help flush out bugs? (how unfinished is it?)

Its not so much buggy as it is incomplete. It will faithfully compile
programs per the language configuration files using whatever tools are
required for a given language. See the configuration files in
llvm/tools/llvmc directory for examples. The main thing that is lacking
is not so much llvmc as it is llvm-ld which currently can't produce
native executables well. This may not matter for your project. There's
probably a few other things wrong with it but I forget the details. To
see it in action, check out the samples directory in the Stacker
project. It currently uses llvmc to compile the sample programs. Same
goes for the test directory.

Reid.

> > Why not just create your pass as a shared object and:
> >
> > opt -load mypass.so -mypass | llc
>
> My pass is an implementation of an analysis group that I wanted to
> make available to machine passes as well as optimization passes. So I
> really do want it in llc, not just its output.
>
> I could easily change my llc to just load my pass, I just wanted to
> check if there was a better way first.

That sounds like the simplest thing to do. Loadable analyses in llc
would be a good feature to add if you wanted to contribute that back.
That way any machine level analyses that are needed could be loaded
(presumably via llvmc).

OK, I have a version of llc that creates passes which are registered
as PassInfo::LLC.
It's essentially a clone of the code in opt and analyze.

Since I'm modifying llc, I have a couple small questions about that code:

opt and analyze (and a couple of other places) add a verifier pass,
but llc doesn't.
This would seem to make sense for llc as well - should I add it, with
the corresponding
hidden -no-verify option?

> Specifically, should I dive
> into llvmc and help flush out bugs? (how unfinished is it?)

Its not so much buggy as it is incomplete. It will faithfully compile
programs per the language configuration files using whatever tools are
required for a given language. See the configuration files in
llvm/tools/llvmc directory for examples. The main thing that is lacking
is not so much llvmc as it is llvm-ld which currently can't produce
native executables well. This may not matter for your project. There's
probably a few other things wrong with it but I forget the details. To
see it in action, check out the samples directory in the Stacker
project. It currently uses llvmc to compile the sample programs. Same
goes for the test directory.

OK, sounds like the solution to my problem is to modify llc, and is
sort of unrelated to llvmc, so I'll stick with what I was doing.

Thanks,
-mike

I can't see any harm in that. However, please make sure that it really
isn't being run. The verifier can be hidden by various levels of
abstraction.

Thanks!

Reid.

Assuming that I get everything with -debug-pass=Structure, then it isn't:

% llc -march=x86 -stats -f tests/IS/npbis.bc -debug-pass=Structure
Pass Arguments: -lowergc -lowerinvoke -lowerswitch -unreachableblockelim
Target Data Layout
Module Pass Manager
  Function Pass Manager
    Lower GC intrinsics, for GCless code generators
-- Lower GC intrinsics, for GCless code generators
    Lower invoke and unwind, for unwindless code generators
-- Lower invoke and unwind, for unwindless code generators
    Lower SwitchInst's to branches
-- Lower SwitchInst's to branches
    Remove unreachable blocks from the CFG
-- Remove unreachable blocks from the CFG
    X86 Pattern Instruction Selection
-- X86 Pattern Instruction Selection
    Live Variable Analysis
    Eliminate PHI nodes for register allocation
    Two-Address instruction pass
    Immediate Dominators Construction
    Dominator Set Construction
-- Immediate Dominators Construction
    Natural Loop Construction
-- Dominator Set Construction
    Live Interval Analysis
-- Natural Loop Construction
-- Live Variable Analysis
-- Eliminate PHI nodes for register allocation
-- Two-Address instruction pass
    Linear Scan Register Allocator
-- Live Interval Analysis
-- Linear Scan Register Allocator
    Live Variable Analysis
    X86 FP Stackifier
-- Live Variable Analysis
-- X86 FP Stackifier
    Prolog/Epilog Insertion & Frame Finalization
-- Prolog/Epilog Insertion & Frame Finalization
    X86 Peephole Optimizer
-- X86 Peephole Optimizer
    X86 AT&T-Style Assembly Printer
-- X86 AT&T-Style Assembly Printer
    Machine Code Deleter
-- Machine Code Deleter

I'll add it. I managed to uncover a bug in my own code with my new
version of llc, though, so I might not have a patch to send tonight,
but I will soon.

Thanks,
-mike

I can't see any harm in that. However, please make sure that it really
isn't being run. The verifier can be hidden by various levels of
abstraction.

Assuming that I get everything with -debug-pass=Structure, then it isn't:

-debug-pass=Structure does list everything. If you add it, please add it in an "#ifndef NDEBUG" block so that it does not penalize the release build.

Thanks!

-Chris

% llc -march=x86 -stats -f tests/IS/npbis.bc -debug-pass=Structure
Pass Arguments: -lowergc -lowerinvoke -lowerswitch -unreachableblockelim
Target Data Layout
Module Pass Manager
Function Pass Manager
   Lower GC intrinsics, for GCless code generators
-- Lower GC intrinsics, for GCless code generators
   Lower invoke and unwind, for unwindless code generators
-- Lower invoke and unwind, for unwindless code generators
   Lower SwitchInst's to branches
-- Lower SwitchInst's to branches
   Remove unreachable blocks from the CFG
-- Remove unreachable blocks from the CFG
   X86 Pattern Instruction Selection
-- X86 Pattern Instruction Selection
   Live Variable Analysis
   Eliminate PHI nodes for register allocation
   Two-Address instruction pass
   Immediate Dominators Construction
   Dominator Set Construction
-- Immediate Dominators Construction
   Natural Loop Construction
-- Dominator Set Construction
   Live Interval Analysis
-- Natural Loop Construction
-- Live Variable Analysis
-- Eliminate PHI nodes for register allocation
-- Two-Address instruction pass
   Linear Scan Register Allocator
-- Live Interval Analysis
-- Linear Scan Register Allocator
   Live Variable Analysis
   X86 FP Stackifier
-- Live Variable Analysis
-- X86 FP Stackifier
   Prolog/Epilog Insertion & Frame Finalization
-- Prolog/Epilog Insertion & Frame Finalization
   X86 Peephole Optimizer
-- X86 Peephole Optimizer
   X86 AT&T-Style Assembly Printer
-- X86 AT&T-Style Assembly Printer
   Machine Code Deleter
-- Machine Code Deleter

I'll add it. I managed to uncover a bug in my own code with my new
version of llc, though, so I might not have a patch to send tonight,
but I will soon.

Thanks,
-mike

-Chris

Attached is my patch that adds a (debug build only) verifier pass and
support for creating passes specified on the command line to llc.

Let me know if it needs changes to be acceptable. One thing I noticed
is that (eg.) opt and analyze have their options and globals in an
anonymous namespace while llc doesn't. I just used llc's convention -
which is preferable?

As to testing - It works as expected with my loadable analysis pass,
but I'm not set up to run llvm-test. I should really get that going.

I didn't add any pass printing code like what opt and analyze have
(under "-p" in opt) because it didn't seem like llc needs to be
printing its passes.

-mike

llc-with-loaded-passes-and-verifier.patch (1.68 KB)

Patch looks good. I've committed it here:

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20050725/027257.html

I ran the deja-gnu tests and a few of the llvm-tests on this and all seems well.

Thanks for the patch!

Reid