Programmatically Toggle Specific LLVM Optimizations

Hi everyone!

First time poster here. Apologies if I am breaking some rules and please let me know so I will not break it again!

I am a summer research student at the Computer Science Department at the University of Toronto and I am working on benchmarking LLVM optimizations.

I tried looking for it online but was not able to find a programmatic way to toggle specific LLVM optimizations. For example, I would like to compile using the -O3 flag but with LICM turned off to see how much the compiled binary degrades in performance. I would like the ability to simultaneously toggle a few optimizations together as well. I would like to benchmark many optimizations so I would like to program the experiments.

So to be really specific, there are two questions:

1. Are there some command line interfaces that can accomplish this task? Is there a way to write a bash script to enumerate through the optimizations and turning them off one a time?

2. If not, what is a good guide to PassManager so that I can program this functionality?

Thanks so much for your help!

Sincerely,
Qiongsi

UTEA Reserach Student 2018, EcoSystem
Computer Science, University of Toronto

Some passes have supported options to disable them, e.g.
-fno-vectorize and -fno-unroll-loops, but there is no general option.
Since it's not useful in general to disable arbitrary options, some
handywork is required. Using the legacy (=default) pass manager here.

To get the passes that are run:
$ clang -O3 -mllvm -debug-pass=Arguments

To optimize separately:
$ clang -Xclang -disable-llvm-passes -S -emit-llvm -o - | opt <your
list of passes> | llc
(not tested, so don't expect to work straight out of the box)

This is not strictly the same as optimizing using clang (pass options
are missing), so there will be performance differences. You could
instead customize PassManagerBuilder::populateModulePassManager() and
PassManagerBuilder::populateFunctionPassManager().

Michael

Hi Michael!

Thanks for your reply. I managed to add a small amount of code to PassManager to have this feature implemented.

One can now turn off a pass by doing

opt -disablepass=licm or clang -mllvm -disablepass=licm

to turn off, for example, loop invariant code motion.

I programmed the command line option to be really hidden. This maybe a useful thing to have in the trunk for people who want to fine tune the optimizations.

Should I submit a patch? I am not sure if my code follows all the best practices of the community, but it is working from my own testing.

Thanks again for your help!

Qiongsi

Unless you intend to upstream you change (which I encourage you to),
there is no need to follow any best practices. If you intend to submit
the patch, the code reviews exist to ensure that the best practices
are followed. You can avoid the most obvious change requests by
considering https://llvm.org/docs/CodingStandards.html .

Please note that there is a newer pass manager. You might get feedback
to update the that pass manager as well/instead or that your use case
is too 'niche' to be useful for others. However, you won't find out
before submitting the patch. And the experience might be useful if you
intend collaborate to LLVM more often.

Michael

Thanks for the fast reply! I will submit a patch and find out!

Sincerely,
Qiongsi