New Pass Manager with flto[=thin] not enabled (??)

Hello,

I recently noticed that the new pass manager was not enabled at regular/thin LTO link step even if ‘-fexperimental-new-pass-manager’ was specified in the compile step and link step commands. Upon closer inspection, it seems there’s so real way to invoke the new pass manager path (‘runNewPMPasses’ in lib/LTO/LTOBackend.cpp) during link step.

‘Conf.UseNewPM’ is used to select the new PM in LTOBackend.cpp, but only two places in the code sets it. The first is through the ‘llvm-lto2’ tool, and the second during the compile step with ‘-fthinlto-index=’. Neither are invoked during LTO/thinLTO link.

tools/llvm-lto2/llvm-lto2.cpp

static cl::opt
UseNewPM(“use-new-pm”,
cl::desc(“Run LTO passes using the new pass manager”),
cl::init(false), cl::Hidden);

Conf.UseNewPM = UseNewPM;

projects/clang/lib/CodeGen/BackendUtil.cpp

void clang::EmitBackendOutput( (…) {

runThinLTOBackend(CombinedIndex.get(), M, HeaderOpts, CGOpts, TOpts,
LOpts, std::move(OS), CGOpts.SampleProfileFile, Action);
return;

}

static void runThinLTOBackend (…) {

Conf.UseNewPM = CGOpts.ExperimentalNewPassManager;

}

My team and I were wondering if this is:

  1. intentional,
  2. if so, why do we not have an option to enable the new PM in the link step
  3. if not, should we pass an option via ‘-plugin-opt’, or via some other mechanism? It’s unclear to me what the preferred method of passing configuration options to LTO/thinLTO link step is.

Any insights/thoughts/comments would be greatly appreciated.

Cheers,

Graham Yiu
LLVM Compiler Development
IBM Toronto Software Lab
Office: (905) 413-4077 C2-707/8200/Markham
Email: gyiu@ca.ibm.com

`-fexperimental-new-pass-manager` is a compile time flag and has no
effect on the link step IIRC.
You may want to teach the driver to pass the correct flag to the LTO
plugin in gold or to lld.

For lld in particular, currently there's only the ability to specifiy
a custom pass pipeline when passing `--lto-newpm-passes` to the
linker. This is because, historically, when I added this flag, the
pass manager work was still in progress and the `-O2/LTO` pipeline
wasn't fleshed entirely (missing passes & functionality).
You can probably add the option to lld now, if you can't/don't know
how to do it, I may take a look this weekend.

Thanks,

Hi David,

Hi David,

So I should note that we’re only interested in the gold linker with LTO/thinLTO. It didn’t occur to me at the time that we also need to account for lld.

I noticed the current way of passing options to gold plugin is slightly hacky, in that there are handful of options we define and pass to gold specifically like -O<#> or -mcpu=[arch]. This seems like it’ll be a maintenance nightmare in the future.

Also, are there ways to embed options into bitcode files from the compile step?

Graham Yiu
LLVM Compiler Development
IBM Toronto Software Lab
Office: (905) 413-4077 C2-707/8200/Markham
Email: gyiu@ca.ibm.com

graycol.gifDavide Italiano —10/03/2017 03:14:18 PM—On Tue, Oct 3, 2017 at 12:08 PM, Davide Italiano davide@freebsd.org wrote: > On Tue, Oct 3, 2017 a

Hi Graham,

This looks like an oversight. Agree that the way of passing options through to the linkers is not great as it requires adding plumbing through the driver. But I think that is the best option for now. It would entail adding a gold-plugin option that would be set from the clang driver and using that to set Conf.UseNewPM. Do you want to send this patch?

Thanks,
Teresa

graycol.gif

HI Teresa,

Yep, I already have a patch, I can post it to phabricator ASAP.

Thanks for the info.

Graham Yiu
LLVM Compiler Development
IBM Toronto Software Lab
Office: (905) 413-4077 C2-707/8200/Markham
Email: gyiu@ca.ibm.com

graycol.gifTeresa Johnson —10/03/2017 04:29:20 PM—Hi Graham, This looks like an oversight. Agree that the way of passing options through

Patch available on Phabricator (https://reviews.llvm.org/D38517).

Graham Yiu
LLVM Compiler Development
IBM Toronto Software Lab
Office: (905) 413-4077 C2-707/8200/Markham
Email: gyiu@ca.ibm.com

graycol.gifTeresa Johnson —10/03/2017 04:29:20 PM—Hi Graham, This looks like an oversight. Agree that the way of passing options through