Showing hotness in LLVM optimization remarks using AutoFDO sampling profile data?

Hello!
(+cc Adam Nemet, since he presented on optimization remarks at LLVM Dev Mtg 2016)

I have a large C++ program, which I am compiling using a sampling profile generated via perf and AutoFDO. I’d like to use this profile in order to show the hotness of each code path that is displayed in the new optimization remarks viewer tool (https://www.youtube.com/watch?v=qq0q1hfzidg).

It seems, however, that the ‘-fdiagnostics-show-hotness’ option emits a warning when used with ‘-fprofile-sample-use’: “warning: argument ‘-fdiagnostics-show-hotness’ requires profile-guided optimization information”.

Looking at the code in Clang that generates the warning, it appears that the warning is emitted unless instrumentation profile data is provided. Am I correct in understanding that sampling profile data cannot be used to determine the hotness of the code paths displayed via opt-viewer?

If this is correct, I am curious if this is a hard limitation, or whether I can do some work on CodeGen or ORE to make it possible to use a sampling profile. I ask because, while I’m excited to use optimization remarks, getting an instrumentation profile of my program would be difficult – it’s performance-critical and I can’t release an instrumented build.

Sorry if this is a dumb question – I don’t have much experience using PGO – and thanks for all the work on opt-viewer! It’s really fun to use.

  • Brian Gesiak

Hi Brian,

It’s great to hear that you find the opt-viewer useful. To your questions, specifically:

Hello!
(+cc Adam Nemet, since he presented on optimization remarks at LLVM Dev Mtg 2016)

I have a large C++ program, which I am compiling using a sampling profile generated via perf and AutoFDO. I’d like to use this profile in order to show the hotness of each code path that is displayed in the new optimization remarks viewer tool (https://www.youtube.com/watch?v=qq0q1hfzidg).

It seems, however, that the ‘-fdiagnostics-show-hotness’ option emits a warning when used with ‘-fprofile-sample-use’: "warning: argument ‘-fdiagnostics-show-hotness’ requires profile-guided optimization information”.

I think this is probably an oversight and we should accept -fprofile-sample-use as well. I think that simply no one has tried to use the opt-viewer with sample-based profiling so far. (We don’t have this support on macOS, so I’ve only been using PGO.)

Looking at the code in Clang that generates the warning, it appears that the warning is emitted unless instrumentation profile data is provided. Am I correct in understanding that sampling profile data cannot be used to determine the hotness of the code paths displayed via opt-viewer?

If this is correct, I am curious if this is a hard limitation, or whether I can do some work on CodeGen or ORE to make it possible to use a sampling profile. I ask because, while I’m excited to use optimization remarks, getting an instrumentation profile of my program would be difficult – it’s performance-critical and I can’t release an instrumented build.

I don’t think there is any hard limitation. The information from instrumentation- and sample-based profiling are exposed through the same interface/analysis (BlockFrequencyInfo) within the compiler which is what the optimization remark facility uses.

So I’d say that this should just work modulo some differences in accuracy due the sampling vs instrumentation. Despite the warning, do we already get hotness in the remarks? It seems we should since we still set DiagnosticsWithHotness in CodeGenOpts [1] and in turn would call setDiagnosticHotnessRequested with true in the LLVM Context [2]. This is what then controls whether we emit hotness in the remark in LLVM [3].

If this does not work, you probably wan to debug through this path of execution and see where we fail. Let me know if I can help further.

Sorry if this is a dumb question – I don’t have much experience using PGO – and thanks for all the work on opt-viewer! It’s really fun to use.

Not dumb at all, thanks for bringing this up.

Adam

  • Brian Gesiak

[1] https://github.com/llvm-mirror/clang/blob/master/lib/Frontend/CompilerInvocation.cpp#L887
[2] https://github.com/llvm-mirror/clang/blob/master/lib/CodeGen/CodeGenAction.cpp#L231
[3] https://github.com/llvm-mirror/llvm/blob/master/lib/Analysis/OptimizationDiagnosticInfo.cpp#L179

Hi Brian,

It’s great to hear that you find the opt-viewer useful. To your
questions, specifically:

Hello!
(+cc Adam Nemet, since he presented on optimization remarks at LLVM Dev
Mtg 2016)

I have a large C++ program, which I am compiling using a sampling profile
generated via perf and AutoFDO. I'd like to use this profile in order to
show the hotness of each code path that is displayed in the new
optimization remarks viewer tool (https://www.youtube.com/
watch?v=qq0q1hfzidg).

It seems, however, that the '-fdiagnostics-show-hotness' option emits a
warning when used with '-fprofile-sample-use': "warning: argument
'-fdiagnostics-show-hotness' requires profile-guided optimization
information”.

I think this is probably an oversight and we should accept
-fprofile-sample-use as well. I think that simply no one has tried to use
the opt-viewer with sample-based profiling so far. (We don’t have this
support on macOS, so I’ve only been using PGO.)

Agreed. Patches to fix the problem are welcome :slight_smile:

David

Adam, thanks!

Despite the warning, do we already get hotness in the remarks?

You’re absolutely right: despite the error, it looks like ORE does include hotness in the output YAML. Fantastic! Thanks for pointing that out.

I’ll try to submit a patch over the weekend to prevent the warning from being printed in the case a sampling profile is used. Of course, feel free to beat me to it. :slight_smile:

  • Brian Gesiak

Adam, thanks!

Despite the warning, do we already get hotness in the remarks?

You’re absolutely right: despite the error, it looks like ORE does include hotness in the output YAML. Fantastic! Thanks for pointing that out.

I’ll try to submit a patch over the weekend to prevent the warning from being printed in the case a sampling profile is used. Of course, feel free to beat me to it. :slight_smile:

It would be great if you could fix this. Feel free to add me as reviewer.

Adam