PGO question

Hi,

I have a question regarding PGO.

I collected profile data with the instrumentation build (-fprofile-instr-generate) and provided for PGO optimization in the second build (with -fprofile-instr-use=xxx.profdata). This works fine.

Then I tried to provide the profile data to opt using the option -pgo-instr-use, but this causes an error with the message: “Not an IR level instrumentation profile”.
I thought this error occurred because the profile data was produced by the instrumentation build from the source level, not IR level. However, if I try to make -pgo-instr-gen for the opt command, the resulting bitcode seems to cause a problem as follows (my environment is 3.9.0, mid Feb trunk).

$ clang -c -o main.bc -O2 -m64 -emit-llvm main.c
$ opt -pgo-instr-gen main.bc -o main_inst.bc
$ llc main_inst.bc -o main_inst.s
instrprof failed to lower an increment
UNREACHABLE executed at /home/suganuma/tools/llvm_new/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:5245!
#0 0x00000000815ecc7a llvm::sys::PrintStackTrace(llvm::raw_ostream&) /home/suganuma/tools/llvm_new/llvm/lib/Support/Unix/Signals.inc:322:0
#1 0x00000000815ed02a PrintStackTraceSignalHandler(void*) /home/suganuma/tools/llvm_new/llvm/lib/Support/Unix/Signals.inc:380:0
#2 0x00000000815eb33e llvm::sys::RunSignalHandlers() /home/suganuma/tools/llvm_new/llvm/lib/Support/Signals.cpp:44:0

#7 0x0000000081577202 /home/suganuma/tools/llvm_new/llvm/lib/Support/ErrorHandling.cpp:117:0
#8 0x000000008139792a llvm::SelectionDAGBuilder::visitIntrinsicCall(llvm::CallInst const&, unsigned int) /home/suganuma/tools/llvm_new/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:5247:0
#9 0x000000008139b11c llvm::SelectionDAGBuilder::visitCall(llvm::CallInst const&) /home/suganuma/tools/llvm_new/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:5811:0
#10 0x00000000813737e4 llvm::SelectionDAGBuilder::visit(unsigned int, llvm::User const&) /home/suganuma/tools/llvm_new/llvm/include/llvm/IR/Instruction.def:186:0

Am I doing something wrong? How can we provide profile data to opt?
What actually I want to do is to dump call graph (cross module) and cfg that reflect counter values from profile data.

Thank you,
–Toshio

Hi,

I have a question regarding PGO.

I collected profile data with the instrumentation build
(-fprofile-instr-generate) and provided for PGO optimization in the second
build (with -fprofile-instr-use=xxx.profdata). This works fine.

Then I tried to provide the profile data to opt using the option
-pgo-instr-use, but this causes an error with the message: "Not an IR level
instrumentation profile".
I thought this error occurred because the profile data was produced by the
instrumentation build from the source level, not IR level.

-fprofile-instr-generate by default still uses Clang frontend based
instrumentation -- that is why you see the warning.

However, if I try to make -pgo-instr-gen for the opt command, the
resulting bitcode seems to cause a problem as follows (my environment is
3.9.0, mid Feb trunk).

$ clang -c -o main.bc -O2 -m64 -emit-llvm main.c
$ opt -pgo-instr-gen main.bc -o main_inst.bc

you also need to use -instrprof to add the lowering pass.

The formal way to enable IR level instrumentation is to use the following
command line:

clang -fprofile-instr-generate -Xclang -fprofile-instrument=llvm main.c

David

Hi David,

Thank you for your reply. I tried both of the ways you suggested:

  1. By specifying -instrprof for opt command, the IR level instrumentation seems to work ok.
    $ clang -c -o main.bc -O2 -m64 -emit-llvm main.c
    $ clang -c -o lbm.bc -O2 -m64 -emit-llvm lbm.c
    $ llvm-link -o _all_combined.bc lbm.bc main.bc
    $ opt -pgo-instr-gen -instrprof _all_combined.bc -o _all_combined_inst.bc
    $ clang -o lbm_inst _all_combined_inst.bc -O2 -m64 -lm -fprofile-instr-generate

I found that the option -fprofile-instr-generate is still necessary in the link step, without which no profile data is produced (I confirmed it by ltrace).
When using the resulting profile in opt command and printing call graphs, is it easy to set call edge counts from the profile data?

$ opt -analyze -pgo-instr-use -print-callgraph _all_combined.bc

Call graph node for function: ‘main’<<0xb7635d60>> #uses=1
CS<0xb764d420> calls function ‘MAIN_parseCommandLine’
CS<0xb764dd78> calls function ‘printf’
CS<0xb764de00> calls function ‘MAIN_initialize’
CS<0xb764e400> calls function ‘LBM_handleInOutFlow’

  1. clang doesn’t seem to accept -fprofile-instrument=llvm
    $ clang -fprofile-instr-generate -Xclang -fprofile-instrument=llvm main.c
    error: invalid PGO instrumentor in argument ‘-fprofile-instrument=llvm’

Thank you,
–Toshio

graycol.gif

Hi David,

Thank you for your reply. I tried both of the ways you suggested:

1) By specifying -instrprof for opt command, the IR level instrumentation
seems to work ok.
$ clang -c -o main.bc -O2 -m64 -emit-llvm main.c
$ clang -c -o lbm.bc -O2 -m64 -emit-llvm lbm.c
$ llvm-link -o _all_combined.bc lbm.bc main.bc
$ opt -pgo-instr-gen -instrprof _all_combined.bc -o _all_combined_inst.bc
$ clang -o lbm_inst _all_combined_inst.bc -O2 -m64 -lm
-fprofile-instr-generate

I found that the option -fprofile-instr-generate is still necessary in the
link step, without which no profile data is produced (I confirmed it by
ltrace).

yes -- it is needed as the driver needs to pass in the profile runtime
library to the linker -- which is signaled by the option.

When using the resulting profile in opt command and printing call graphs,
is it easy to set call edge counts from the profile data?

$ opt -analyze -pgo-instr-use -print-callgraph _all_combined.bc

Call graph node for function: 'main'<<0xb7635d60>> #uses=1
CS<0xb764d420> calls function 'MAIN_parseCommandLine'
CS<0xb764dd78> calls function 'printf'
CS<0xb764de00> calls function 'MAIN_initialize'
CS<0xb764e400> calls function 'LBM_handleInOutFlow'
...

There is no explicit way of doing this now -- especially for cg
post-inline. This is currently being worked on.

2) clang doesn't seem to accept -fprofile-instrument=llvm
$ clang -fprofile-instr-generate -Xclang -fprofile-instrument=llvm main.c
error: invalid PGO instrumentor in argument '-fprofile-instrument=llvm'

This was added recently. You can update your clang/llvm source and rebuild

clang.

thanks,

David

graycol.gif