[RFC][flang][cmake][perf-training] Optimize flang with PGO and BOLT

Subprojects Flang
, ,
1

This Request for Comments (RFC) proposes a set of the CMake cache files and surrounding changes required for improving the compilation time of the Flang compiler by using the PGO and BOLT optimizations.

Motivation and Background

The indented beneficiaries of this change are the LLVM-based toolchain publishers. With this change they would be able to incorporate the Flang optimization routine into the toolchain building process. As for today, the toolchain publishers already do the same thing with the Clang compiler. The change proposed in this document is mostly reusing the same two-stage methodology as used for optimizing Clang, intentionally combining optimization of Clang and Flang into one building process. The motivation behind this choice was to avoid addition of unnecessary burden of re-running mostly the same steps in order to obtain both Clang and Flang optimized.

Proposed extension

Following the Clang’s suit, a set of CMake cache files is being proposed:

  1. flang/cmake/caches/PGO.cmake, which can be used to generate a multi-stage instrumented compiler; the instrumented compiler is being profiled on the training set of programs to compile, and the final optimized compiler is being built using the profile obtained during the training.

  2. flang/cmake/caches/BOLT.cmake, which can be used to initiate the post-link binaries optimization using runtime profiling.

  3. flang/cmake/caches/BOLT-PGO.cmake, which combines both BOLT and PGO for better optimization.

Other CMake cache files (flang/cmake/caches/PGO-stage2.cmake and flang/cmake/caches/PGO-stage2-instrumented.cmake) are the auxiliaries included by the flang/cmake/caches/PGO.cmake cache file.

As a proof of concept, two pull requests were opened ([1] and [2]), with most comments and evaluations added under [1] (see the comment [3]).

The two pull requests represent the use of a two different selections of the PGO training codes. The first one ([1]) proposes a simple set of hello world and alike Fortran programs, the other one ([2]) proposes the reuse of the flang test suite already present in the flang directory tree.

As the evaluation presented in the comment [3] showed, the first approach gives the most of the improvement on compiling a set of randomly picked Fortran applications (dbcsr, polyhedron pb11, netlib’s lapack), while the second approach gives even more improvement, yet at cost of the longer preparation time and significantly bigger storage occupancy (200GB at least) needed for storing the performance profiling data.

The benefits of improving the compilation time of the flang compiler are undeniable. Compiling larger projects with flang is known to be time consuming, which results in a degradation of the user experience.

[1] [flang][cmake][perf-training] Optimize flang with PGO and BOLT by pawosm-arm · Pull Request #197947 · llvm/llvm-project · GitHub

[2] [flang][cmake][perf-training] Optimize flang with PGO and BOLT by pawosm-arm · Pull Request #198863 · llvm/llvm-project · GitHub

[3] [flang][cmake][perf-training] Optimize flang with PGO and BOLT by pawosm-arm · Pull Request #197947 · llvm/llvm-project · GitHub

4 Likes
2

This is nice to see. I think one of LLVM’s greatest optimization strengths is around profile guided optimization. We’ve seen wins of ~40% on the clang/C++ side using this approach.

It’s a bit surprising to me that the performance uplift here is only about 10-20%, but there could be a variety of reasons for that. perf-training on a small set of sample programs is likely to not be enough training data or very representative. A test suite is likely large enough, but still not representative of real world execution given they’re typically designed to look at edge cases that come up rarely.

When we build a PGO optimized clang we get the benefit that LLVM itself is written in C++, so we can just use that for perf-training.

3

One of the observation @chill has made when saw the timing results for the first time was that with so much time spent in kernel it’s hard to optimize anything at all. So this 10-20% isn’t that bad and I doubt we can do any better. The suspected culprit is the MLIR part which due to its multithreaded execution spends a lot of time in kernel waiting for futexes (as observed in strace -f).

4

The solution to that is to PGO+ThinLTO+(BOLT/Propeller) optimize the kernel. :stuck_out_tongue:

That would definitely explain those results and is pretty interesting. I didn’t realize flang ended up going down those paths during compilation.

5

Yeah, FIR and HLFIR are MLIR dialects.

6

Thanks for working on this and posting the RFC. I think this is a high ROI change that delivers significant compile-time benefits with good UX (simple enabling), with a great degree of code reuse between Flang and Clang. It would be nice to make project onboarding even easier but it’s beyond the scope of this change.

significantly bigger storage occupancy (200GB at least) needed for storing the performance profiling data.

I assume you were using instrumentation profiling (the default for both FDO and BOLT), so I would suggest giving sampling a try if you have access to LBR-capable hosts:

Also, do you have a breakdown of the speedup into LTO/PGO/BOLT?

7

Regarding extending PGO/BOLT CMake automation to other projects: I posted my thoughts in Generalizing Clang-BOLT automation

1 Like
8

I would suggest giving sampling a try if you have access to LBR-capable hosts

Sadly I don’t have access to said hosts. Would be great if someone with such access could extend my work further, if it eventually lands.

Also, do you have a breakdown of the speedup into LTO/PGO/BOLT?

I did. Sort of. Basically, I’ve tried possible combinations that you can get from using my PR (the one which makes use of the check-flang codes to train the optimizer). No external test suites were used.

The results (compilation times) are summarized in the following table:

                 dbcsr      pb11      lapack
base             82.83      52.63     210.05
pgo              74.24      49.58     194.31
pgo+lto          74.39      49.87     193.86
bolt             77.88      50.02     192.46
pgo+bolt         71.45      46.19     177.64
pgo+bolt+lto     71.39      46.24     177.60

Note that PGO is in huge advantage here: the total sum of the PGO’s .profraw file sizes is around 5GB, while the total sum of the BOLT’s .fdata file sizes is around 200GB.

Also note that LTO (namely, setting -DPGO_INSTRUMENT_LTO=Thin) has no visible impact.

9

Thanks, this is valuable information.

Regarding fdata files: unfortunately instrumentation can only produce fdata which can be pretty large (lots of redundancies in symbol names). Perhaps you can add compression to manage the file size.

Regarding LTO: it would make sense to measure its effect without PGO/BOLT as it doesn’t require the profile. Weird that it had little effect.

10

Regarding LTO: I’ve tried to build LLVM (from the same SHA as with all the other experiments) adding the -flto=thin flag. And there is a difference, so LTO when not combined with other optimizations can be beneficial:

            dbcsr       pb11     lapack
base        83.20      52.62     206.77
lto         78.20      49.34     195.03
11

To summarize the results, the average compile time improvements using check-flang are the following:

  • no profile:
    • LTO: 6%
  • one profiling round:
    • PGO: 7.5% (LTO+PGO 7.4%)
    • BOLT: 6.1% (no data point for LTO+BOLT)
  • two profiling rounds:
    • PGO+BOLT: 13.5% (LTO+PGO+BOLT 13.5%)

So to me it makes sense to have both PGO and BOLT enabled due to clear synergy. I’d keep ThinLTO enabled even though it’s not currently showing clear gain on this workload as it should have little relative build-time overhead, and future improvements might bring benefits.

3 Likes