Hello,
I would like to discuss enabling HLFIR lowering by default in Flang soon. High level MLIR operations for Flang were proposed by @jeanPerier in [RFC] Add higher level operations in FIR and ⚙ D134285 [flang][RFC] Adding higher level FIR ops to ease expression lowering. Accompanied by a new FIR value type (hlfir.expr), the new representation (HLFIR) allowed simplifying existing lowering PFT to MLIR, and also helped supporting Fortran features that were hard to support with the direct PFT-to-FIR lowering.
For example, CPU2006/465.tonto and CPU2017/621.wrf_s,628.pop2_s benchmarks are now successfully compiling with the HLFIR lowering, while hitting unimplemeted TODOs with the FIR lowering. Some polymorphic types cases are also supported only with the HLFIR lowering. In our internal testing we see about 600 new tests passing with the HLFIR lowering, while the amount of tests that work with the FIR lowering and do not work with the HLFIR lowering is 21.
On the performance side, the HLFIR lowering also provides benefits such as allocating much smaller temporary arrays (e.g. in Polyhedron/test_fpu2 and CPU2017/527.cam), and applying more complex post-lowering “conflict” analysis for LHS and RHS parts of the array assignments (e.g. see @tblah’s work in ⚙ D157107 [flang][hlfir] add an optimized bufferization pass).
With all the work that has been put into the HLFIR lowering, I think we are almost ready to make the HLFIR lowering the default, but I would like to set several short-term goals that we have to achieve first. Please let me know if you want to add something in the list or something looks off to you.
C1: Keep Flang lowering covered with LIT tests as much as possible without delaying HLFIR enabling too much.
- The HLFIR lowering is covered by LIT tests in
flang/test/Lower/HLFIR. While there are fewer tests than for the FIR lowering (inflang/test/Lower), I think we may switch to the HLFIR lowering without migrating all the tests. The complete migration fromflang/test/Lowertoflang/test/Lower/HLFIRmay be done after the switch. The quality of LIT testing will degrade in the menatime, but not much, given that the HLFIR lowering has decent amount of LIT tests as well. - I would like to adjust
flang/test/Lower/OpenACCtests to work with the HLFIR lowering (e.g. as done in ⚙ D157560 [flang][hlfir] Propagate acc.declare from hlfir.declare to fir.declare. for a single test). I consider this a blocker for switching to the HLFIR lowering, and I will be working on this. - I suppose, teams actively working on OpenMP will want to do the same for
flang/test/Lower/OpenMPtests, so that the testing coverage does not degrade too much. I would consider this a blocker as well, but I have no resources to work on that. I would greatly appreciate help with this.
C2: Keep end-to-end testing in “good shape”:
- As already mentioned, SPEC CPU 2000, 2006 and 2017 benchmarks compile and run successfully (w/o OpenMP) with the HLFIR lowering. We just need to keep it the same until we switch to the HLFIR lowering.
- Some tests from
llvm-test-suiteonly work with the HLFIR lowering (https://github.com/llvm/llvm-test-suite/blob/main/Fortran/gfortran/regression/EnabledFilesHLFIR.cmake), but some tests do not work either hitting a TODO or showing incorrect behavior in runtime (https://github.com/llvm/llvm-test-suite/blob/main/Fortran/gfortran/regression/DisabledFilesHLFIR.cmake). I would say that the disabled tests might be resolved after the switch to the HLFIR lowering, so we can just keepllvm-test-suitelists untouched. If you think that some of the disabled tests must be fixed before the switch, please let me know and I will work on them. - I do not have information about the SPEC CPU benchmarks or other tests that use OpenMP and won’t work with the HLFIR lowering. If you think we need to add some of them as blockers for the switch, please let me know.
- Many thanks to @kiranchandramohan who has already started fixing bugs with HLFIR lowering for OpenMP!
P1: Keep performance on par with the FIR lowering (preliminary performance data (multiple tabs): HLFIR-perf.xlsx - Google Sheets).
- I would like to keep SPEC CPU2006 and CPU2017 ref data set 1 copy performance on par with the FIR lowering, meaning we either achieve the same performance with the HLFIR lowering right away, or have exact explanation of the performance regressions not exceeding 25% and agree on the plans for fixing each of them. @tblah is pretty close on resolving some part of the huge regression of 548.exchange2 (other benchmarks will be improved as well). After that we should be able to do additional analysis and define our next steps. Please let me know if switching to the HLFIR lowering with pending performance regressions is not acceptable for your team.
- I would like to have Polyhedron performance issues addressed the same way as SPEC CPU above (i.e. regressions of more than 25% regressions are not acceptable). Basically, we have to resolve inefficiencies in temporary arrays creation and array assignments. These issues will affect majority of Fortran programs, so switching to the HLFIR lowering with slowing down the rest of the world does not sound right.
These are all the checkboxes that I want to check before we switch. Please share your thoughts. Thank you!
Slava