MLIR for arm SME : Reducing tile data transfers

[MacDue]

@nujaa

scf.for %arg3 = %c0 to %dim step %c4_vscale {
  scf.for %arg4 = %c0 to %dim_1 step %c4_vscale {
    load ZaTile
    scf.for %arg5 = %c0 to %dim_0 step %c1 {
      load A
      load B
      MOPA A, B ZaTile
    }
    store ZaTile

This exact lowering is what you get if you use IREE, which already does this hoisting, it is not included in the -test-lower-to-arm-sme pipeline as that was mainly intended for functional correctness tests (not optimal code).

Edit: In IREE it’s the --iree-codegen-optimize-tensor-insert-extract-slices pass (which occurs right after vectorization) which does the hoisting of the reads/writes. I think everything that pass does is available upstream (just maybe not packaged into a single pass).