[RFC] MLIR buildbot with expensive pattern checks enabled

MLIR
1

Hi folks,

We (Arm) would like to contribute an MLIR buildbot with expensive API checks enabled (-DMLIR_ENABLE_EXPENSIVE_PATTERN_API_CHECKS=ON). For that, we need ninja check-mlir to pass. At the moment, however, more than 40 tests fail with the expensive checks enabled (including the e2e tests).

These failures span a number of different areas, many of which I’m not familiar with, so fixing all of them as a prerequisite for bringing up the buildbot would be difficult.

Instead, I propose adding a LIT config feature, e.g. xfail_when_expensive_checks_on, which affected tests could use to be conditionally marked as XFAIL when expensive checks are enabled. This would let us bring up the buildbot while still clearly tracking the tests that need to be fixed.

Would this be an acceptable path forward?

Thank you,
-Andrzej

2

Ready to be reviewed: #216323, thanks!

Btw, we aim to have this bot online by the end of September. Hopefully even sooner than that :crossed_fingers:

3

Do you mind posting a list of tests currently failing here in hopes folks see relevant things and pick them up? Forums have higher visibility then github PRs.

4

Sure, good idea @ftynse !

mlir/test/Conversion/SCFToAffine/scf-to-affine.mlir
mlir/test/Conversion/SCFToOpenMP/reductions.mlir
mlir/test/Conversion/SCFToOpenMP/scf-to-openmp.mlir
mlir/test/Conversion/SCFToOpenMP/vector-reduction.mlir
mlir/test/Dialect/Affine/canonicalize.mlir
mlir/test/Dialect/EmitC/wrap-func-in-class.mlir
mlir/test/Dialect/Linalg/continuous-tiling-full.mlir
mlir/test/Dialect/Linalg/decompose-pack-tile.mlir
mlir/test/Dialect/Linalg/decompose-unpack-tile.mlir
mlir/test/Dialect/Linalg/drop-unit-extent-dims.mlir
mlir/test/Dialect/Linalg/transform-op-peel-and-vectorize-conv.mlir
mlir/test/Dialect/Linalg/transform-op-tile-pack-unpack.mlir
mlir/test/Dialect/Linalg/transform-tile-and-winograd-rewrite.mlir
mlir/test/Dialect/OpenACC/acc-if-clause-lowering.mlir
mlir/test/Dialect/OpenACC/compute-region-canonicalize.mlir
mlir/test/Dialect/SCF/canonicalize.mlir
mlir/test/Dialect/SCF/for-loop-peeling.mlir
mlir/test/Dialect/SparseTensor/encoding_with_symbols.mlir
mlir/test/Dialect/SparseTensor/fuse_sparse_convert_into_producer.mlir
mlir/test/Dialect/X86/move-acc-for-contract-loop.mlir
mlir/test/Dialect/X86/vector-contract-bf16-to-fma.mlir
mlir/test/Dialect/X86/vector-contract-to-packed-type-dotproduct.mlir
mlir/test/Dialect/XeGPU/array-len-op-unit.mlir
mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/pack-unpack-mmt4d.mlir
mlir/test/Integration/Dialect/Linalg/CPU/ArmSVE/pack-scalable-inner-tile.mlir
mlir/test/Integration/Dialect/Linalg/CPU/ArmSVE/pack-unpack-mmt4d.mlir
mlir/test/Integration/Dialect/Linalg/CPU/mmt4d.mlir
mlir/test/Integration/Dialect/Linalg/CPU/pack-dynamic-inner-tile.mlir
mlir/test/Integration/Dialect/Linalg/CPU/pack-unpack-mmt4d.mlir
mlir/test/Integration/Dialect/Linalg/CPU/unpack-dynamic-inner-tile.mlir
mlir/test/Integration/Dialect/SparseTensor/CPU/dual_sparse_conv_2d.mlir
mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_block_matmul.mlir
mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_conv_1d_nwc_wcf.mlir
mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_conv_2d.mlir
mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_conv_2d_55.mlir
mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_conv_2d_nchw_fchw.mlir
mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_conv_2d_nhwc_hwcf.mlir
mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_conv_3d.mlir
mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_conv_3d_ndhwc_dhwcf.mlir
mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_conversion_block.mlir
mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_dilated_conv_2d_nhwc_hwcf.mlir
mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_pooling_nhwc.mlir
mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_strided_conv_2d_nhwc_hwcf.mlir
mlir/test/Transforms/test-strict-pattern-driver.mlir
5

I had a clean run locally a few months ago: many failures have the same root cause. One remaining PR I have on-going is [MLIR] Fix ErasedOpsListener false positives for newly created ops/blocks by joker-eph · Pull Request #192291 · llvm/llvm-project · GitHub (I don’t know how many instances it fixes today in this list, I will rebase and check!).

6

So [MLIR] Fix ErasedOpsListener false positives for newly created ops/blocks by joker-eph · Pull Request #192291 · llvm/llvm-project · GitHub is fixing:

  MLIR :: Conversion/SCFToOpenMP/reductions.mlir
  MLIR :: Conversion/SCFToOpenMP/scf-to-openmp.mlir
  MLIR :: Conversion/SCFToOpenMP/vector-reduction.mlir
  MLIR :: Dialect/Linalg/drop-unit-extent-dims.mlir
  MLIR :: Transforms/test-strict-pattern-driver.mlir

And [MLIR] Fix computeTiedSuccessorInputs for single-iteration scf.for by joker-eph · Pull Request #188986 · llvm/llvm-project · GitHub is fixing these 26 tests:

  MLIR :: Dialect/Linalg/continuous-tiling-full.mlir
  MLIR :: Dialect/Linalg/decompose-pack-tile.mlir
  MLIR :: Dialect/Linalg/decompose-unpack-tile.mlir
  MLIR :: Dialect/Linalg/transform-op-peel-and-vectorize-conv.mlir
  MLIR :: Dialect/Linalg/transform-op-tile-pack-unpack.mlir
  MLIR :: Dialect/Linalg/transform-tile-and-winograd-rewrite.mlir
  MLIR :: Dialect/SCF/canonicalize.mlir
  MLIR :: Dialect/SCF/for-loop-peeling.mlir
  MLIR :: Dialect/SparseTensor/fuse_sparse_convert_into_producer.mlir
  MLIR :: Integration/Dialect/Linalg/CPU/mmt4d.mlir
  MLIR :: Integration/Dialect/Linalg/CPU/pack-dynamic-inner-tile.mlir
  MLIR :: Integration/Dialect/Linalg/CPU/pack-unpack-mmt4d.mlir
  MLIR :: Integration/Dialect/Linalg/CPU/unpack-dynamic-inner-tile.mlir
  MLIR :: Integration/Dialect/SparseTensor/CPU/dual_sparse_conv_2d.mlir
  MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_block_matmul.mlir
  MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_1d_nwc_wcf.mlir
  MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_2d.mlir
  MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_2d_55.mlir
  MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_2d_nchw_fchw.mlir
  MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_2d_nhwc_hwcf.mlir
  MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_3d.mlir
  MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_3d_ndhwc_dhwcf.mlir
  MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conversion_block.mlir
  MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_dilated_conv_2d_nhwc_hwcf.mlir
  MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_pooling_nhwc.mlir
  MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_strided_conv_2d_nhwc_hwcf.mlir
7

Hi folks, the aforementioned bot has just been added:

ATM it is attached to the staging buildmaster. We hope to move it to the main buildmaster before the end of the month.

Thank you for fixing the issues!

8

The bot has been moved to production: mlir-aarch64-expensive-checks

Please help us keep it :green_circle: :folded_hands: