How to speed up tests (`check-mlir`) on Mac?

I’ve recently switched my laptop from a Dell with a 12th Gen Intel i7-1255U to an Apple laptop with an M1 chip. The M1 is much quicker to compile LLVM + MLIR from scratch with 20 minutes versus 40 minutes.

However, running the tests via cmake --build . --target check-mlir takes longer on the M1. It takes about 50 seconds there whereas the Dell does it in less than 20. I suspect this is due to some security setting on Mac. Anyone here who knows what I should do on the Mac to speed things up?

What impacts the timing:

  • the kind of build (Debug vs Release vs ReleaseAsserts)
  • whether you enabled the example (in particular the standalone example runs cmake, which is slow on Mac)
  • whether you enabled the integration tests
  • the number of backends you enabled.

Lit also always records the timing for every test and writes it in ./tools/mlir/test/.lit_test_times.txt .

I asked ChatGPT to write a script to sort the file for me:


import sys

def read_key_value_pairs(file_path):
    key_value_pairs = []
    with open(file_path, 'r') as file:
        for line in file:
            key, value = line.strip().split()
            key_value_pairs.append((float(key), value))
    return key_value_pairs

def sort_and_print(file_path):
    key_value_pairs = read_key_value_pairs(file_path)
    sorted_pairs = sorted(key_value_pairs, key=lambda x: x[0])
    for key, value in sorted_pairs:
        print(f"{key} {value}")

if __name__ == "__main__":
    if len(sys.argv) != 2:
        print("Usage: python file_path")
        file_path = sys.argv[1]


$ python ~/ ./tools/mlir/test/.lit_test_times.txt  | tail -n 20
3.809423 Integration/Dialect/SparseTensor/CPU/sparse_matmul.mlir
3.991368 Integration/Dialect/SparseTensor/CPU/sparse_conversion_sparse2dense.mlir
4.562855 Integration/Dialect/SparseTensor/CPU/sparse_binary.mlir
5.541758 mlir-reduce/multiple-function.mlir
5.694897 mlir-reduce/crashop-reduction.mlir

Something you can do also is use the make flag -DLLVM_LIT_ARGS="-v --time-trace-output=$PWD/lit_times.json" when you configure LLVM, then any lit run will generate a trace that can be visualized in
For example:

1 Like

Since it took 20 minutes to build (all of LLVM + MLIR) on your M1, I’m guessing it’s a release build. 50s is a lot for the testing time. As an alternative to what Mehdi mentioned, you can time the tests on the cmd-line with:

$ LIT_OPTS="--time-tests" ninja check-mlir

This will tell you what’s slowing things down.

1 Like

It turns out that it was caused by -DCMAKE_BUILD_TYPE=Debug. I did set it to Release earlier, but did not do a clean rebuild. Now after a clean rebuild, it’s fixed. Time is about 10 seconds now.

Thanks Medhi and Uday