How to speed up tests (`check-mlir`) on Mac?

MLIR
1

I’ve recently switched my laptop from a Dell with a 12th Gen Intel i7-1255U to an Apple laptop with an M1 chip. The M1 is much quicker to compile LLVM + MLIR from scratch with 20 minutes versus 40 minutes.

However, running the tests via cmake --build . --target check-mlir takes longer on the M1. It takes about 50 seconds there whereas the Dell does it in less than 20. I suspect this is due to some security setting on Mac. Anyone here who knows what I should do on the Mac to speed things up?

2

What impacts the timing:

  • the kind of build (Debug vs Release vs ReleaseAsserts)
  • whether you enabled the example (in particular the standalone example runs cmake, which is slow on Mac)
  • whether you enabled the integration tests
  • the number of backends you enabled.

Lit also always records the timing for every test and writes it in ./tools/mlir/test/.lit_test_times.txt .

I asked ChatGPT to write a script to sort the file for me:

#!/usr/bin/python

import sys

def read_key_value_pairs(file_path):
    key_value_pairs = []
    with open(file_path, 'r') as file:
        for line in file:
            key, value = line.strip().split()
            key_value_pairs.append((float(key), value))
    return key_value_pairs

def sort_and_print(file_path):
    key_value_pairs = read_key_value_pairs(file_path)
    sorted_pairs = sorted(key_value_pairs, key=lambda x: x[0])
    for key, value in sorted_pairs:
        print(f"{key} {value}")

if __name__ == "__main__":
    if len(sys.argv) != 2:
        print("Usage: python script_name.py file_path")
    else:
        file_path = sys.argv[1]
        sort_and_print(file_path)

Example:

$ python ~/lit_test_time.py ./tools/mlir/test/.lit_test_times.txt  | tail -n 20
...
3.809423 Integration/Dialect/SparseTensor/CPU/sparse_matmul.mlir
3.991368 Integration/Dialect/SparseTensor/CPU/sparse_conversion_sparse2dense.mlir
4.562855 Integration/Dialect/SparseTensor/CPU/sparse_binary.mlir
5.541758 mlir-reduce/multiple-function.mlir
5.694897 mlir-reduce/crashop-reduction.mlir

Something you can do also is use the make flag -DLLVM_LIT_ARGS="-v --time-trace-output=$PWD/lit_times.json" when you configure LLVM, then any lit run will generate a trace that can be visualized in https://ui.perfetto.dev
For example:

Screenshot 2023-07-19 at 11.17.07 AM
Screenshot 2023-07-19 at 11.17.07 AM1752×499 144 KB

1 Like
3

Since it took 20 minutes to build (all of LLVM + MLIR) on your M1, I’m guessing it’s a release build. 50s is a lot for the testing time. As an alternative to what Mehdi mentioned, you can time the tests on the cmd-line with:

$ LIT_OPTS="--time-tests" ninja check-mlir

This will tell you what’s slowing things down.

1 Like
4

It turns out that it was caused by -DCMAKE_BUILD_TYPE=Debug. I did set it to Release earlier, but did not do a clean rebuild. Now after a clean rebuild, it’s fixed. Time is about 10 seconds now.

Thanks Medhi and Uday