Understanding lit testing with MLIR

Beginners
1

Hi all,

I’m currently working on a barebones MLIR project and writing my own dialect. I’m having a bit of trouble understanding how the lit test suite works, and whether I am actually testing the desired behavior of my programs or not. So far, I have two operations, constant (assignment) and inc (increment) for a simple I32 type implemented in my dialect:

module {
    // CHECK-LABEL: test_type_syntax
    func.func @test_type_syntax(%arg0: !parallel.counter) -> !parallel.counter {
        // CHECK: parallel.counter
        return %arg0 : !parallel.counter
    }

    // CHECK-LABEL: test_increment_syntax
    func.func @test_increment_syntax(%arg0: !parallel.counter) -> !parallel.counter {
        // CHECK: parallel.constant
        %0 = parallel.constant 42 : !parallel.counter

        // CHECK: %[[RES:.*]] = parallel.counter.inc %arg0 : !parallel.counter
        %1 = parallel.inc %0 : !parallel.counter

        return %1: !parallel.counter
    }
}

However, I’m having a hard time understanding how lit runs on this test file and how we actually check the desired behavior of test_increment_syntax. In my understanding, all I am doing here is verifying that parallel.constant is being called correctly in %0. Then, I verify that parallel.inc is called correctly and that it creates a result to be pattern-matched by %[[RES:.*]]. Is this understanding generally sound?

Moreover, none of this testing actually checks the desired behavior of my program. In other words, I am not verifying that my parallel.inc op is incrementing the value stored by %0 by 1. Is there any way I can verify this behavior directly in the test suite, or is this something I have to design into the behavior of my language, either through writing an MLIR verifier or implementing some equality operator?

2

Hi @bzhangg :waving_hand:

Here’s a way to think about a LIT test at a high level:

  • Your MLIR input is the test input,
  • FileCheck check-lines represent the expected output,
  • The RUN line (which isn’t included in your example) specifies what to test.

Everything else are fine implementation details.

Any LIT test is focused on checking the expected output of the “transformation” specified by the RUN line.

How do you define “desired behavior” in your case?

It sounds like you’re looking for end-to-end (e2e) or integration tests. In that case, we use mlir-runner. Here’s an “in-tree” example:

I hope this helps! Let me know if you have any more questions or need further clarification.

-Andrzej