Mlir-opt tosa-to-linalg pass printing no output

MLIR
#1

Hi everyone!
We were trying to translate the tosa.conv2d operation into linalg operations using mlir-opt, and we found that tensors with int8 element type do not show any output.
Specifically, if we use the command mlir-opt -tosa-to-linalg on the file

func @conv(%img: tensor<1x29x29x4xi8>, %filter: tensor<16x3x3x4xi8>) -> tensor<1x27x27x16xi8> {
  %c0 = arith.constant 0 : i8
  %bias = tensor.from_elements %c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0: tensor<16xi8>
  %0 = "tosa.conv2d"(%img, %filter, %bias) {dilation = [1, 1], pad = [0, 0, 0, 0], stride = [1, 1]} : (tensor<1x29x29x4xi8>, tensor<16x3x3x4xi8>, tensor<16xi8>) -> tensor<1x27x27x16xi8>
  return %0 : tensor<1x27x27x16xi8>
}

no output is printed. I wanted to ask if this is an expected behavior
Thank you!

#2

Could you check if the command completed successfully? (E.g., what was the process return status).

#3

Thank you for the quick response. :slightly_smiling_face: The command is not successfully completed as it is returning 1, but there aren’t any error messages showing up.

#4

SG, then yes that’s a bug - somewhere there is a return failure which should be using an emitError or emitOpError instead. Returning with print-after-all should narrow down where it is failing.

#5

Unfortunately in the majority of cases I saw, silent failures are because of a parser silently failing, or the verifier silently failing on load, all before you get to the pass pipeline!

In particular this IR fails without running any pass pipeline, just round-tripping through mlir-opt has the same result.

#6

If just fixed it, so if you retry at HEAD, it’ll fail after parsing in a verbose manner now:

<stdin>:4:8: error: 'tosa.conv2d' op quantizationattr is required for quantized type, and not allowed for float type
  %0 = "tosa.conv2d"(%img, %filter, %bias) {dilation = [1, 1], pad = [0, 0, 0, 0], stride = [1, 1]} : (tensor<1x29x29x4xi8>, tensor<16x3x3x4xi8>, tensor<16xi8>) -> tensor<1x27x27x16xi8>
       ^
<stdin>:4:8: note: see current operation: %2 = "tosa.conv2d"(%arg0, %arg1, %1) {dilation = [1, 1], pad = [0, 0, 0, 0], stride = [1, 1]} : (tensor<1x29x29x4xi8>, tensor<16x3x3x4xi8>, tensor<16xi8>) -> tensor<1x27x27x16xi8>
#7

Thank you everyone for your quick response! It was really helpful

#8

Sorry I didn’t see this sooner, and thank you to @mehdi_amini for the fix that unlocked the error message :slight_smile:

As the error says, the problem is that TOSA treats this as a quantized datatype, and needs the corresponding quantization attribute. It could be trivially fixed by setting the zero points to 0s:

func @conv(%img: tensor<1x29x29x4xi8>, %filter: tensor<16x3x3x4xi8>) -> tensor<1x27x27x16xi8> {
  %c0 = arith.constant 0 : i8
  %bias = tensor.from_elements %c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0,%c0: tensor<16xi8>
  %0 = "tosa.conv2d"(%img, %filter, %bias) {dilation = [1, 1], pad = [0, 0, 0, 0], quantization_info = {input_zp = 0 : i32, weight_zp = 0 : i32}, stride = [1, 1]} : (tensor<1x29x29x4xi8>, tensor<16x3x3x4xi8>, tensor<16xi8>) -> tensor<1x27x27x16xi8>
  return %0 : tensor<1x27x27x16xi8>
}

This works fine and emits TOSA to LinAlg conversion with the latest head.

#9

I see what the error was. Thank you for your explanation!