How should I fix "failed to legalize operation 'builtin.unrealized_conversion_cast' that was explicitly marked illegal"

[xTayEx]

Hello everyone!
I am new to MLIR and I am struggling to fix the following error when lowering tosa.transpose to llvm.

transpose.mlir:4:14: note: see current operation: %14 = "builtin.unrealized_conversion_cast"(%13) : (!llvm.struct<(ptr, ptr, i64, array<1 x i64>, array<1 x i64>)>) -> memref<2xi32>

here is the code

module {
  func.func @generated_func(%arg0: tensor<2x3xf32>) -> tensor<3x2xf32> {
    %cst = arith.constant dense<0.000000e+00> : tensor<3x2xf32>
    %cst_0 = arith.constant dense<[1, 0]> : tensor<2xi32>
    %0 = "tosa.transpose"(%arg0, %cst_0) : (tensor<2x3xf32>, tensor<2xi32>) -> tensor<3x2xf32>
    return %0 : tensor<3x2xf32>
  }
}

here is the pass pipeline I use to lower the code snippet to llvm

mlir-opt --pass-pipeline="builtin.module(func.func(tosa-to-arith), convert-elementwise-to-linalg, arith-bufferize, func.func(linalg-bufferize), func.func(tensor-bufferize), func-bufferize, func.func(buffer-deallocation), memref-expand, func.func(convert-linalg-to-loops), convert-scf-to-cf, convert-linalg-to-llvm, convert-arith-to-ll
vm, expand-strided-metadata, finalize-memref-to-llvm, reconcile-unrealized-casts)" ~/transpose.mlir

I have checked this post

@nicholaitukanov

Hi I am a beginner and I am struggling to understand the following error:

ukernel.mlir:25:5: error: failed to legalize operation 'builtin.unrealized_conversion_cast' that was explicitly marked illegal
    %cmem: memref<1xvector<12x32xf32>>
    ^
ukernel.mlir:25:5: note: see current operation: %20 = "builtin.unrealized_conversion_cast"(%19) : (!llvm.struct<(ptr<array<12 x vector<32xf32>>>, ptr<array<12 x vector<32xf32>>>, i64, array<1 x i64>, array<1 x i64>)>) -> memref<1xvector<12x32xf32>>

The error comes from the following code snippet:

func.func @vec_op_acc(
    %amem: memref<256xvector<12xf32>>, %bmem: memref<256xvector<32xf32>>,
    %cmem: memref<1xvector<12x32xf32>>
)
{
    %i0 = arith.constant 0 : index

    affine.for %k = 0 to 256 {
        %a = vector.load %amem[%k] : memref<256xvector<12xf32>>, vector<12xf32>
        %b = vector.load %bmem[%k] : memref<256xvector<32xf32>>, vector<32xf32>
        %c = vector.load %cmem[%i0] : memref<1xvector<12x32xf32>>, vector<12x32xf32>
        %res = vector.outerproduct %a, %b, %c : vector<12xf32>, vector<32xf32>
        vector.store %res, %cmem[%i0] : memref<1xvector<12x32xf32>>, vector<12x32xf32>
    }

    return
}

I am using mlir-opt -pass-pipeline="convert-vector-to-llvm,convert-memref-to-llvm,convert-func-to-llvm,reconcile-unrealized-casts" ukernel.mlir > lowered/ukernel.lowered.mlir to lower the snippet to llvm MLIR.

Any insight would be greatly appreciated.

Thank you and apologies if this is a dumb question!

According to this post, it said that the error means type conversion wasn’t completed by some previous passes, and the solution is to remove reconcile-unrealized-casts to see the IR and look for unrealized_conversion_cast operations that are not paired with the casts going into the other direction. Then, add the missing passes. I removed reconcile-unrealized-casts, and the unrealized_conversion_cast operation is

builtin.unrealized_conversion_cast %22 : !llvm.struct<(ptr, ptr, i64, array<1 x i64>, array<1 x i64>)> to memref<2xi32>

The question is, I don’t know what passes should I add to fix the error.

Any insight would be greatly appreciated.
Thank you and apologies if this is a dumb question!

[tpopp]

You can hopefully use --mlir-print-ir-after-failure mentioned here: Debugging Tips - MLIR to obtain the full IR (or I normally use --mlir-print-ir-before-all or something similar to that).

In general with these cases, either the producer to or consumer of the cast is the problem. In your case, the cast is creating a memref, so I would look at the consumer and you’ll probably see an operation that is not part of the llvm dialect and was not lowered.

[xTayEx]

Thank you for your reply!
I used --mlir-print-ir-before-all and checked the IR dump. I found that after the tosa-to-arith, tosa.transpose is not lowered.

How should I properly lower tosa-transpose? What pass(es) should I use? Or do you think there is a better dialect to achieve my goal? In general, I am looking to generate code that will do the tensor transpose.

[tpopp]

I don’t interact with tosa, but it looks like you need tosa-to-linalg (and then might need other passes to lower linalg down). Regarding a better dialect, I can’t really say. It looks like tosa support is enough for at least this. You could also try using linalg which would support this. Potentially affine supports this, but I don’t know. There are then other options, but they are in codebases outside of LLVM.

[xTayEx]

Thanks for the tips! I will try tosa-to-linalg and see if it works!

[agostini01]

During our lowering pipeline, with llvm commit 339a7687e1c036a5f91 we are using the following passes to get from tosa to linalg on tensors:

mlir-opt \
  -pass-pipeline="builtin.module(func.func(tosa-to-tensor, tosa-to-linalg-named, tosa-to-linalg, tosa-to-arith))" \
  01_tosa.mlir \
  -o 02_linalg_on_tensors.mlir

Hope this helps.

[xTayEx]

Awesome! I tried this solution, and it works! Thank you so much!
Also, I found that using func.func(tosa-to-linalg) followed by empty-tensor-to-alloc-tensor is another solution for my case