Getting 'error: operand #N does not dominate this use' while writing a rewrite pattern

minkyuk · May 17, 2023, 7:34am

Hi,

I encountered an error while writing a custom rewrite pattern. I’m new to MLIR, and currently struggling to handle it.

Error:

error: operand #1 does not dominate this use
note: see current operation: %12 = "tfl.mul"(%8, %13) {fused_activation_function = "NONE", tac.device = "GPU", tac.inference_type = "FLOAT"} : (tensor<1x1x4x2xf32>, tensor<1x1x4x2xf32>) -> tensor<1x1x4x2xf32>
note: operand defined here (op in the same block)

I’m trying to add a MinimumOp with the code below.
It seems the error occurs because the replaced operation and the successive operation are in the same block.
While debugging, I found out the same code works with different location of replace.(commented in the code)

mlir::Value operand_depth0 = tanh_op.getOperand();
TFL::MulOp mul_op_depth1 = operand_depth0.getDefiningOp<MulOp>();
if(!mul_op_depth1) return failure();

mlir::Value lhs_depth1 = mul_op_depth1.getLhs();
mlir::Value rhs_depth1 = mul_op_depth1.getRhs();
TFL::MulOp mul_op_depth2 = lhs_depth1.getDefiningOp<MulOp>();
TFL::AddOp add_op_depth2 = rhs_depth1.getDefiningOp<AddOp>();
if(!mul_op_depth2 || !add_op_depth2) {
    TFL::AddOp add_op_depth2 = lhs_depth1.getDefiningOp<AddOp>();
    TFL::MulOp mul_op_depth2 = rhs_depth1.getDefiningOp<MulOp>();
    if(!mul_op_depth2 || !add_op_depth2) return failure();
}
  
mlir::Value lhs_depth2 = add_op_depth2.getLhs();
TFL::MulOp mul_op_depth3 = lhs_depth2.getDefiningOp<MulOp>();
if(!mul_op_depth3) return failure();
  
mlir::Value lhs_depth3 = mul_op_depth3.getLhs();
TFL::MulOp mul_op_depth4 = lhs_depth3.getDefiningOp<MulOp>();
if(!mul_op_depth4) return failure();

float upper_bound = 10.0;
auto clipped_upper = InsertMinimumOp(mul_op_depth4.getLoc(), mul_op_depth4.getLhs(), upper_bound, &rewriter);
auto mul_converted = rewriter.create<TFL::MulOp>(mul_op_depth4.getLoc(), clipped_upper.getResult(), clipped_upper.getResult(), rewriter.getStringAttr("NONE"));
rewriter.replaceOp(mul_op_depth4, mul_converted.getResult());
// rewriter.replaceOp(mul_op_depth4, clipped_upper.getResult()); // shows same error

// Below code works!
// float upper_bound = 10.0;
// auto clipped_upper = InsertMinimumOp(mul_op_depth1.getLoc(), mul_op_depth1.getRhs(), upper_bound, &rewriter);
// auto mul_converted = rewriter.create<TFL::MulOp>(mul_op_depth1.getLoc(), clipped_upper.getResult(), clipped_upper.getResult(), rewriter.getStringAttr("NONE"));
// rewriter.replaceOp(mul_op_depth1, mul_converted.getResult());

// Below code works!
// float upper_bound = 10.0;
// auto clipped_upper = InsertMinimumOp(tanh_op.getLoc(), operand_depth0, upper_bound, &rewriter);
// auto mul_converted = rewriter.create<TFL::MulOp>(tanh_op.getLoc(), clipped_upper.getResult(), clipped_upper.getResult(), rewriter.getStringAttr("NONE"));
// rewriter.replaceOp(tanh_op, mul_converted.getResult());

return success();

And this is how the input graph looks like.
The rewrite pattern works for Tanh and Mul(right before Tanh) and doesn’t work with the others.

I wonder why the associating blocks show different behavior depending on the location and how can I deal with the dominance issue with those error cases.
Thank you

troggo · May 17, 2023, 3:57pm

This error is because you’ve broken the constraint of op with an SSACFG region (e.g. func.func). You can read more about the requirements here, but the summary is that to resolve the error you’ll have to ensure that operations are in topological order after your pass.

Since you’re replacing mul_op_depth4 you likely want to create your new operations at the same position as that op, i.e. by doing rewriter.setInsertionPoint(mul_op_depth4) before creating new operations.

minkyuk · May 18, 2023, 7:41am

@troggo Thank you for your suggestion.
With rewriter.setInsertionPoint(), I could run the program without the error.

rewriter.setInsertionPoint(mul_op_depth4);
auto clipped_upper = insertMinimumOp(mul_op_depth4.getLoc(), mul_op_depth4.getLhs(), upper_bound, &rewriter);
auto mul_converted = rewriter.create<TFL::MulOp>(mul_op_depth4.getLoc(), clipped_upper.getResult(), clipped_upper.getResult(), rewriter.getStringAttr("NONE"));
rewriter.replaceOp(mul_op_depth4, mul_converted.getResult());

However, in the result, ten MinimumOp are added even I just added once.

TFL::MinimumOp insertMinimumOp(Location loc, Value input, float upper_bound, OpBuilder* builder) {
    
    mlir::RankedTensorType const_type = mlir::RankedTensorType::get({1}, builder->getF32Type());
    mlir::DenseElementsAttr const_attr = mlir::DenseFPElementsAttr::get(const_type, upper_bound);
    auto const_op = builder->create<TFL::ConstOp>(loc, const_attr);

    return builder->create<TFL::MinimumOp>(loc, input, const_op);
}

Any idea about this behavior?

Also, I thought the loc parameter for rewriter.create<TFL::MulOp>(loc, ...) is responsible for the insertion location. What is the difference between this loc in create method and rewriter.setInsertionPoint()?

minkyuk · May 18, 2023, 9:41am

While debugging, I found out that with the original lhs operand of the created MulOp(mul_converted), it produces the intended graph.

// works as intended
auto mul_converted = rewriter.create<TFL::MulOp>(mul_op_depth4.getLoc(), mul_op_depth4.getLhs(), clipped_upper.getResult(), rewriter.getStringAttr("NONE"));

But whenever I put the result of the created MinimumOp to the lhs of MulOp(including the target case), multiple MinimumOps are generated as above.

// cases of multiple MinimumOp generated
auto mul_converted = rewriter.create<TFL::MulOp>(mul_op_depth4.getLoc(), clipped_upper.getResult(), clipped_upper.getResult(), rewriter.getStringAttr("NONE")); // target
auto mul_converted = rewriter.create<TFL::MulOp>(mul_op_depth4.getLoc(), clipped_upper.getResult(), mul_op_depth4.getRhs(), rewriter.getStringAttr("NONE"));

With those workarounds, I suspect setting the input for insertMinimumOp with mul_op_depth4.getLhs() could be a problem? Or can it be a bug?

auto clipped_upper = insertMinimumOp(mul_op_depth4.getLoc(), mul_op_depth4.getLhs(), upper_bound, &rewriter);

mehdi_amini · May 18, 2023, 4:09pm

This is because your pattern is just adding a min using the result of the existing min, so you do mul(x, min(y)) → mul(x, min(min(y)).
Since the driver is trying to reach a fixed-point but gives up after 10 attempts you end up with 10 min().

You can run tf-opt with --debug and observe the pattern pre/post application I think.

minkyuk · May 18, 2023, 4:56pm

@mehdi_amini Thank you for your reply.

I didn’t get this, because I added a MinimumOp with an operand of MulOp, not the result of the MinimumOp.

TFL::MinimumOp insertMinimumOp(Location loc, Value input, float upper_bound, OpBuilder* builder) {
    
    mlir::RankedTensorType const_type = mlir::RankedTensorType::get({1}, builder->getF32Type());
    mlir::DenseElementsAttr const_attr = mlir::DenseFPElementsAttr::get(const_type, upper_bound);
    auto const_op = builder->create<TFL::ConstOp>(loc, const_attr);

    return builder->create<TFL::MinimumOp>(loc, input, const_op);
}
...
rewriter.setInsertionPoint(mul_op_depth4);
auto clipped_upper = insertMinimumOp(mul_op_depth4.getLoc(), mul_op_depth4.getLhs(), upper_bound, &rewriter);
auto mul_converted = rewriter.create<TFL::MulOp>(mul_op_depth4.getLoc(), clipped_upper.getResult(), clipped_upper.getResult(), rewriter.getStringAttr("NONE"));
rewriter.replaceOp(mul_op_depth4, mul_converted.getResult());

So the intended behavior is like this.
Original pattern : mul(x, x)
After Rewrite : mul(min(x), min(x))

mehdi_amini · May 19, 2023, 1:31am

Do the two x need to the same? That is are you checking that mul.lhs == mul.rhs? Or are you matching mul(x, y)?

In any case it seems that after you rewrite to mul(min(x), min(x)), your pattern will match again mul(x', x') on the resulting IR with x' = min(x).

If the input IR is mul(min(x), ...), then “the operand of MulOp” is the exact same thing as “the result of the MinimumOp”.

You’re not showing enough of what you’re matching / rewriting to really pin-point the issue right now.

minkyuk · May 19, 2023, 4:30am

Yes, what I’m trying to do is to insert a MinimumOp before x^2 in specific pattern. So, starting form tanh, I found the location(mul_op_depth4, where lhs=rhs) at which I want to insert the MinimumOp and added MinimumOp to make it mul(min(x), min(x)).

I believe this is an expected behavior according to this Application Recursion aspect. (I didn’t know about it. Thank you!).
Then is it possible to apply the pattern only once?

mehdi_amini · May 19, 2023, 4:43am

No: you need to make sure the pattern matching phase bails out with return failure and does not match the IR it will generate itself (that is something as simple as starting with: if (mulOp.getLhs().getDefiningOp<TFL::MinimumOp>()) return failure();)

minkyuk · May 19, 2023, 4:54am

Great. Thanks a lot