Tosa.reciprocal' op operand #0 must be tensor of number values, but got 'tensor<f64>'

Incubator torch-mlir
1

Hello everyone. I seem to be having some problems lately and was hoping someone could give me some advice.
The thing is this. I’m using torch-mlir to convert models to MLIR, but I’m finding that it generates a lot of linalg.generic when the output type is LINALG_ON_TENSORS, which is actually not what I was expecting, I was hoping for something like linalg.matmul linalg.conv (because I want to run it on a customized gas pedal to facilitate me to write the related pass,linalg.generic is not easy to convert to a customized gas pedal), so my idea was to convert to tosa, but I ran into problems, I set the type of the output to TOSA.

error: "mambaforge/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py":349:0: 'tosa.reciprocal' op operand #0 must be tensor of number values, but got 'tensor<f64>'
note: "/mambaforge/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py":349:0: see current operation: %232 = "tosa. reciprocal"(%105) : (tensor<f64>) -> tensor<f64>

%232 = "tosa. reciprocal"(%105) : (tensor<f64>) -> tensor<f64> I don’t really see a problem.I don’t know how I should fix this. My thinking behind this is to use -tosa-to-linalg-named.
I hope someone can help me, thanks!

2

Could you maybe give a bit more context? Maybe show the input IR and the output of --debug? Did it already fail when parsing or after a pass? If after a pass, which pass exactly?

It is hard to find the problem just from the description

3

Thanks for the reply,I see what you mean,but there is only so much useful information (mlir-opt is not used here,python is used here),but I can provide the source code.

import torch
import torch_mlir

from transformers import AutoTokenizer, AutoModelForSequenceClassification

def prepare_sentence_tokens(hf_model: str, sentence: str):
    tokenizer = AutoTokenizer.from_pretrained(hf_model)
    return torch.tensor([tokenizer.encode(sentence)])

class OnlyLogitsHuggingFaceModel(torch.nn.Module):
    """Wrapper that returns only the logits from a HuggingFace model."""

    def __init__(self, model_name: str):
        super().__init__()
        self.model = AutoModelForSequenceClassification.from_pretrained(
            model_name,  # The pretrained model name.
            # The number of output labels--2 for binary classification.
            num_labels=2,
            # Whether the model returns attentions weights.
            output_attentions=False,
            # Whether the model returns all hidden-states.
            output_hidden_states=False,
            torchscript=True,
        )
        self.model.eval()

    def forward(self, input):
        # Return only the logits.
        return self.model(input)[0]

# Suppress warnings
import warnings
warnings.simplefilter("ignore")
import os
os.environ["TOKENIZERS_PARALLELISM"] = "true"

# The HuggingFace model name to use
model_name = "philschmid/MiniLM-L6-H384-uncased-sst2"

# The sentence to run the model on
sentence = "The quick brown fox jumps over the lazy dog."

print("Parsing sentence tokens.")
example_input = prepare_sentence_tokens(model_name, sentence)
print("Instantiating model.")
model = OnlyLogitsHuggingFaceModel(model_name)
print("Compiling with Torch-MLIR")
tosa_mlir = torch_mlir.compile(
    model,
    example_input,
    output_type=torch_mlir.OutputType.TOSA,
    use_tracing=True)

with open('MiniLM-L6-H384-uncased-sst2.mlir','w') as f:
    f.write(tosa_mlir.operation.get_asm())

print((tosa_mlir.operation.get_asm()))

If the code is written like the following, that will run fine.

tosa_mlir = torch_mlir.compile(
    model,
    example_input,
    output_type=torch_mlir.OutputType.LINALG_ON_TENSORS,
    use_tracing=True)
4

The model works fine for me when using a clean torch-mlir with the correct dependencies. Which version of torch-mlir are you using? Maybe yours is too old?

1 Like
5

Thank you very much for the feedback, I’ll check it out later .

6

Thanks! I solve the problem by using python 3.11(before is 3.10).

1 Like