How to call MLIR function by c++?

MLIR
1

I have a mlir file to run by llvm-request-c-wrappers.

mlir file

module {
  func.func private @printMemrefF32(memref<*xf32>)
  func.func private @printMemrefI64(memref<*xi64>)
  func.func @forward(%arg0: memref<32000x4096xf32>, %arg1: memref<1x13xi64>) -> memref<1x13x4096xf32> {
    %c0 = arith.constant 0 : index
    %c1 = arith.constant 1 : index
    %c13 = arith.constant 13 : index
    %c4096 = arith.constant 4096 : index
    %U = memref.cast %arg0: memref<32000x4096xf32> to memref<*xf32>
    call @printMemrefF32(%U) : (memref<*xf32>) -> ()
    %a = memref.cast %arg1: memref<1x13xi64> to memref<*xi64>
    call @printMemrefI64(%a) : (memref<*xi64>) -> ()
    %alloc = memref.alloc() {alignment = 64 : i64} : memref<1x13x4096xf32>
    scf.for %arg2 = %c0 to %c1 step %c1 {
      scf.for %arg3 = %c0 to %c13 step %c1 {
        scf.for %arg4 = %c0 to %c4096 step %c1 {
          %0 = memref.load %arg1[%arg2, %arg3] : memref<1x13xi64>
          %1 = arith.index_cast %0 : i64 to index
          %2 = memref.load %arg0[%1, %arg4] : memref<32000x4096xf32>
          memref.store %2, %alloc[%arg2, %arg3, %arg4] : memref<1x13x4096xf32>
        }
      }
    }
    return %alloc : memref<1x13x4096xf32>
  }
}

lower pipeline

--convert-scf-to-cf
--expand-strided-metadata
--lower-affine
--memref-expand
--arith-expand
--convert-arith-to-llvm
--finalize-memref-to-llvm
--convert-arith-to-llvm
--convert-math-to-llvm
--convert-math-to-libm
--convert-func-to-llvm
--llvm-request-c-wrappers
--reconcile-unrealized-casts

struct define

typedef long int intptr_t;
template <class ElementType>
struct Tensor_descriptor_ {
    ElementType *allocated;
    ElementType *aligned;
    intptr_t offset=0;
    intptr_t sizes[2];
    intptr_t strides[2];
};

I use this struct to pass the parameter %arg0 and %arg1

But there are some errors. When I call @printMemref, it will segmentation fault and output

Unranked Memref base@ = 0x555eafeaee88 rank = 2 offset = 93865166696072 sizes = [140171248787080, 140170715836432] strides = [140170715836432, 0] data =

Is there a wrong define of Memref struct?

Thank you for your answer!

2

See an example of C++ invoking mlir which calls a callback: https://github.com/llvm/llvm-project/blob/976244bb845ca6e59139fffdf0372e4aba962ff1/mlir/unittests/ExecutionEngine/Invoke.cpp#L252

3

I want to use llvm-request-c-wrappers to invoke mlir, instead of JIT. Do you have some ideas? Thank you for your help.

4

Edit: sorry, I read the wrong post as the original question.
But it would be helpful to see how exactly you call the mlir_ciface wrapper from your C++ code

5

-llvm-request-c-wrappers does not invoke MLIR, it only sets the llvm.emit_c_interface attribute on the function(s) so that the proper interface and entry point is emitted.

If you build a .o from that, you should be able to link it and call it from C++ using similar code to what you can see around the invoke method.

This post describes a similar situation to yours: How to compile and link with other c/c++ programs

The reply here gives a step-by-step path, along with an example of an extern “C” declaration like you’d need: How to compile and link with other c/c++ programs - #10 by Wheest

6 
@${MLIR_OPT} ./test.mlir \
		-pass-pipeline="builtin.module(func.func(tosa-to-linalg-named),func.func(tosa-to-linalg),func.func(tosa-to-tensor),func.func(tosa-to-arith))" | \
	${MLIR_OPT} \
		--arith-expand \
		--empty-tensor-to-alloc-tensor \
		--linalg-bufferize \
		--convert-linalg-to-loops \
		--func-bufferize \
		--arith-bufferize \
		--tensor-bufferize \
		--finalizing-bufferize \
		--convert-vector-to-scf \
		--convert-scf-to-cf \
		--expand-strided-metadata \
		--lower-affine \
		--convert-vector-to-llvm \
		--memref-expand \
		--convert-arith-to-llvm \
		--finalize-memref-to-llvm \
		--convert-arith-to-llvm \
		--convert-math-to-llvm \
		--convert-math-to-libm \
		--convert-func-to-llvm \
		--llvm-request-c-wrappers \
		--reconcile-unrealized-casts | \
	${MLIR_TRANSLATE} \
		-mlir-to-llvmir | \
	${LLVM_AS} | \
	${LLC} -filetype=obj --relocation-model=pic -O0 -o test.o
g++ -g test_run.cc test.o ../../llvm/build/lib/libmlir_c_runner_utils.so -o test

extern "C" Tensor_descriptor_<float>* forward(Tensor_descriptor_<float>*, Tensor_descriptor_<long long>*);

int main() {
  Tensor_descriptor_<float> *arg0 =
      (Tensor_descriptor_<float> *)malloc(sizeof(Tensor_descriptor_<float>));
  Tensor_descriptor_<long long> *arg1 = (Tensor_descriptor_<long long> *)malloc(
      sizeof(Tensor_descriptor_<long long>));
  arg0->allocated = (float *)malloc(sizeof(float) * 32000 * 4096);
  arg0->aligned = arg0->allocated;
  for (int i = 0; i < 32000 * 4096; i++) {
    arg0->aligned[i] = 1;
  }
  arg0->sizes[0] = 32000;
  arg0->sizes[1] = 4096;
  arg0->strides[0] = 4096;
  arg0->strides[1] = 1;

  arg1->allocated = new long long[13];
  arg1->aligned = arg1->allocated;
  for (int i = 0; i < 13; i++) {
    arg1->aligned[i] = 1;
  }
  arg1->sizes[0] = 13;
  arg1->sizes[1] = 1;
  cout << "-----------------" << endl;
  Tensor_descriptor_<float>* result_alloc = forward(arg0, arg1);
  return 0;
}

This is my c++ code to call mlir. But in my case, I can’t use mlir_ciface_forward to define this mlir function. It will be undefined reference. When I use forward to name this mlir function, it will be right. Are there any errors in my lower pipeline or c++ code? Thank you for your help.

7

It looks like you’re calling the forward function directly instead of calling a c-wrapper around it. When you look at the generated LLVMIR, your function signature looks something like this:
llvm.func @forward(%arg0: !llvm.ptr, %arg1: !llvm.ptr, %arg2: i64, %arg3: i64, %arg4: i64, %arg5: i64, %arg6: i64, %arg7: !llvm.ptr, %arg8: !llvm.ptr, %arg9: i64, %arg10: i64, %arg11: i64, %arg12: i64, %arg13: i64) -> !llvm.struct<(ptr, ptr, i64, array<3 x i64>, array<3 x i64>)>
So all of the struct members of the memref descriptors have been promoted to function arguments. Your extern declaration looks very different.

That’s because no wrapper is generated for your function. When you add the attributes {llvm.emit_c_interface} to your func::FuncOp, that should happen. The last caveat is that because your func::FuncOp returns a memref, this will not show up in the same way in c-wrapper. Instead, the pointer to the corresponding descriptor gets turned into the first function argument.

For example, this:

module {
    func.func @mlir_main(%in : memref<1xi64>) -> memref<1xi64> attributes { llvm.emit_c_interface } {
        return %in : memref<1xi64>
    } 
}

Gets a c wrapper that looks like this:

llvm.func @_mlir_ciface_mlir_main(%arg0: !llvm.ptr, %arg1: !llvm.ptr)

And I can call it like this:

extern "C" void _mlir_ciface_mlir_main(Descriptor *, Descriptor *);

int main() {
  Descriptor in;
  Descriptor out;

  in.allocated = (int64_t *)0xdeadbeef;
  _mlir_ciface_mlir_main(&out, &in);

  std::cout << "The output pointer was " << out.allocated << "\n";

  return 0;
}
8

When I add attributes { llvm.emit_c_interface } in my mlir function and lower it to llvmir, it change to this.

llvm.func @forward(%arg0: !llvm.ptr, %arg1: !llvm.ptr, %arg2: i64, %arg3: i64, %arg4: i64, %arg5: i64, %arg6: i64) -> !llvm.struct<(ptr, ptr, i64, array<3 x i64>, array<3 x i64>)> attributes {llvm.emit_c_interface}

It seems like it is not helpful.

9

There should be a second function that looks like this:

llvm.func @_mlir_ciface_forward(%arg0: !llvm.ptr, %arg1: !llvm.ptr, %arg2: !llvm.ptr)

It does not replace the original, it is just a wrapper around it.

10

OK, I see. Thank you for your help. To add this attributes {llvm.emit_c_interface} to mlir function, I should how to change my lower pipeline. It seems like my current lower pipeline can’t add this to my mlir function

11

Yes, you could either add it by hand or use the --llvm-request-c-wrappers pass that you are already using. But you have to use it before --convert-func-to-llvm otherwise it will add the attribute after the lowering and you will not get a wrapper.

12

Thank you for your help. I got it!