In place memory bufferization

I’m trying to support a use case where a single func op contains a single region and single block with multiple ops that perform compute operations and pass data via tensors between them. So, the starting state is before any bufferization is done.
The compute ops will then be clustered into groups, and each group will be outlined to a different func op together with any required non compute ops.
After outlining, the module will be bufferized and the desired result should be such that data returned by one func and received by another should be placed on the same memory buffer and not on different memory buffers with copies between them.
A simple example of the initial state would be the following:

module {
  func.func @main(%t: tensor<2xf32>) -> (tensor<4xf32>) {
    %c = tensor.empty() : tensor<4xf32>
    %a = tensor.empty() : tensor<2xf32>
    %b = tensor.empty() : tensor<2xf32>
    %1 = linalg.exp ins(%t : tensor<2xf32>) outs(%a: tensor<2xf32>) -> tensor<2xf32>
    %2 = linalg.negf ins(%t : tensor<2xf32>) outs(%b: tensor<2xf32>) -> tensor<2xf32>
    %inserted_slice = tensor.insert_slice %1 into %c[0] [2] [1] : tensor<2xf32> into tensor<4xf32>
    %inserted_slice1 = tensor.insert_slice %2 into %inserted_slice[2] [2] [1] : tensor<2xf32> into tensor<4xf32>
    %d = tensor.empty() : tensor<4xf32>
    %3 = linalg.abs ins(%inserted_slice1 : tensor<4xf32>) outs(%d: tensor<4xf32>) -> tensor<4xf32>
    return %3 : tensor<4xf32>
  }
}

One possible desired output after bufferization would be the following:

module {
  func.func @producer1(%arg0: memref<4xf32, 1>, %arg1: memref<2xf32, 1>) -> memref<4xf32, 1> {
    %subview = memref.subview %arg0[0] [2] [1] : memref<4xf32, 1> to memref<2xf32, strided<[1]>, 1>
    linalg.exp ins(%arg1 : memref<2xf32, 1>) outs(%subview : memref<2xf32, strided<[1]>, 1>)
    return %arg0 : memref<4xf32, 1>
  }
  func.func @producer2(%arg0: memref<4xf32, 1>, %arg1: memref<2xf32, 1>) -> memref<4xf32, 1> {
    %subview = memref.subview %arg0[2] [2] [1] : memref<4xf32, 1> to memref<2xf32, strided<[1], offset: 2>, 1>
    linalg.negf ins(%arg1 : memref<2xf32, 1>) outs(%subview : memref<2xf32, strided<[1], offset: 2>, 1>)
    return %arg0 : memref<4xf32, 1>
  }
  func.func @consumer(%arg0: memref<4xf32, 1>) -> memref<4xf32, 1> {
    %alloc = memref.alloc() {alignment = 64 : i64} : memref<4xf32, 1>
    linalg.abs ins(%arg0 : memref<4xf32, 1>) outs(%alloc : memref<4xf32, 1>)
    return %alloc : memref<4xf32, 1>
  }
  func.func @main(%arg0: memref<2xf32, 1>) -> memref<4xf32, 1> {
    %alloc = memref.alloc() {alignment = 64 : i64} : memref<4xf32, 1>
    %0 = call @producer1(%alloc, %arg0) : (memref<4xf32, 1>, memref<2xf32, 1>) -> memref<4xf32, 1>
    %1 = call @producer2(%0, %arg0) : (memref<4xf32, 1>, memref<2xf32, 1>) -> memref<4xf32, 1>
    %2 = call @consumer(%1) : (memref<4xf32, 1>) -> memref<4xf32, 1>
    return %2 : memref<4xf32, 1>
  }
}

In this outcome the outputs of exp and negf and input to abs all use the same buffer.

On the contrary one possible non desired outcome would be:

module {
  func.func private @producer1(%arg0: memref<2xf32, 1>) -> memref<2xf32, 1> {
    %alloc = memref.alloc() {alignment = 64 : i64} : memref<2xf32, 1>
    linalg.exp ins(%arg0 : memref<2xf32, 1>) outs(%alloc : memref<2xf32, 1>)
    return %alloc : memref<2xf32, 1>
  }
  func.func private @producer2(%arg0: memref<2xf32, 1>) -> memref<2xf32, 1> {
    %alloc = memref.alloc() {alignment = 64 : i64} : memref<2xf32, 1>
    linalg.negf ins(%arg0 : memref<2xf32, 1>) outs(%alloc : memref<2xf32, 1>)
    return %alloc : memref<2xf32, 1>
  }
  func.func private @consumer(%arg0: memref<2xf32, 1>, %arg1: memref<2xf32, 1>) -> memref<4xf32, 1> {
    %alloc = memref.alloc() {alignment = 64 : i64} : memref<4xf32, 1>
    %alloc_0 = memref.alloc() {alignment = 64 : i64} : memref<4xf32, 1>
    %subview = memref.subview %alloc[0] [2] [1] : memref<4xf32, 1> to memref<2xf32, strided<[1]>, 1>
    memref.copy %arg0, %subview : memref<2xf32, 1> to memref<2xf32, strided<[1]>, 1>
    %subview_1 = memref.subview %alloc[2] [2] [1] : memref<4xf32, 1> to memref<2xf32, strided<[1], offset: 2>, 1>
    memref.copy %arg1, %subview_1 : memref<2xf32, 1> to memref<2xf32, strided<[1], offset: 2>, 1>
    linalg.abs ins(%alloc : memref<4xf32, 1>) outs(%alloc_0 : memref<4xf32, 1>)
    return %alloc_0 : memref<4xf32, 1>
  }
  func.func @main(%arg0: memref<2xf32, 1>) -> memref<4xf32, 1> {
    %0 = call @producer1(%arg0) : (memref<2xf32, 1>) -> memref<2xf32, 1>
    %1 = call @producer2(%arg0) : (memref<2xf32, 1>) -> memref<2xf32, 1>
    %2 = call @consumer(%0, %1) : (memref<2xf32, 1>, memref<2xf32, 1>) -> memref<4xf32, 1>
    return %2 : memref<4xf32, 1>
  }
}

In this case they do not use the same buffer and copies are done.

I’ve been trying to understand if it is possible to use the one shot bufferize pass to achieve the desired result, and if so how the outline step should be done. Another goal is to try to have the outline step as simple as possible.
To this end I made several experiments with the one shot bufferize pass via different outlining options.

The simplest outlining option I tried was to outline each group with all other non compute ops it depends on:

module {
  func.func @producer1(%arg0: tensor<2xf32>) -> (tensor<2xf32> ) {
    %a = tensor.empty() : tensor<2xf32>
    %1 = linalg.exp ins(%arg0 : tensor<2xf32>) outs(%a: tensor<2xf32>) -> tensor<2xf32>
    return %1 : tensor<2xf32>
  }

  func.func @producer2(%arg0: tensor<2xf32>) -> (tensor<2xf32> ) {
    %b = tensor.empty() : tensor<2xf32>
    %1 = linalg.negf ins(%arg0 : tensor<2xf32>) outs(%b: tensor<2xf32>) -> tensor<2xf32>
    return %1 : tensor<2xf32>
  }

  func.func @consumer(%arg0: tensor<2xf32>, %arg1: tensor<2xf32>) -> (tensor<4xf32> ) {
    %0 = tensor.empty() : tensor<4xf32>
    %1 = tensor.empty() : tensor<4xf32>
    %inserted_slice = tensor.insert_slice %arg0 into %0[0] [2] [1] : tensor<2xf32> into tensor<4xf32>
    %inserted_slice1 = tensor.insert_slice %arg1 into %inserted_slice[2] [2] [1] : tensor<2xf32> into tensor<4xf32>
    %2 = linalg.abs ins(%inserted_slice1 : tensor<4xf32>) outs(%1: tensor<4xf32>) -> tensor<4xf32>
    return %2 : tensor<4xf32>
  }

  func.func @main(%t: tensor<2xf32>) -> (tensor<4xf32>) {
      %0 = call @producer1(%t) : (tensor<2xf32>) -> (tensor<2xf32>)
      %1 = call @producer2(%t) : (tensor<2xf32>) -> (tensor<2xf32>)
      %2 = call @consumer(%0, %1) : (tensor<2xf32>, tensor<2xf32>) -> (tensor<4xf32>)
      return %2 : tensor<4xf32>
  }
}

The outcome of bufferization is not desirable since exp negf and abs do not share the same buffer:

module {
  func.func @producer1(%arg0: memref<2xf32, 1>) -> memref<2xf32, 1> {
    %alloc = memref.alloc() {alignment = 64 : i64} : memref<2xf32, 1>
    linalg.exp ins(%arg0 : memref<2xf32, 1>) outs(%alloc : memref<2xf32, 1>)
    return %alloc : memref<2xf32, 1>
  }
  func.func @producer2(%arg0: memref<2xf32, 1>) -> memref<2xf32, 1> {
    %alloc = memref.alloc() {alignment = 64 : i64} : memref<2xf32, 1>
    linalg.negf ins(%arg0 : memref<2xf32, 1>) outs(%alloc : memref<2xf32, 1>)
    return %alloc : memref<2xf32, 1>
  }
  func.func @consumer(%arg0: memref<2xf32, 1>, %arg1: memref<2xf32, 1>) -> memref<4xf32, 1> {
    %alloc = memref.alloc() {alignment = 64 : i64} : memref<4xf32, 1>
    %alloc_0 = memref.alloc() {alignment = 64 : i64} : memref<4xf32, 1>
    %subview = memref.subview %alloc[0] [2] [1] : memref<4xf32, 1> to memref<2xf32, strided<[1]>, 1>
    memref.copy %arg0, %subview : memref<2xf32, 1> to memref<2xf32, strided<[1]>, 1>
    %subview_1 = memref.subview %alloc[2] [2] [1] : memref<4xf32, 1> to memref<2xf32, strided<[1], offset: 2>, 1>
    memref.copy %arg1, %subview_1 : memref<2xf32, 1> to memref<2xf32, strided<[1], offset: 2>, 1>
    linalg.abs ins(%alloc : memref<4xf32, 1>) outs(%alloc_0 : memref<4xf32, 1>)
    return %alloc_0 : memref<4xf32, 1>
  }
  func.func @main(%arg0: memref<2xf32, 1>) -> memref<4xf32, 1> {
    %0 = call @producer1(%arg0) : (memref<2xf32, 1>) -> memref<2xf32, 1>
    %1 = call @producer2(%arg0) : (memref<2xf32, 1>) -> memref<2xf32, 1>
    %2 = call @consumer(%0, %1) : (memref<2xf32, 1>, memref<2xf32, 1>) -> memref<4xf32, 1>
    return %2 : memref<4xf32, 1>
  }
}

Another option I tried was to replace the small tensor.empty ops with an extract from the large one similar to what is done in eliminate empty tensors pass, and outline in a less trivial way before bufferization:

module {
  func.func @producer1(%arg0: tensor<2xf32>, %buffer: tensor<4xf32>) -> (tensor<4xf32> ) {
    %extracted_slice = tensor.extract_slice %buffer[0] [2] [1] : tensor<4xf32> to tensor<2xf32>
    %1 = linalg.exp ins(%arg0 : tensor<2xf32>) outs(%extracted_slice: tensor<2xf32>) -> tensor<2xf32>
    %inserted_slice = tensor.insert_slice %1 into %buffer[0] [2] [1] : tensor<2xf32> into tensor<4xf32>
    return %inserted_slice : tensor<4xf32>
  }

  func.func @producer2(%arg0: tensor<2xf32>, %buffer: tensor<4xf32>) -> (tensor<4xf32> ) {
    %extracted_slice = tensor.extract_slice %buffer[2] [2] [1] : tensor<4xf32> to tensor<2xf32>
    %1 = linalg.exp ins(%arg0 : tensor<2xf32>) outs(%extracted_slice: tensor<2xf32>) -> tensor<2xf32>
    %inserted_slice = tensor.insert_slice %1 into %buffer[2] [2] [1] : tensor<2xf32> into tensor<4xf32>
    return %inserted_slice : tensor<4xf32>
  }

  func.func @consumer(%buffer: tensor<4xf32>) -> (tensor<4xf32> ) {
    %0 = tensor.empty() : tensor<4xf32>
    %1 = linalg.abs ins(%buffer : tensor<4xf32>) outs(%0: tensor<4xf32>) -> tensor<4xf32>
    return %1 : tensor<4xf32>
  }

  func.func @main(%t: tensor<2xf32>) -> (tensor<4xf32>) {
      %buffer = tensor.empty() : tensor<4xf32>
      %0 = call @producer1(%t, %buffer) : (tensor<2xf32>, tensor<4xf32>) -> (tensor<4xf32>)
      %1 = call @producer2(%t, %0) : (tensor<2xf32>, tensor<4xf32>) -> (tensor<4xf32>)
      %2 = call @consumer(%1) : (tensor<4xf32>) -> (tensor<4xf32>)
      return %2 : tensor<4xf32>
  }
}

In this case the output of bufferization did lead to a single buffer, but there are additional copies that were generated:

module {
  func.func @producer1(%arg0: memref<2xf32, 1>, %arg1: memref<4xf32, 1>) -> memref<4xf32, 1> {
    %subview = memref.subview %arg1[0] [2] [1] : memref<4xf32, 1> to memref<2xf32, strided<[1]>, 1>
    linalg.exp ins(%arg0 : memref<2xf32, 1>) outs(%subview : memref<2xf32, strided<[1]>, 1>)
    %subview_0 = memref.subview %arg1[0] [2] [1] : memref<4xf32, 1> to memref<2xf32, strided<[1]>, 1>
    memref.copy %subview, %subview_0 : memref<2xf32, strided<[1]>, 1> to memref<2xf32, strided<[1]>, 1>
    return %arg1 : memref<4xf32, 1>
  }
  func.func @producer2(%arg0: memref<2xf32, 1>, %arg1: memref<4xf32, 1>) -> memref<4xf32, 1> {
    %subview = memref.subview %arg1[2] [2] [1] : memref<4xf32, 1> to memref<2xf32, strided<[1], offset: 2>, 1>
    linalg.exp ins(%arg0 : memref<2xf32, 1>) outs(%subview : memref<2xf32, strided<[1], offset: 2>, 1>)
    %subview_0 = memref.subview %arg1[2] [2] [1] : memref<4xf32, 1> to memref<2xf32, strided<[1], offset: 2>, 1>
    memref.copy %subview, %subview_0 : memref<2xf32, strided<[1], offset: 2>, 1> to memref<2xf32, strided<[1], offset: 2>, 1>
    return %arg1 : memref<4xf32, 1>
  }
  func.func @consumer(%arg0: memref<4xf32, 1>) -> memref<4xf32, 1> {
    %alloc = memref.alloc() {alignment = 64 : i64} : memref<4xf32, 1>
    linalg.abs ins(%arg0 : memref<4xf32, 1>) outs(%alloc : memref<4xf32, 1>)
    return %alloc : memref<4xf32, 1>
  }
  func.func @main(%arg0: memref<2xf32, 1>) -> memref<4xf32, 1> {
    %alloc = memref.alloc() {alignment = 64 : i64} : memref<4xf32, 1>
    %0 = call @producer1(%arg0, %alloc) : (memref<2xf32, 1>, memref<4xf32, 1>) -> memref<4xf32, 1>
    %1 = call @producer2(%arg0, %0) : (memref<2xf32, 1>, memref<4xf32, 1>) -> memref<4xf32, 1>
    %2 = call @consumer(%1) : (memref<4xf32, 1>) -> memref<4xf32, 1>
    return %2 : memref<4xf32, 1>
  }
}

An additional experiment I tried was to just bufferize without outlining, and the outcome does include the undesired extra buffers and copies:

module {
  func.func @main(%arg0: memref<2xf32, 1>) -> memref<4xf32, 1> {
    %alloc = memref.alloc() {alignment = 64 : i64} : memref<4xf32, 1>
    %alloc_0 = memref.alloc() {alignment = 64 : i64} : memref<2xf32, 1>
    %alloc_1 = memref.alloc() {alignment = 64 : i64} : memref<2xf32, 1>
    linalg.exp ins(%arg0 : memref<2xf32, 1>) outs(%alloc_0 : memref<2xf32, 1>)
    linalg.negf ins(%arg0 : memref<2xf32, 1>) outs(%alloc_1 : memref<2xf32, 1>)
    %subview = memref.subview %alloc[0] [2] [1] : memref<4xf32, 1> to memref<2xf32, strided<[1]>, 1>
    memref.copy %alloc_0, %subview : memref<2xf32, 1> to memref<2xf32, strided<[1]>, 1>
    %subview_2 = memref.subview %alloc[2] [2] [1] : memref<4xf32, 1> to memref<2xf32, strided<[1], offset: 2>, 1>
    memref.copy %alloc_1, %subview_2 : memref<2xf32, 1> to memref<2xf32, strided<[1], offset: 2>, 1>
    %alloc_3 = memref.alloc() {alignment = 64 : i64} : memref<4xf32, 1>
    linalg.abs ins(%alloc : memref<4xf32, 1>) outs(%alloc_3 : memref<4xf32, 1>)
    return %alloc_3 : memref<4xf32, 1>
  }
}

I also tried to first eliminate the small tensor.empty ops before bufferizing:

module {
  func.func @main(%arg0: tensor<2xf32>) -> tensor<4xf32> {
    %0 = tensor.empty() : tensor<4xf32>
    %extracted_slice = tensor.extract_slice %0[0] [2] [1] : tensor<4xf32> to tensor<2xf32>
    %extracted_slice1 = tensor.extract_slice %0[2] [2] [1] : tensor<4xf32> to tensor<2xf32>
    %2 = linalg.exp ins(%arg0 : tensor<2xf32>) outs(%extracted_slice : tensor<2xf32>) -> tensor<2xf32>
    %3 = linalg.negf ins(%arg0 : tensor<2xf32>) outs(%extracted_slice1 : tensor<2xf32>) -> tensor<2xf32>
    %inserted_slice = tensor.insert_slice %2 into %0[0] [2] [1] : tensor<2xf32> into tensor<4xf32>
    %inserted_slice_0 = tensor.insert_slice %3 into %inserted_slice[2] [2] [1] : tensor<2xf32> into tensor<4xf32>
    %4 = tensor.empty() : tensor<4xf32>
    %5 = linalg.abs ins(%inserted_slice_0 : tensor<4xf32>) outs(%4 : tensor<4xf32>) -> tensor<4xf32>
    return %5 : tensor<4xf32>
  }
}

In this case the outcome included one additional small buffer and 2 copies:

module {
  func.func @main(%arg0: memref<2xf32, 1>) -> memref<4xf32, 1> {
    %alloc = memref.alloc() {alignment = 64 : i64} : memref<4xf32, 1>
    %subview = memref.subview %alloc[0] [2] [1] : memref<4xf32, 1> to memref<2xf32, strided<[1]>, 1>
    %subview_0 = memref.subview %alloc[2] [2] [1] : memref<4xf32, 1> to memref<2xf32, strided<[1], offset: 2>, 1>
    %alloc_1 = memref.alloc() {alignment = 64 : i64} : memref<2xf32, 1>
    linalg.exp ins(%arg0 : memref<2xf32, 1>) outs(%subview : memref<2xf32, strided<[1]>, 1>)
    linalg.negf ins(%arg0 : memref<2xf32, 1>) outs(%alloc_1 : memref<2xf32, 1>)
    %subview_2 = memref.subview %alloc[0] [2] [1] : memref<4xf32, 1> to memref<2xf32, strided<[1]>, 1>
    memref.copy %subview, %subview_2 : memref<2xf32, strided<[1]>, 1> to memref<2xf32, strided<[1]>, 1>
    %subview_3 = memref.subview %alloc[2] [2] [1] : memref<4xf32, 1> to memref<2xf32, strided<[1], offset: 2>, 1>
    memref.copy %alloc_1, %subview_3 : memref<2xf32, 1> to memref<2xf32, strided<[1], offset: 2>, 1>
    %alloc_4 = memref.alloc() {alignment = 64 : i64} : memref<4xf32, 1>
    linalg.abs ins(%alloc : memref<4xf32, 1>) outs(%alloc_4 : memref<4xf32, 1>)
    return %alloc_4 : memref<4xf32, 1>
  }
}

Any assistance in understanding what analysis the one shot bufferize pass can do to help with sharing buffers between func ops and reducing copies would be greatly appreciated. @matthias-springer

In this case the output of bufferization did lead to a single buffer, but there are additional copies that were generated:

The copies should fold away with -cse -canonicalize. It is expected that tensor.insert_slice sometimes bufferizes to memref.copy ops that have no effect (copies between two identical buffers). Is the resulting IR what you expected?

Generally speaking, if you want to avoid an allocation inside of a function, a “destination tensor” (%buffer) must be passed to that function. Your listing with the extract_slice/insert_slice ops looks good to me.

Yes, except for the copies.

That is an important point.
I am trying to understand if it would be preferable to outline before or after bufferization. An advantage for doing it pre bufferization would be if it made outlining much more simple. But it seems that outlining pre bufferization and getting the desired bufferization outcome would require doing an analysis while the IR is still in tensors, and understand which tensors should become buffers that will be shared between func ops. I wanted to know if it would be possible to avoid this analysis before outlining, and perhaps the one shot bufferize pass could make the required changes to achieve the desired buffer sharing, but from your answer I understand that this is not the case

Yes, except for the copies.

The copies are gone with -cse -canonicalize, right?

I am trying to understand if it would be preferable to outline before or after bufferization.

I would outline after bufferization for two reasons:

  1. The function signature could be simpler. Conceptually, the bufferized function does not have to return anything, so maybe it makes the outlining simpler.
  2. There are certain limitations wrt. function calls in One-Shot Bufferize. E.g., recursion is not supported. But also functions calls are not as thoroughly tested in general, so there is a higher chance that things could go wrong. (It should work though.)

I wanted to know if it would be possible to avoid this analysis before outlining, and perhaps the one shot bufferize pass could make the required changes to achieve the desired buffer sharing, but from your answer I understand that this is not the case

All buffer sharing/reuse is driven by Destination Passing Style (DPS). So you still have to manually set up the functions / calls in a suitable way.

Yes, for copies that have no affect this flow will remove them.
But if you look at the last example I wrote, it runs one shot on a single func and generates 2 copies

  • One that has no affect and would be eliminated by this flow
  • One that copies between 2 buffers and does have an affect and will not be removed

This example is basically the case where you would have a tosa.concat then transform it to use tensor insert and extract instead. I believe it is a rather common use case, but instead of getting a single large shared buffer, we get one large and one small, and then must have a copy to move data between them (the other half of the data is used in place with the first copy that doesn’t have an effect).
wouldn’t you expect that for this use case the output of the one shot would be to get a single large buffer? If not, is live range a consideration for this?

Since one shot bufferize is a pass than can work in module level, wouldn’t it make sense to extend the pass to support optimizations also for non DPS dialects?
func dialect is widely used, but from what you wrote, my understanding is that even in cases where the call graph is statically known and buffer sharing between func ops is possible, it won’t be done unless the func ops are written in a very specific way.

ping @matthias-springer

The One-Shot Bufferize pass is quite sensitive to op ordering. This IR will bufferize such that there is only one allocation and no copy:

  func.func @main(%t: tensor<2xf32>) -> (tensor<4xf32>) {
    %c = tensor.empty() : tensor<4xf32>

    %ex1 = tensor.extract_slice %c[0][2][1] : tensor<4xf32> to tensor<2xf32>
    %1 = linalg.exp ins(%t : tensor<2xf32>) outs(%ex1: tensor<2xf32>) -> tensor<2xf32>
    %in1 = tensor.insert_slice %1 into %c[0] [2] [1] : tensor<2xf32> into tensor<4xf32>

    %ex2 = tensor.extract_slice %in1[2][2][1] : tensor<4xf32> to tensor<2xf32>
    %2 = linalg.negf ins(%t : tensor<2xf32>) outs(%ex2: tensor<2xf32>) -> tensor<2xf32>
    %in2 = tensor.insert_slice %2 into %in1[2] [2] [1] : tensor<2xf32> into tensor<4xf32>

    %3 = linalg.abs ins(%in2 : tensor<4xf32>) outs(%in2: tensor<4xf32>) -> tensor<4xf32>
    return %3 : tensor<4xf32>
  }

The following IR, while computing the same thing, results in an extra buffer copy:

  func.func @main(%t: tensor<2xf32>) -> (tensor<4xf32>) {
    %c = tensor.empty() : tensor<4xf32>

    %ex1 = tensor.extract_slice %c[0][2][1] : tensor<4xf32> to tensor<2xf32>
    %1 = linalg.exp ins(%t : tensor<2xf32>) outs(%ex1: tensor<2xf32>) -> tensor<2xf32>

    %ex2 = tensor.extract_slice %c[2][2][1] : tensor<4xf32> to tensor<2xf32>
    %2 = linalg.negf ins(%t : tensor<2xf32>) outs(%ex2: tensor<2xf32>) -> tensor<2xf32>

    %in1 = tensor.insert_slice %1 into %c[0] [2] [1] : tensor<2xf32> into tensor<4xf32>
    %in2 = tensor.insert_slice %2 into %in1[2] [2] [1] : tensor<2xf32> into tensor<4xf32>

    %3 = linalg.abs ins(%in2 : tensor<4xf32>) outs(%in2: tensor<4xf32>) -> tensor<4xf32>
    return %3 : tensor<4xf32>
  }

It should be possible to fix that by improving the analysis (OneShotAnalysis.cpp). The main problem is that we do not really reason about tensor subsets in the bufferization analysis yet. (Some very simple cases are handled.) I.e., we know that a buffer is written to, but we do not really take into account the fact that only a small part of it is written to.

Any improvement to this would be great. But it’s a lot of work and the devil is in the detail. Most of what we have today was driven by the use cases that I had.

Since one shot bufferize is a pass than can work in module level, wouldn’t it make sense to extend the pass to support optimizations also for non DPS dialects?

Non-DPS ops are difficult to support. The reason why we have DPS is so that the user can inject additional hints about the buffer in which a result should materialize. Without DPS, the problem is kind of similar to register allocation, which is NP-complete. If you have an idea how to bufferize efficiently without DPS, please let me know!

The current BufferizableOpInterface interface doesn’t support getting information about tensor subsets, but such information would be required for the analysis to have broader reasoning about subsets, which could lead to the elimination of allocations and copies in various cases that involve tensor subsets.
One way to get such information would be to extend the interface.
Another more limited way, would be to add knowledge in the analysis pass of the relevant behavior of several specific ops that handle subsets for example {tensor.insert, tensor.insert_slice, tensor.extract, tensor.extract_slice}.
Both options would require the same updates in the analysis, but the second option wouldn’t require updates in an interface that is widely used.
@matthias-springer any thoughts about the right way to go about adding such functionality to the analysis?

The internal data structures that must be extended in the analysis are aliasInfo and equivalenceInfo. At the moment they just store sets of SSA values. We something more fine grained.

I think the BufferizableOpInterface should also be extended a little bit, because you need a way to pass fine-grained aliasing/equivalence information to the analysis. The relevant interface methods are getAliasingValues and getAliasingOpOperands. We already store a BufferRelation in AliasingValue. That field is currently an enum: the only possible values for the buffer relation are Unknown and Equivalent. There’s even a TODO in there:

/// Specifies a fine-grain relationship between buffers to enable more analysis.
enum class BufferRelation {
  Unknown,
  // TODO: ResultContainsOperand,
  // TODO: OperandContainsResult,
  Equivalent
};

We could replace BufferRelation with a C++ struct that contains all the necessary information. I’m not sure what exactly that struct would look like. Probably something similar to HyperrectangularSlice.

Or maybe we don’t need BufferRelation in the BufferizableOpInterface at all. We could build something on top of SubsetOpInterface. One reason for adding this interface was to enable stronger bufferization analyses.