[RFC] Should We Restrict the Usage of 0-D Vectors in the Vector Dialect?

[banach-space]

@rengolin

Also, the proposers need to work with the folks raising issues (Mahesh, Stella, others) to make sure the use actually does not impact external projects in a way that they are not able to work on.

+1

In particular, folks, if I am misinterpreting/misunderstanding your concerns, please correct me!

@Groverkss

The problem with 0-d vector support in vector dialect isn’t 0-d vectors themselves, it’s how operations handle them as special cases.

+1

@Groverkss

The above definitions make it clear when a operation should support 0-D vectors
(Category 1), when it shouldn’t (Category 2) and when it depends on what it’s targetting (Category 3).

You have just proposed a much needed update to this section of the Vector dialect on “Hardware Vector Ops” vs “Virtual Vector Ops” (that document has served us very well, but is a bit out-dated):

• 'vector' Dialect - MLIR

As in, we should use the taxonomy that you proposed to update that document.

@Groverkss

• Put each operation in one of these two categories, and only allow 0-d vectors for Category 1.

This is precisely what I had in mind, but takes things much further. Thank you for sharing!

@Groverkss

Split vector.extract into two operations and make both of them Category 1.

TBH, I’m a bit concerned about introducing a new Vector Op - this feels like vector.extractelement/vector.insertelement, and we should be mindful of:

• https://discourse.llvm.org/t/rfc-psa-remove-vector-extractelement-and-vector-insertelement-ops-in-favor-of-vector-extract-and-vector-insert-ops

Perhaps we can avoid that? See below.

@Groverkss

The current RFC is trying to make these operations strictly Category 2, which is why I’m -1 on this RFC.

No Let me clarify (by special-casing my example):

Allow:

// 1. %src + %dst are the _indexed_ inputs and can be 0-D
// 2. if (numIndices == srcRank) --> result is _always_ f32
// 3. The restriction only applies to the non-indexed argument.
%0 = vector.extract %src[] : f32 from vector<f32>
%1 = vector.insert %v, %dst[] : f32 into vector<f32>

Disallow:

// 1. %src + %dst are the _indexed_ inputs that can be 0-D
// 2. if (numIndices == srcRank) --> result is f32 _or_ vector<f32>
// 3. No restriction on the non-index arguments.
%0 = vector.extract %src[] : vector<f32> from vector<f32>
%1 = vector.insert %v, %dst[] : vector<f32> into vector<f32>

The ambiguity is caused by the fact that, when numIndices == srcRank, the non-indexed argument could either be f32 or vector<f32>. We need to restrict that. I proposed f32, you are suggesting vector<f32>. Whichever one we pick, we should stick to it consistently.

Put differently, I am still hoping that we can re-use vector.insert/vector.extract. If we discover otherwise, then we can just introduce vector.extract_scalar as you proposed.

Btw, I find the split into the “indexed” and “non-indexed” arguments very helpful. To me, we have always been missing that when reasoning about “read”/“write” operations. We should incorporate that into your taxonomy.

@Groverkss

An operation defined as Category 1 can be used as a Category 2 or Category operation, but the other way around is not true.

Hm, if Category 1 allows 0-D vectors and Category 2 does not allow 0-D vectors, shouldn’t this statement be reversed?

An operation defined as Category 3 can be used as a Category 2 or Category 1 operation, but the other way around is not true.

As in, Category 3 is the strictest and hence these Ops are also valid Category 2 and 1 Ops? Or:

• Bucket 1 = Category 1 + Category 2 + Category 3
• Bucket 2 = Category 2 + Category 3
• Bucket 3 = Category 3

Also, @Groverkss, thanks again for all your effort going into improving this

-Andrzej