Should we have 0-D Vectors?

The 0-D vector vs scalar case has been showing up in a bunch of corner cases recently.
The latest is that<?xf32>, tensor<?xf32>) -> tensor<f32> cannot vectorize on tensors due to mistmatches between tensor<f32> and vector<f32>.

There are a bunch of tradeoffs involved that I don’t think are worth unpacking just yet if there is a simple consensus that can emerge.

TL;DR, I am contemplating allowing vector<T> for T of type float or integer (and maybe index once the DataLayout representation for index is a little more fleshed out but that’s beside the point).

vector<T> would canonicalize to T where it makes sense.
Before/at the point of lowering to the LLVM dialect, all vector<T> would have become T.

Have people thought about this corner case and have formed opinions on the topic?


Conceptually, I don’t see a problem of allowing 0D vectors since we have nD vectors. There is no strict relation with hardware vectors anymore.

I don’t see a problem here either, makes sense to me.

Sounds good to me since they are not zero-sized but just 0-d. Their size/layout I believe is determined in a way identical to vector<1xT> in all ways.

SGTM. I’ve always felt this was weird. And it seems to be a totally systematic extension of the current behavior.

On the topic of canonicalizing vector<T> to T, it feels wrong to me. At the hardware level, changing vector<T> to T can result in an expensive scalar readback operation (analogous to changing tensor<T> to T, but at a different hierarchy level). I suspect it should be done based on a cost model similar to what we do (or plan to do) for Detensorize.

1 Like

Isn’t that about not turning a 1D vector with 1 element to scalar vs a 0D one? E.g., in Nicholas case it is a scalar we just represent it as a 0D vector, but it is just two forms of representing the same thing (or rather this a way of representing a pure scalar as a vector so that we have consistent behavior for scalar operands too). So a vector<0xT> becomes a T only superficially as it was really T all along, While a 1D 1 element vector I agree could be costly/is actually a representational change that needs to be driven.

vector<T> (not vector<0xT>) effectively carries a bit of information that the value is stored in the vector register file (as opposed to the scalar register file). At some point in the compilation stack that needs to become load-bearing and cannot be canonicalized away.

For example, I’ve worked on targets where reading a vector element into a scalar register is illegal.