[RFC] Dynamic Vector Semantics for the MLIR Vector Dialect

[zhanghb97]

Hi @clattner, thank you for your reply and feedback!

@clattner

why do you think this should be part of the vector type, instead of a new variable length vector type?

Existing SIMD/vector architectures can be categorized into fixed-length (X86 AVX series), scalable-length (Arm SVE), and dynamic-length (RVV). I think the vector type should cover these representations. Fixed and Scalable cases are already well-supported, so this proposal wants to integrate dynamic features into the existing vector type.

Moreover, we had several principles in mind when designing this proposal:

• Not to break existing users of fixed and scalable Vectors.
• To make dynamic vector computations compatible with the existing Vector Dialect ecosystem.
• To reuse as much existing infrastructure as possible to avoid fragmentation.

Throughout our discussions, we considered many solutions. Based on these principles, we finalized this design, aiming for a balance between functionality and reusability.

@clattner

To give one specific concern, any operation that depends on vector.get_vl and vector.set_vl will implicitly have side effects now of reading this state register, and therefore all common add/sub/mul ops on vectors get more difficult to work with.

vector.get_vl and vector.set_vl are mainly designed for dynamic vector architectures (like RVV), where the hardware (dependent on state registers) determines the vector computation length at runtime.

@clattner

Another is that you’ll break a core operation of the current simd type which is “give me the dimensions”

For fixed/scalable SIMD architectures, we hope to convert from dynamic representations to fixed/scalable vectors, thereby avoiding side effects and the issue of lacking static information in SIMD architectures.

@clattner

• Make things like “add” take an additional operand which is the current vector length and do a scheduling pass later to insert the side effecting set_vl’s.

This was one of the alternative options. However, this approach may alter the current Vector Dialect ecosystem a lot, and the mixed-use of a vector length operand with fixed/scalable vector types may create confusing semantics. Hence, this method was not included in our final proposal.

@clattner

Duplicate all the operations, e.g. have a “add” and “add_variable_length” that both work on vector types but have different assumptions (e.g. verifier can reject wrong permutations).

The earlier RVV Dialect proposal had a similar idea, but there seemed to be concerns about fragmentation. This approach seems to split existing SIMD operations and dynamic vector computations.

@clattner

Given that you’d pervasively make thing side effecting and introduce other semantics (e.g. they can’t call “get length” on them anymore, it seems like it’s a false equivalence to make these “the same”.

From an implementation perspective, we can use a series of checks to avoid crash when taking static attributes from dynamic vectors. As for the semantics, it depends on whether we think that a vector abstraction should have a definite length of information. This could be an open topic.

In summary, our proposal is designed from a reusability perspective. Personally, I would also be very open to using a new representation and type to support dynamic vector computations.