we are working on adding support for complex types in LHLO.
There seems to be no way to do simple ops like Add, Mul, Sub on complex numbers or create complex number from two scalars or extract real/imaginary parts from a given Value of ComplexType. Should these ops be defined in ComplexOps.td, similarly to how ExtractElementOp is defined in VectorOps.td?
Should the arithmetic operations be defined for ComplexType only or AddFOp should be extended to support complex numbers like it does with vectors and tensors now? I personally prefer having ops in ComplexOps.td because most of the operations wonât be elementwise.
Should ComplexType(f32) be lowered !llvm.type<"<2 x float>"> when we go to LLVM?
And the most important question: is anyone working on that right now because it looks like ComplexType was added a long time ago.
Iâd prefer to put them in their own dialect for now. Extending AddF is hard already due to the name. This is also in line with the idea of making Standard go away eventually, so we can start by not adding more to it.
I am not particularly opposed to adding support for complex numbers to standard floating-point arithmetic operations at their current stage, one can argue that it is a reasonably straightforward extension. It may get trickier if (or rather when) we start adding support for fast-math flags on these operations or if we need to consider semantic edge cases like saturation to infinity or rounding modes. Also note that MLIRâs complex types can have both integer and float elements, so you will likely need to replicate both integer and complex arithmetics.
A handful of complex-only operations can go into a separate dialect, like cplx.create : (?,?) -> complex<?>, cplx.re : complex<?> -> ?, cplx.im : complex<?> -> ? and cplx.conjugate : complex<?> -> complex<?>. Or can go to standard given that the type is also standard. A side question is the dialect name: complex seems to be a keyword reserved for the type⌠My general preference is towards more modular dialects, so I would vote for creating a separate dialect whether we reuse std arithmetics or not.
I am not sure I understand this. What prevents us from having elementwise addition of tensors with complex elements?
I think it should be lowered to two standard f32 before even going to LLVM. This will let you benefit from all kinds of lower-level simplifications on standard arithmetics. If we decide otherwise, Iâd suggest an array or a struct rather than a vector to avoid messing up with the vectorizer.
Just a random opinion, but⌠until the standard dialect actually goes away, I donât think it makes sense to avoid evolving it. This will just make a mess, pushing logical extensions somewhere else.
To be clear, Iâm not arguing that adding complex numbers to std makes sense, Iâm just pushing back against the sentiment.
Is there actually any plan or proposal to eliminate the standard dialect? I know that some people would like to see that, but it isnât clear to me how a replacement would be better in practice.
+1 Trying to define a new dialect here isnât useful â because these ops will likely move again. (For eg., if there is a arith ops dialect, these may go there.) In addition, youâll have to keep switching dirs for these few ops in the meanwhile. Better to keep them in one place until there is a convincing partitioning - thatâll also make the partitioning easier. Besides, itâs not clear why arithmetic ops on complex types warrant a dialect separate from the std dialect or say âarithmetic opsâ even with std partitioned.
Have all complex ops in Ops.td, seperately from std float/int arith ops.
Have all complex ops in Ops.td, reusing std float/int arith ops.
Have all complex ops in ComplexOps.td.
Have only im, re, create, conjugate in ComplexOps.td and arith ops in Ops.td, reusing std float/int arith ops.
Have only im, re, create, conjugate in ComplexOps.td and arith ops in Ops.td separately from std float/int arith ops.
I am opposed to (1) and (2), because I think that creation and extraction of real/imaginary parts and other specific ops should be in a separate dialect.
@bondhugula are there plans to have âarith ops dialectâ or some other partitioning of Std?
Besides, itâs not clear why arithmetic ops on complex types warrant a dialect separate from the std dialect or say âarithmetic opsâ even with std partitioned.
Having complex-specific ops in ComplexOps.tdwould be consistent with what was done in VectorOps.td. The main question is what to do with complex arithmetics and also functions like Exp, Log, Sqrt, etc.
Reusing std arithmetics would be much easier if we had single AddOp instead of AddIOp and AddFOp. Is there some special reasong for that? Even if we have AddOp only, we can lower to add/fadd correctly when going to LLVM.
@ftynse, sorry. What I meant was that if you represent complex numbers as 2D vectors, then only Add, Sub will be trivial and elementwise, for division, multiplication the vector components start âmixingâ and we will have to write custom lowering. We would have to that in any case though.
You are right that any elementwise op on tensor<? x f32> would stay elementwise on tensor<? x complex<f32>>.
This has been discussed numerous times, and is even stated in the header of the dialect documentation, but I donât think anybody is actively working on this right now.
I would tend to go for consistency. That is, if you introduce new ops for âaddâ and âmulâ, you should also introduce new ops for âexpâ and âsqrtâ. The latter is an interesting example, actually. On the float domain only, sqrt-of-negative is an error (or a nan), but on the complex domain, we can have an sqrt : f? -> complex<f?> that always works.
We followed LLVM IR for those. I think of the reasons, beyond code generation, is that these ops care about disjoint sets of additional properties (that MLIR currently does not model): integer arithmetics cares about over/underflow while float arithmetics cares about fast-math and floating point environment. Having a combination of these properties on a single op sounds quite awkward, hence my initial comment about it getting tricky when MLIR starts adding support for these options.
Just a comment. Having sqrt : f32->complex<f32> wonât be allowed because of SameOperandAndResultType. One has to create complex from f32 first and only then call sqrt. In that case we can have one Exp op that can take either f? or complex(f?) but two different lowerings.
@ftynse then we are going for (1): having complex ops in Ops.td but separately from all std int/float arithmetic operations with names like AddCOp, MulCOp, etc. What would the names for Exp(complex) be? ExpCOp?
I donât really have a preference for (1) or any other option, I just find it easier to have a record of the rationale for taking one of those. Creating a new dialect implies defining the goals and the scope of the dialect, to put it in the doc header if anything
Hi @pifon2a ,
I was curious if this went anywhere. Did you end up implementing a âComplexâ dialect? We would like to add support for things like zgemm, but before doing anything I am try to understand what is already there.
I think, for dialects, our