Sparse tensor lowering to GPU

[jim22k]

Has anyone tried to target GPUs with sparse tensors?

[jim22k]

Looks like @aartbik has been working on this in stealth mode

Sparse Compiler and GPU Code-Generation

It is time to continue the older sparse compiler thread into a new topical thread. Even though the MLIR sparse compiler is obviously meant as a re-targetable tool to exploit sparsity, most development so far has been focused on generating sparse code that runs on a CPU. Even though CPUs are least used for ML problems, having the ability to easily exploit unstructured sparsity still makes this a viable approach for accelerating sparse problems with very high sparsities. Recently, however, we s…

[aartbik]

@jim22k

Looks like @aartbik has been working on this in stealth mode

Ha! It was not my intention to operate in stealth mode. I had merely overlooked this question.
But yes, expect a lot more progress on this front really soon!

[jim22k]

Really looking forward to this!

[vak]

I’ve tried multiplication of the extremely sparse tensors both with PyTorch and TensorFlow on RTX4090 and they are much slower than dense.

So basically GPU is no help for sparse matrix multiplications?

[aartbik]

@vak

So basically GPU is no help for sparse matrix multiplications?

That seems a rather quick conclusion even though, indeed, accelerating general sparse computations on GPU can be very challenging. What sparse frameworks or addons did you use for PyTorch and Tensorflow experiments?