Use sparse structured methods where possible

[jessegrabowski]

Description

For elemwise functions that are idempotent at 0 such that f(0) = 0, it is safe to always use a structured_ function, rather than converting back to dense.

Such functions are sinh, tanh, arcsin, arctan, arcsinh, deg2rad, rad2deg, expm1, and log1p.

I also removed the dense_override on .transpose, which now calls self.T. I think this is fine for now, since we're only supporting sparse matrices, not general sparse arrays. Previously, sparse.T returned a sparse, but sparse.transpose returned a dense, which is baffling.

copy seems like another low-hanging fruit, but I'm leaving it for another time because it was slightly harder to reason about than the functions in this PR.

Related Issue

• Closes #
• Related to Sparse TensorVariables are not using the Sparse Elemwise methods #272

Checklist

• Checked that the pre-commit linting/style checks pass
• Included tests that prove the fix is effective or that the new feature works
• Added necessary documentation (docstrings and/or example notebooks)
• If you are a pro: each commit corresponds to a relevant logical change

Type of change

• New feature / enhancement
• Bug fix
• Documentation
• Maintenance
• Other (please specify):

[ricardoV94]

Do you want to add the mT property as well? Will make code that works with tensors more likely to work with sparse variables as well

[ricardoV94]

Re: copy is identity function, we can make it return ViewOp()(self).

One thing with these Elemwise operations is that they box the sparse part like: sparse.data -> f(data) -> sparse_again. If we have multiple Elemwise in succession they won't get fused. We may need a rewrite (not a blocker for this PR ofc).

In general I'm not a big fan of this eager implementation. I would rather do it as a rewrite: elemwise(x.todense()) -> trick used here. Again not a blocker, specially because I haven't thought about all the ramifications

[jessegrabowski]

I would rather do it as a rewrite: elemwise(x.todense()) -> trick used here.

I don't like this. To me, the point of the structured ops is that you can keep everything sparse. I would write my program with dense Ops following elementwise(x.todense()), when I could/should be using sparse operations.

You could object and say we shouldn't be forcing people to use the sparse version of every op, but that's how things current work.

[ricardoV94]

I don't think we're talking about the same thing. The way elemwise sparse operations are implemented is not a good first representation. Much like scan, it leaks irrelevant implementation details in a way that muddles the meaning of the computation.

# this is roughly what the @structured decorator does
def log1p(x):
  data, indices, indptr = csm_properties(x)
  new_data = pt.log1p(data)
  return sparse(new_data, indices, indptr)

This is not a good first representation of sparse log1p. It's even worse for chains of operations. It's also backend specific, we would never try to write like that in the jax backend.

What I was saying is: If we need this form we can introduce it later in the backends that need it (if at all).