Should MArray be backed by `Memory`?

[Tokazama]

Now that we have Memory, would it make more sense to use that than a mutable struct with a tuple? The current MArray is limited in it's ability to mutate element types that are not straightforward isbitstype and I think that both Memory and MArray only get special allocations if they fit within Julia's GC pool sizes.

[mateuszbaran]

StaticArrays.jl already has SizedArray which can be backed by Memory. A big use case of MArray is that it is stack-allocated when the variable doesn't escape the function. AFAIK Julia can't do that with Memory yet.

[Tokazama]

As far as I can tell from EscapeAnalyasis docs, Memory canbe escaped. The constant size and guarantee that there won't be a memory reallocation like Vector if resizing occurs goes a long way to making this information easier to track.

[mateuszbaran]

It seems to require Julia 1.12 at least, and is less robust than MArray:

julia> using StaticArrays, BenchmarkTools

julia> function f()
           A = @MArray [1 2; 3 4]
           return det(A * A)
       end

f (generic function with 1 method)

julia> @btime f()
  4.565 ns (0 allocations: 0 bytes)
4.0

julia> 

julia> function f()
           A = [1 2; 3 4]
           return det(A * A)
       end
f (generic function with 1 method)

julia> @btime f()
  273.718 ns (8 allocations: 416 bytes)
4.000000000000012

[vchuravy]

That comparison is a bit Apple and Oranges since it is still using an Array which has worse overheads than an MArray version with Memory could have.

But I am also not sure that the HeapToStack transform can handle memory as reliable as the tuples.

[mateuszbaran]

Yes, it's not an entirely honest comparison; I just wanted to find a case where an array is not escaping but there is an allocation. This is a closer comparison:

function f()
    A = SizedMatrix{2,2}([1 2; 3 4])
    return det(A * A)
end

julia> @btime f()
  13.486 ns (2 allocations: 112 bytes)
4.0

Feel free to tell me if there is any way to have f non-allocating using Memory.

[Tokazama]

Using Memory is better but still has issues...

function f()
    mem = Memory{Int}(undef, 4)
    mem[1] = 1
    mem[2] = 3
    mem[3] = 2
    mem[4] = 4
    A = SizedMatrix{2, 2}(mem)
    return det(A * A)
end

julia> @btime f()
  54.984 ns (4 allocations: 240 bytes)
4.0

Edit:
Looks like A * A generates Matrix. If you manage it by hand it gets a little better.

function f()
    A = SizedMatrix{2, 2}(Memory{Int}(undef, 4))
    A[1, 1] = 1
    A[1, 2] = 2
    A[2, 1] = 3
    A[2, 2] = 4
    B = SizedMatrix{2, 2}(Memory{Int}(undef, 4))
    # B = A * A
    B[1, 1] = (A[1, 1] * A[1, 1]) + (A[1, 2] * A[2, 1])
    B[1, 2] = (A[1, 1] * A[1, 2]) + (A[1, 2] * A[2, 2])
    B[2, 1] = (A[2, 1] * A[1, 1]) + (A[2, 2] * A[2, 1])
    B[2, 2] = (A[2, 1] * A[1, 2]) + (A[2, 2] * A[2, 2])
    # det(B)
    (B[1] * B[4]) - (B[3] * B[2])
end

julia> @btime f()
  33.723 ns (2 allocations: 128 bytes)

[vchuravy]

Note that in order for allocation ellision to happen for both MArray and MArray, you can't have bounds checking code.
The error capture will block the ellision.
The hope is that with SizedMatrix being statically known, the bounds check can be removed, but it warrants a closer look if that happens.