feat: SpikeAsync

[Kelvin-Ng]

Overview

See ASYNC_API.md for a detailed description of the API and design.

The current implementation is tested and ready to merge.

Known Limitations: Head-of-Line Blocking

The scheduler can exhibit head-of-line (HoL) blocking, which may be surprising to users.

In practice, I've worked around this by manually adding extra dependencies between operations to constrain the scheduler's decision space. However, this makes real-world usage of SpikeAsync more involved than the simple model described in ASYNC_API.md.

Why It Happens

The scheduler performs a topological sort at scheduling time. When only logical dependencies are specified, multiple valid topological orderings exist — and some of them are suboptimal. Different scheduling algorithms will select different orderings among the valid ones.

The current algorithm uses FIFO ordering. Consider the following example:

async def foo():
    await use_resource_a()  # X
    await use_resource_a()  # Y
    await use_resource_b()  # Z
    await use_resource_b()  # W

Running 3 instances concurrently:

for i in range(3):
    spike_async.submit(foo())

When use_resource_*() is called, the request is immediately pushed to the corresponding resource's FIFO queue. Because all 3 instances of X are enqueued before any instance of Y, the resulting schedule is:

A: [X0][X1][X2][Y0][Y1][Y2]
B:                 [Z0][Z1][Z2][W0][W1][W2]

Resource B sits idle while resource A drains the backlog — a classic HoL blocking scenario. The ideal interleaving would be:

A: [X0][Y0][X1][Y1][X2][Y2]
B:         [Z0][W0][Z1][W1][Z2][W2]

Workaround

We can achieve optimal interleaving without modifying the scheduler by adding cross-instance dependency edges:

1. X(i+1) → Y(i) — prevents X's from piling up on resource A
2. Z(i+1) → W(i) — prevents Z's from piling up on resource B

prev_y = None
prev_w = None

for i in range(3):
    x = spike_async.use_resource_a(deps=[prev_y] if prev_y else [])

    y = spike_async.use_resource_a(deps=[x])

    z = spike_async.use_resource_b(deps=[y] + ([prev_w] if prev_w else []))

    w = spike_async.use_resource_b(deps=[z])

    prev_y = y
    prev_w = w

w.wait()

Future Work

I'm considering alternative scheduling algorithms, but there is no one-size-fits-all solution — different algorithms perform best under different workload patterns. I believe this warrants further discussion and is best treated as future work.