Skip to content

Comments

feat: isolate async handler execution on dedicated worker event loop#273

Open
siwachabhi wants to merge 1 commit intomainfrom
feat/worker-loop-async-isolation
Open

feat: isolate async handler execution on dedicated worker event loop#273
siwachabhi wants to merge 1 commit intomainfrom
feat/worker-loop-async-isolation

Conversation

@siwachabhi
Copy link
Contributor

Summary

  • Introduces a dedicated persistent worker event loop in a background daemon thread to isolate all async handler execution from the main uvicorn event loop
  • Async handlers containing blocking calls (e.g. time.sleep, synchronous HTTP) previously froze /ping health checks, causing container termination — this fix ensures /ping always responds promptly
  • Implements three-way handler dispatch: async generators bridged via queue.Queue, regular async via run_coroutine_threadsafe, sync via run_in_threadpool
  • Propagates contextvars across event loop boundaries using copy_context() + Django asgiref _restore_context pattern (Python 3.10+ compatible)

Test plan

  • 18 new tests in TestWorkerLoopInvocation covering:
    • Async handler runs on worker thread (not main)
    • Sync handler runs in thread pool (worker loop NOT created)
    • Blocking async handler does NOT block /ping
    • Context propagation to async and sync handlers
    • Async generator bridging and streaming
    • Fire-and-forget create_task survives handler return
    • Exception propagation from worker loop
    • Lazy initialization and loop reuse
    • Concurrent async invocations
    • functools.partial and callable class dispatch
    • HEALTHY_BUSY ping status during background tasks
    • End-to-end HTTP tests for sync streaming, async, and async gen streaming
  • Updated existing test_async_handler_runs_on_worker_loop to assert worker thread isolation
  • All 105 runtime tests pass
  • Full test suite: 950 passed, 0 failures
  • Pre-commit hooks (ruff lint, ruff format, trailing whitespace) all pass

Async handlers that contain blocking calls (e.g. time.sleep, synchronous
HTTP requests) previously ran on the main uvicorn event loop, freezing
/ping health checks and causing container termination. This introduces a
dedicated persistent worker event loop in a background thread that
isolates all handler execution from the main loop.

Three-way handler dispatch:
- Async generators: bridged to sync generators via queue.Queue on worker loop
- Regular async: run_coroutine_threadsafe + wrap_future on worker loop
- Sync: run_in_threadpool (starlette thread pool)

Context propagation uses contextvars.copy_context() with the Django
asgiref _restore_context pattern for Python 3.10+ compatibility.
@siwachabhi siwachabhi requested a review from a team February 21, 2026 10:04
@siwachabhi siwachabhi deployed to auto-approve February 21, 2026 10:04 — with GitHub Actions Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant