feat: fanout queue mode for broadcast delivery#26
Merged
Conversation
Fanout queues let every node process every job, unlike standard queues
where one node claims each job via FOR UPDATE SKIP LOCKED. Useful for
broadcast events like session revocation, notifications, and cross-node
chat in cloud-native deployments without distributed Erlang.
Each node polls with a time window (default 120s) and tracks seen job
IDs in a local ETS table for dedup. Workers must be idempotent since
jobs may be re-processed on node restart. Old jobs are pruned
automatically after 2x the window.
Config: {fanout_queues, [{<<"broadcast">>, 5, #{window => 120}}]}
🔴 Code Coverage — 49.9%640 of 1282 lines covered. ✅ ELP LintNo diagnostics. ℹ️ 11 OTP CVEs auto-ignored (already fixed in running version)These CVEs are patched in the installed OTP version but NVD data
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
shigoto_fanout_queuemodule — every node processes every job (vs single-consumer standard queues){fanout_queues, [{Name, Concurrency, #{window => Seconds}}]}Design
Standard queues use
FOR UPDATE SKIP LOCKED— one node claims each job. Fanout queues use a simpleSELECTwithinserted_at >= now() - window— all nodes read the same rows. Each node tracks seen IDs in a local ETS set to avoid re-processing within a session.On node restart, the ETS is empty so recent jobs within the window are re-processed. Workers must be idempotent. The source of truth is always the database — fanout is best-effort push.
Use case
Session revocation, notifications, chat delivery in cloud-native deployments where distributed Erlang is not available.
Test plan