runtime-next: add minimum interval for post-txn triggers

[dgreer-dev]

Description:

Adds an optional minimum fire interval (debounce) to materialization webhook triggers, for the V2 runtime. Fixes #3060.

Triggers previously fired once per committed transaction that materialized data, so a burst of transactions produced a burst of webhook deliveries (potentially costly). With an interval set, a burst collapses into at most one delivery per interval, whose payload covers the union of the collapsed transactions. The old connector-side dbt trigger had an equivalent debounce, this generalizes it to runtime triggers.

Mechanically: each trigger config accumulates a pending window in a leader-lifetime accumulator, keyed by the config's method+URL. At commit time the accumulator is persisted and configs whose interval has elapsed fire with their accumulated window. The rest keep accumulating and fire either at the first commit at-or-after their interval elapses (fast path) or, if the task goes quiet, from the leader's idle loop once the deadline passes (so a debounced window never hangs waiting for a next transaction).

Workflow steps:

Add interval to a trigger config on a materialization:

triggers:
  config:
    - url: "https://cloud.getdbt.com/api/v2/accounts/123/jobs/456/run/"
      headers:
        Authorization: "Token dbt-token"
      payloadTemplate: |
        {"cause": "Estuary trigger {{run_id}}"}
      interval: 30m

Unset preserves today's fire-every-transaction behavior; existing specs are unchanged. The field is per-config, so e.g. a Slack notification can fire every transaction while a dbt run on the same materialization is debounced to 30m.

Documentation links affected:

• site/docs/concepts/materialization/materialization-triggers.md — updated in this PR: interval added to the spec example, properties table, and SOPS-excluded-fields list.

Notes for reviewers:

• Two fire paths, one decision function. A due window normally fires with the transaction that made it due (commit path, no added latency). If the task goes quiet — frontiers only arrive when journal content advances — HeadIdle fires due windows from idle and otherwise sleeps until the earliest pending deadline, so a burst's tail is delivered within its interval even with no further commits. The idle fire requires Tail::Done and rotates the Tail into its normal Trigger → Persist → Done sequence, so it can't race a commit-path fire.
• Persisted format change + compatibility. The durable "trigger-params" blob changes from a single TriggerVariables to a {method+url → TriggerVariables} map. Both runtimes decode through the new serde-untagged models::triggers::PersistedTriggerParams (the formats are mutually unambiguous), so: V2 recovering a pre-upgrade blob fans it out to all configs (its original fire-all semantics), and V1 after a V2→V1 rollback merges a map blob into one window (its normal fire-all behavior). Residual: a binary downgrade to a release before this PR has neither fallback — don't downgrade past this release while a trigger is mid-debounce.
• Debounce state is in-memory (last_fire): a leader failover resets the window, worst case one early fire — same as pre-debounce behavior. Recovery re-fires the persisted accumulator verbatim (at-least-once), which can also mean one early fire post-recovery.
• Duplicate configs (same method+URL) aren't supported since the key is the debounce identity: first config wins, later duplicates WARN at task startup and never fire. A pending window whose config was removed on republish is dropped with a WARN rather than failing the task (failing would crash-loop, since the config stays gone for the build's life).
• V1 is otherwise untouched: it accepts but ignores interval (no debounce), and its only code change is the tolerant PersistedTriggerParams decode above.

[github-actions]

🚀 Preview deployed to https://docs.estuary.dev/pr-preview/pr-3110/

📄 Changed pages:

• /concepts/materialization-triggers/