feat(ops): add voice-call escalations and OpsEscalationManager#7
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 068378f557
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| export async function runFull(): Promise<void> { | ||
| const context: TaskContext = {}; | ||
| const escalation = new OpsEscalationManager(new TwilioVoiceAdapter()); |
There was a problem hiding this comment.
Creating OpsEscalationManager with new TwilioVoiceAdapter() means every escalation is voice-only, so a voice-call failure (Twilio voice outage, invalid voice capability, or call rejection) drops the alert instead of falling back to SMS/WhatsApp even though PriorityNotificationRouter was added for that purpose. This makes critical pipeline failure notifications single-point-of-failure in production.
Useful? React with 👍 / 👎.
| import { getLogger } from "../../logger/index.js"; | ||
| import { type IMarketProvider, type MarketQuote, RateLimiter } from "./types.js"; | ||
|
|
||
| const config = getOpsConfig(); |
There was a problem hiding this comment.
Calling getOpsConfig() at module scope forces full env validation during import, so importing providers/orchestrator fails immediately when any required ops variable is missing, before runFull() reaches its config task, retry loop, or decision-required escalation path. This reintroduces eager startup side effects and makes the pipeline brittle in partially configured environments.
Useful? React with 👍 / 👎.
| await escalation.notifyDecisionRequired({ | ||
| stage: task.name, | ||
| summary: "Ops pipeline failed and requires operator decision before continuing.", | ||
| }); | ||
| throw error; |
There was a problem hiding this comment.
Inside the task failure handler, await escalation.notifyDecisionRequired(...) runs before rethrowing the original error; if notification delivery throws (for example, Twilio API timeout), this new error escapes and masks the real task failure. In incidents where both the task and notifier fail, operators lose the root-cause exception from the pipeline stage.
Useful? React with 👍 / 👎.
1 participant
Motivation
Description
TwilioVoiceAdapterthat posts TwiML to the Twilio Calls API with escaped speech content.OpsEscalationManagerto notify on progress threshold crossings and to send decision-required escalations; wired it into the orchestrationrunFull()pipeline so progress updates and terminal failures trigger escalations.voiceas aNotificationChanneland to prefervoice/sms/whatsapporder forCRITICALalerts, with aPriorityNotificationRouterimplementation.resetOpsConfigCache()insrc/config/schema.tsand removed eageropsConfigside-effects so adapters and logger resolve config lazily at send time.src/logger/index.tsthat readLOG_LEVELfromprocess.envwithout depending on ops config.CoinGecko,Binance,AlphaVantage), a simpleRateLimiter,SelfHealingSupervisorfor managed services, aStripeBillingServicehelper, and anscripts/ops-run.tsCLI runner to exercise the pipeline.src/notifications/ops-escalation.test.ts) and a small test tweak tosrc/commands/quantum.test.ts.Testing
pnpm lintand lint passed with the TwiML escaping fix applied.pnpm buildwhich completed successfully.pnpm vitest run src/notifications/ops-escalation.test.ts src/commands/quantum.test.tsand both test files passed (20tests total).Codex Task