[umbrella][P1] telegram channel reliability (#245 follow-up)

[s2agi]

[umbrella][P1] telegram channel reliability (#245 follow-up)

Goal: kill three reproducible failure modes that Vincent hit today (telegram pairing & DM reliability across multi-node restarts).

Origin: Vincent /goal — "telegram 优化，我服了". Dispatched as the deep cut of #245 connectivity hardening, plus a regression caught today on TMCode负责人 where allowFrom got wiped by an upgrade/restart cycle.

Key insight that resets the design space (vs #245 task E's earlier verdict):

connectTelegram is a self-contained polling loop inside agent-node (agent-node/src/cli.ts:2289, started unconditionally at module top-level by line 2530:
for (const channel of TELEGRAM_CHANNELS) connectTelegram(channel)).
It is not a Claude-code-cli internals concern — Claude never touches the poller. Every failure mode below is fixable in our own codebase (agent-node + agent-network), zero upstream PR dependency. The earlier "Claude-internal channel lifecycle is hard" framing was about a different surface (the resume-time MCP channel reattach), not the bot poller.

---

P1 — allowFrom persistence regression (today's TMCode负责人 incident)

Smoking gun — agent-network/bin/cli.ts:1465-1481:

function writeTelegramChannelConfig(nodeId, botToken, allowId) {
  ...
  writeFileSync(join(channelDir, "access.json"), JSON.stringify({
    dmPolicy: "allowlist",
    allowFrom: [allowId],         // ← obliterates any pre-existing allowFrom
    groups: {},                   // ← obliterates any group assignments
    pending: {},                  // ← obliterates any pending pairings
  }, null, 2) + "\n");
}

Called from two sites that are part of routine UX flows:

• cli.ts:1868 — anet node create wizard (initial setup)
• cli.ts:4486 — anet channel add telegram <node> ... (CLI re-add, frequently re-run when troubleshooting)

Either path destroys user data on re-run. Any user who has been gradually built up onto allowFrom over weeks is reset to a single id (or empty list) the next time the operator re-runs the wizard or channel add.

Hard constraint to land first (P1, blocker for the rest):

• writeTelegramChannelConfig MUST read the existing access.json if it exists and merge:
  • keep prior dmPolicy unless explicitly overridden
  • allowFrom: union with the new id, never replace
  • keep prior groups and pending
• --bot-token only writes .env when explicitly passed
• Regression test in CI: seed a fake access.json with 3 ids in allowFrom + 2 entries in pending + 1 entry in groups, run anet channel add telegram <node> --bot-token X --allow Y, assert the merged file still contains the original 3 + new Y in allowFrom, original pending and groups intact.

---

Mode 1 — restart 后 poller 不起 / 通道没 re-attach

Three independent root causes; need targeted fixes (not one mega-rewrite).

1a. anet channel add after anet node start — poller never picks up new channel

• agent-node/src/cli.ts:446 evaluates TELEGRAM_CHANNELS at module top-level. Adding a channel to config.json while agent-node is running has zero effect; anet node resume doesn't help (it wakes Claude, not agent-node).
• Fix A2 (low, deferred): agent-node watches config.json mtime → re-init TELEGRAM_CHANNELS and call connectTelegram(newChannel) for any new entry. Edge cases (token rotation, removal) make this fiddly; deferred behind A1.

1b. Bot token invalid → process.exit(1) (cli.ts:2303) kills the whole agent-node

• agent-node dies → tmux respawns it → same bad token → dies again → restart-storm.
• Fix A3 (low): replace process.exit(1) with commhub_report_status({ status: "warn", note: "telegram bot token invalid (<channel-dir>)" }) + return from connectTelegram. agent-node stays alive, other channels keep working, the failing channel surfaces in anet doctor.

1c. Poller silently dead inside the running process

• The outer while (true) catches getUpdates network errors (cli.ts:2357-2359), but a thrown exception from inside the queue drain (e.g. telegramSend failure causing unhandled promise rejection at line 2354) can leave processing = false with no caller — the loop continues but the queue never drains.
• Fix A1 (low, primary): agent-node maintains <channel-dir>/health.json with {lastPoll, offset, msgsSinceBoot, lastError}. A watchdog timer in agent-node checks every 30s: if now - lastPoll > 90s → log + re-invoke connectTelegram(channel) + bump a restart counter. Counter exposed in anet channel status.

Sub-task scope estimate: ~80 LOC across A1 + A3, mostly in agent-node. A2 deferred — watchdog covers the symptom, hot-reload is a nice-to-have.

---

Mode 2 — 配对 UX (pairing + allowlist)

access.json was designed with dmPolicy / pending / groups (agent-network/bin/cli.ts:1474-1479), but agent-node/src/cli.ts:2204-2208 only ever reads allowFrom. dmPolicy and pending are written once and never read — a half-implemented pairing UX. Result:

• allowFrom.length === 0 → wide open (everyone accepted).
• allowFrom.length > 0 → strict allowlist; non-allowed senders are silently dropped. The bot gives no signal, the operator never sees the attempt — exactly Vincent's "够不着" complaint.

B1 — implement pending pairing (medium, primary)

Reject path in telegramAllowed:

• Write pending[<userId>] = { firstSeen, lastMessage, label, attempts } to access.json.
• Reply to the user: "等待管理员批准" + a copy of their own user id so they can show the operator.
• Log to commhub with status waiting_pairing.
• Throttle: at most one 等待批准 message per <userId> per 24 h.

B3 — anet channel add wizard prompts the operator to pair (medium)

At creation:

1. operator gives bot token, no --allow
2. wizard prints: "打开 Telegram → 给 @<bot-username> 发任意消息（3 分钟内）→ 我把你的 id 写入 allowFrom"
3. wizard polls getUpdates until first message arrives → writes that from.id to allowFrom
4. timeout returns to manual --allow flow

B4 — anet doctor surfaces pending pairings prominently (low)

doctor already enumerates channels per #245 task E. Add: for each channel, if Object.keys(access.pending).length > 0, print a yellow banner:

⚠️ 节点 TMCode负责人 有 2 个待批配对：
   alice (id=12345, last seen 14:32) — 运行 `anet channel approve TMCode负责人 alice`
   bob   (id=67890, last seen 14:35) — 运行 `anet channel approve TMCode负责人 bob`

plus a new sub-command anet channel approve <node> <user> that moves the id from pending to allowFrom.

B2 — commhub_telegram_approve MCP tool (deferred)

Same intent as B4's CLI sub-command, but exposed as MCP so the operator's agent can approve without leaving its turn. Defer unless B4 proves too clicky.

---

Mode 3 — 死活可见 (poller liveness)

anet doctor + anet channel status (added in #245 task E) currently show configuration (access.json contents, dmPolicy, allowFrom). They do not show whether the poller is currently polling.

C1 — write <channel-dir>/health.json from agent-node (low)

Shared file with Mode 1 A1's watchdog. Fields: lastPoll (ISO timestamp), offset, msgsSinceBoot, lastError (object or null), restartCount (watchdog interventions).

C2 — anet channel status shows freshness (low)

Decorate the existing anet channel status output:

• Last poll: 12s ago (green ✅ if < 60 s)
• Last poll: 23 min ago (yellow ⚠️ if 60 s – 10 min)
• Last poll: 4 h ago (red 🔴 if > 10 min) — → poller likely dead, run anet node stop && anet node start
• • Watchdog restarts since boot: <n>
• • Last error: <message> if non-null

C3 — anet doctor integrates health.json (low)

For each enumerated channel, read health.json. If stale → red entry + fix suggestion. If lastError non-null → surface it.

Sub-task scope estimate: ~50 LOC across C1 + C2 + C3, shares health.json with Mode 1 A1.

---

9 sub-tasks (proposed)

1. P1-merge — writeTelegramChannelConfig reads-merge-writes (allowFrom union + preserve pending/groups/dmPolicy) + CI regression test
2. P1-B1 — implement pending pairing in telegramAllowed
3. P1-B4 — anet doctor pending banner + anet channel approve <node> <user>
4. P2-A1 — health.json watchdog (agent-node)
5. P2-C1+C2+C3 — health.json read paths (anet doctor / anet channel status)
6. P3-A3 — friendly bot-token-invalid path (no process.exit(1))
7. P3-B3 — anet channel add interactive pair wizard
8. P4-A2 — config.json hot-reload (optional, may be cut)
9. integration — end-to-end test: spin up Docker bot fixture, exercise pair → allowlist → reject → pending → approve → allowed loop

Priority ordering (per Vincent's pain map)

• P1: P1-merge + P1-B1 + P1-B4 — resolves today's allowFrom wipe + the "silently 够不着" complaint
• P2: P2-A1 + P2-C1+C2+C3 — resolves "看不出死活" + auto-recovery for in-process poller deaths
• P3: P3-A3 + P3-B3 — UX polish for invalid tokens + onboarding
• P4: P4-A2 — drop if not free

Constraints / non-goals

• All work lives in agent-node + agent-network. No Claude / upstream dependency.
• No breaking changes to access.json schema — existing nodes upgrade in place. The merge fix in P1-merge is itself a non-breaking read-then-write.
• No new MCP server. (B2 deferred behind B4's CLI.)
• Don't touch ~/.commhub or any prod hub state during dev/test. Use Docker fixtures.

Owner + ETA

• Owner: 通信工程马 (lead), may delegate one P2/P3 sub-task to 通信SDK马 depending on bandwidth.
• ETA: P1 (3 sub-tasks) — 1 evening, ~5-8 h. Full P2 — +1 evening. P3 — opportunistic. P4 — optional. No overnight rush.

---

Author-Agent: 通信工程马

[s2agi]

🆕 P0 sub-task (added 2026-06-19) — anet node create should auto-protect .anet/ from git clean

Root cause found today (TMCode负责人 repeated wipe incident):

• 真正的 wiper = git clean -fd run from inside a tmcode build/release script
• Asymmetric signature confirmed: .anet/node_modules/ (gitignored by default Node convention) survived; .anet/nodes/ (untracked, NOT in .gitignore) was nuked
• This is the precise fingerprint of git clean -fd on a project where .anet/ was never added to .gitignore
• Plugin's defaultAccess() shape (dmPolicy:"pairing", allowFrom:[], pending:{}) showed up post-wipe because plugin's readAccessFile ENOENT path returns default → next Vincent DM writes default + new pending to disk → looks like "access.json got wiped" when actually the entire node dir was rebuilt from ENOENT

Sub-task (new P0, more universal than P1-merge):

• anet node create detects whether cwd is a git repo
• If yes AND .anet/ not already in .gitignore → either:
  • (default) auto-append .anet/ to project's .gitignore + log [anet] note: added '.anet/' to .gitignore so git clean won't nuke your node state
  • (opt-out) --no-gitignore-anet flag for the rare user who really wants .anet/ tracked
• Optional polish: anet doctor checks all existing nodes in cwd-rooted git repos and warns if .anet/ is untracked + un-gitignored
• Optional polish: anet node create also emits a top-level README.anet.md or comment in .gitignore explaining why
• Regression test: in a fresh git repo, run anet node create X, assert .gitignore contains .anet/, then run git clean -fd and assert X still exists.

Why this beats P1-merge as the primary fix:

• P1-merge (writeTelegramChannelConfig no-clobber) prevents one specific wipe path (CLI re-add)
• gitignore auto-protect prevents the ENTIRE class of git clean -fd wipes from any source (build scripts, dev cleanup, CI), independent of what triggered the clean
• Vincent's TMCode负责人 incident proves the gitignore path is the practical attack vector, not the CLI re-add path

Priority: This sub-task lands BEFORE P1-merge in the implementation order. P1-merge stays in scope as an independent correctness fix (it would still bite future users who re-run channel add).

Owner: 通信工程马
ETA: ~30 min (small CLI patch + test)

— Author-Agent: 通信工程马

[s2agi]

🔬 Live-fire data point (2026-06-20) — channel-attach-but-poller-not-spawned

TM负责人 in /home/vansin/tmteam exhibited the exact P2-watchdog symptom we hypothesized: Claude shows the channels banner (plugin:telegram@claude-plugins-official, server:commhub inject directly in this session) but no bun server.ts poller process exists for the channel.

Symptom

• anet node start TM负责人 → node alive, Claude responsive, channels banner shown
• bot.pid absent in channels/telegram/
• No bun server.ts process has TELEGRAM_BOT_TOKEN matching the node's .env
• access.json + .env + token are all correct
• → Vincent's bot DMs fail silently (poller can't ack what isn't running)

Smoking-gun differentiator

The failing start was a resume (claude --session-id <uuid> in the cmdline). After clearing the session field from config.json (jq 'del(.session)') and starting fresh, the poller spawned on the same Claude boot.

Hypothesis: Claude's plugin lifecycle (--channels plugin:telegram@...) re-spawns pollers on fresh session start but skips that step on resume. Result: resumed sessions show the channels banner (configured) but have no live poller (process).

Implications for #246 sub-tasks

P2 watchdog scope — sharpened:

• C1 (health.json from poller) assumes the poller is alive enough to write it. If poller never spawned, C1 has no signal. Need a complementary check on the CLI side:
  • anet channel status <node> should additionally read channels/<type>/bot.pid if it exists, then verify the pid is actually alive AND the process is bun server.ts AND its env has the matching TELEGRAM_BOT_TOKEN. If any of those fail (or bot.pid is missing entirely while the channel string is in config.json.channels) → red flag "channel attached but poller not running — try anet node stop && anet node start --new-session".
  • anet doctor integrates this check across all nodes.

A1 watchdog (agent-node side) is NOT the primary defender here — the plugin spawns its own poller in a separate process tree; agent-node never owns this poller. agent-node's connectTelegram (the original audit target) is a parallel/legacy implementation not used by TMCode/tmteam nodes.

New low-hanging fix idea (P2-add): when anet node start detects config.json.channels contains a plugin:* or telegram entry but no bot.pid appears under channels/<type>/ within ~10s of Claude boot → warn the operator with the same stop && start --new-session advice.

Other collateral findings worth tracking (small papercuts)

1. anet node start in non-TTY (nohup) silently degrades to claude --print (one-shot mode), which immediately errors Input must be provided either through stdin or as a prompt argument when using --print and exits. The CLI should refuse non-TTY for long-lived starts with a friendly error pointing at tmux new -d -s <name> instead.

2. anet node stop <name> kills the tmux session along with the process. Fresh-start then requires rebuilding the tmux session; otherwise anet node start (from nohup) falls into (1). Document this in the connectivity-runbook + consider anet node stop --keep-tmux flag.

3. The --new-session flag still suffers from (1) — it just creates a new session id but goes through the same --print codepath when stdin isn't a TTY.

4. anet doctor / anet channel status should explicitly note when cwd is not a git repo (git rev-parse --show-toplevel fails), so the gitignore auto-protect we plan to add doesn't silently no-op.

— Author-Agent: 通信工程马

[s2agi]

🆕 sub-task — detect/warn when multiple nodes share the same telegram bot token

Live-fire false-positive caught today (2026-06-20):

TM负责人 (/home/vansin/tmteam) and TMCode负责人 (/home/vansin/tmcode) were both pollering the same bot token (839207..., bot username @vincent_macmini_bot, display name "TMCode负责人"). Two pollers → both call getUpdates → updates land on whichever poller wins the race. Vincent was actually DMing a different bot (display name "TM负责人") that had NO node polling it — so his messages were silently dropped (no poller, no inbox writes, no Claude wake).

Compounding the trap: I had Claude (in TM负责人 node) send a confirmation via plugin:telegram — which went to @vincent_macmini_bot's outbound chat thread. The message arrived in getUpdates for the other bot's recipient, but since Vincent was talking to a different bot, he never saw it. Looked like ✅ bidirectional verified, was actually a false positive.

Sub-task — bot-token conflict detection (P1)

anet channel add and anet node create should:

1. Call Telegram's getMe on the supplied bot token at config time → get username + id.
2. Scan all sibling .anet/nodes/*/channels/telegram/.env for TELEGRAM_BOT_TOKEN. Hash or compare tokens — if any other node already uses the same token, refuse and warn:

   ❌ bot @vincent_macmini_bot (id 12345) is already wired to node TMCode负责人.
   Telegram bots can only be polled by one process at a time. Re-use one of:
     - point TM负责人 at a different bot
     - delete the channel from TMCode负责人 first
   See: <docs URL>
   

3. anet doctor / anet channel status should walk all nodes' .env tokens and call getMe on each → flag any token shared by >1 node (red) and any token whose bot's username doesn't match the node's display name (yellow — likely the operator is talking to a different bot than they think they configured).

Bonus

anet channel status <node> should emit the bot's username + display name from getMe (not just the token prefix) so the operator immediately sees "this node polls @vincent_macmini_bot (TMCode负责人)" and notices the mismatch with their intent.

Priority: P1 (alongside merge fix + pending pairing) — silent multi-node bot conflicts are an even worse footgun than allowFrom wipe because the wipe at least produces a visible empty file, whereas the conflict has zero local symptoms until Vincent reports "I'm not getting replies."

— Author-Agent: 通信工程马