fix: Anthropic proxy timeout — native SSE streaming + hardened HTTP client

[worldofgeese]

Problem

When using the Anthropic API through a corporate proxy (e.g. https://$URL/claude/v1/messages), completions from Opus consistently fail:

1. Client-side timeout (before this PR): CompletionError: ProviderError: error sending request for url — the 120s base reqwest client timeout expired before Opus finished thinking.
2. Proxy-side timeout (after bumping client timeout): 504 Gateway Timeout — the proxy killed the idle TCP connection because the non-streaming Anthropic path sends a single POST and waits for the entire response as one blob, with no intermediate data to keep the connection alive.

Root Cause

Three issues compound:

1. No Anthropic SSE streaming. The stream() method for ApiType::Anthropic calls attempt_completion() (non-streaming) then wraps the result in a fake stream via stream_from_completion_response(). Corporate proxies with gateway timeouts (typically 120–300s) kill the idle connection before Opus finishes generating.

2. Base HTTP client timeout too low. Both reqwest::Client::builder() sites use a 120s flat timeout with no connect_timeout or tcp_keepalive.

3. Error chain discarded. Every .map_err(|e| CompletionError::ProviderError(e.to_string())) only captures the top-level reqwest error. The actual cause (timeout, connection reset, proxy disconnect) is silently lost.

The Fix

Commit 1: Hardened HTTP client + error diagnostics

src/llm/manager.rs

• Bump base timeout: 120s → 300s
• Add connect_timeout(30s) — fail fast on connection establishment
• Add tcp_keepalive(30s) — keeps proxy connections alive
• Add pool_idle_timeout(90s) — prevents stale pooled connections

src/llm/model.rs

• Add .timeout(STREAM_REQUEST_TIMEOUT_SECS) to call_anthropic() request builder (1800s, matching OpenAI)
• Replace e.to_string() with format!("{e:#}") for full error cause chain preservation

Commit 2: Native Anthropic SSE streaming (~200 lines)

Adds stream_anthropic() that:

• Sends "stream": true in the request body
• Parses all Anthropic SSE event types: message_start, content_block_start/delta/stop, message_delta, message_stop, error, ping
• Handles text, tool_use (incremental JSON accumulation), and thinking blocks
• Tracks usage (input/cached/output tokens)
• Handles OAuth tool name reverse-mapping

Supporting changes:

• anthropic/params.rs: Added body field to AnthropicRequest and anthropic_messages_url() helper
• anthropic.rs: Re-exported new symbols

Commit 3: Compilation fixes + code review improvements

• Fixed ReasoningContent::Text variant (struct syntax with signature field)
• Fixed message_id to preserve Option<String> type
• Fixed StreamingCompletionResponse constructor call
• Added tracing::warn! for malformed tool JSON (was silently replaced with {})

Testing

• cargo check --release: ✅ clean
• cargo test: 623 passed, 2 failed (pre-existing on v0.3.3 — config::tests::test_llm_provider_tables_parse_with_env_and_lowercase_keys, config::tests::toml_round_trip_with_named_instances)
• Based on release tag v0.3.3

Impact

• Anthropic path: Real SSE streaming — continuous data keeps proxy connections alive
• All providers: TCP keepalive + full error chains
• No config schema changes, no new dependencies, no behavioral changes for non-proxy users

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ce47c112-6418-45d8-9fd4-de47a4bcda16

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}