Skip to content

fix: auto-route Ollama Cloud models to ollama.com when API key is set#381

Merged
0xLeif merged 2 commits intomainfrom
fix/ollama-cloud-routing
May 8, 2026
Merged

fix: auto-route Ollama Cloud models to ollama.com when API key is set#381
0xLeif merged 2 commits intomainfrom
fix/ollama-cloud-routing

Conversation

@corvid-agent
Copy link
Copy Markdown
Collaborator

Summary

  • Cloud models (e.g. qwen3-coder:480b-cloud) now auto-route to https://ollama.com when OLLAMA_API_KEY is configured, instead of going through localhost:11434 where the local daemon can't forward auth
  • fledge ai status now shows API key source (env/config/not set) and whether cloud routing is active
  • fledge ai use prompts for API key when selecting a cloud model without one configured
  • fledge config list now shows env var override indicators for OLLAMA_HOST and OLLAMA_API_KEY
  • Empty API keys are filtered as None — prevents sending Authorization: Bearer with no token

Host resolution precedence

  1. OLLAMA_HOST env var (explicit user override — always wins)
  2. Non-default config host (user explicitly configured a custom endpoint)
  3. Cloud host (https://ollama.com) when API key present AND model is cloud
  4. Default localhost (http://localhost:11434)

Test plan

  • 844 unit tests pass
  • is_cloud_model correctly matches -cloud variants
  • resolve_effective_host routes cloud models to ollama.com with key
  • Cloud model stays local when no key is present
  • Explicit OLLAMA_HOST env var overrides cloud auto-routing
  • Custom config host overrides cloud auto-routing
  • Non-cloud models stay local even with API key set
  • build_provider errors early when cloud model lacks API key
  • Empty API key string treated as missing

Closes #380

🤖 Generated with Claude Code

Cloud models (e.g. qwen3-coder:480b-cloud) sent through localhost fail
with 401 because the local daemon doesn't forward client Bearer tokens.
When OLLAMA_API_KEY is configured and the model is a cloud model, route
directly to https://ollama.com instead of localhost:11434.

Changes:
- llm: add resolve_effective_host() with cloud auto-routing
- llm: add is_cloud_model() detector, bail early when cloud model
  lacks API key, filter empty API keys as None
- ai: show API key source and cloud routing in `fledge ai status`
- ai: prompt for API key when `fledge ai use` selects a cloud model
- config_cmds: show env var override indicators in `fledge config list`

Closes #380

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@corvid-agent corvid-agent requested a review from a team as a code owner May 8, 2026 02:53
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Corvin says...

      _
    <(;\  .oO(oh no...)
     |/(\
      \(\\
      " "\\

"Even the dumpster of code seems empty today."

CI Summary

Check Status
Dependency Audit ✅ Passed
Integration (3 OS) ❌ failure
Lint (fmt + clippy) ❌ failure
Spec Validation ❌ failure
Tests (3 OS) ✅ Passed

Powered by corvid-pet

@corvid-agent
Copy link
Copy Markdown
Collaborator Author

👋 CorvidAgent is looking into this.

- rustfmt: reformat function signatures
- clippy: replace useless format!() with .to_string()
- spec: document 3 new exports (resolve_effective_host, is_cloud_model,
  DEFAULT_OLLAMA_CLOUD_HOST), add cloud routing invariants and error case

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@corvid-agent
Copy link
Copy Markdown
Collaborator Author

All three CI failures addressed in 3c89cdd:

  • Lint (fmt): Ran cargo fmt — reformatted function signatures in ai.rs, config_cmds.rs, llm.rs
  • Lint (clippy): Replaced format!(" (auto-routed to cloud)") with " (auto-routed to cloud)".to_string() in ai.rs:175
  • Spec Validation: Documented 3 new exports (DEFAULT_OLLAMA_CLOUD_HOST, is_cloud_model, resolve_effective_host) in specs/llm/llm.spec.md — strict mode now passes with 0 warnings

Verified locally: cargo fmt --check ✅, cargo clippy -D warnings ✅, specsync check --force --strict 0 warnings ✅

@corvid-agent
Copy link
Copy Markdown
Collaborator Author

👋 CorvidAgent is looking into this.

@github-actions github-actions Bot dismissed their stale review May 8, 2026 03:10

Superseded by updated review.

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Corvin says...

      _
    <(^\  .oO(Caw! ^v^)
     |/(\
      \(\\
      " "\\

"Caw! Your code sparkles like a dropped french fry."

CI Summary

Check Status
Dependency Audit ✅ Passed
Integration (3 OS) ✅ Passed
Lint (fmt + clippy) ✅ Passed
Spec Validation ✅ Passed
Tests (3 OS) ✅ Passed

Powered by corvid-pet

@corvid-agent
Copy link
Copy Markdown
Collaborator Author

CI failures resolved in 3c89cdd:

  • Lint (fmt): Reformatted multi-line function signatures to match rustfmt style
  • Lint (clippy): Replaced format!(" (auto-routed to cloud)") with " (auto-routed to cloud)".to_string() (clippy::useless_format)
  • Spec Validation: Documented 3 new exports (resolve_effective_host, is_cloud_model, DEFAULT_OLLAMA_CLOUD_HOST) and added cloud routing invariants + error case to the LLM spec

All checks now passing — spec-check, lint, tests (3 OS), audit, and integration (3 OS). ✅

@corvid-agent
Copy link
Copy Markdown
Collaborator Author

👋 CorvidAgent is looking into this.

2 similar comments
@corvid-agent
Copy link
Copy Markdown
Collaborator Author

👋 CorvidAgent is looking into this.

@corvid-agent
Copy link
Copy Markdown
Collaborator Author

👋 CorvidAgent is looking into this.

@0xLeif 0xLeif merged commit 77b2ca0 into main May 8, 2026
12 checks passed
@0xLeif 0xLeif deleted the fix/ollama-cloud-routing branch May 8, 2026 03:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ollama Cloud API key not passed through in fledge config/ai/ask

2 participants