feat(dev): self-hosted LiteLLM proxy compose service

[BjoernSchotte]

Summary

README.md and CONTRIBUTING.md treat "a LiteLLM gateway URL" as a prerequisite without documenting where it comes from. New contributors trying the docker-compose flow either reach for a hosted Berri instance (account + key + cost) or skip the platform entirely because no model calls work without a gateway.

This PR adds an optional compose service that runs the upstream ghcr.io/berriai/litellm:main-stable proxy as part of the same docker-compose project. Operators opt in by adding -f docker-compose.litellm.yml to the up command:

docker compose -f docker-compose.yml -f docker-compose.litellm.yml up

What's included

• docker-compose.litellm.yml — service definition. Listens on :4000, has a Python-based healthcheck against /health/liveliness, reads ANTHROPIC_API_KEY / OPENAI_API_KEY from .env, requires ANTHROPIC_API_KEY to be set (uses ${VAR:?...} so misconfig fails at compose-parse time, not at request time).
• litellm-config.yaml.example — minimal config with two Anthropic models wired up and OpenAI entries commented out behind OPENAI_API_KEY. Master key matches the platform's LITELLM_API_KEY example. The actual litellm-config.yaml is in .gitignore (via .env* pattern? — check during review; happy to add an explicit litellm-config.yaml entry if not).
• README.md — new "Self-hosted LiteLLM gateway (optional)" subsection under Self-hosting that walks the copy-config / set-key / compose-up flow.

What is unchanged

• The base docker-compose.yml is untouched. Operators who already have a hosted LiteLLM proxy can keep ignoring the new file.
• No new required env vars. ANTHROPIC_API_KEY only becomes required if you opt into the new compose file.
• The platform's existing LITELLM_API_BASE / LITELLM_API_KEY contract is unchanged.

Why not just docs?

The flow is small enough that pointing at LiteLLM's own setup docs would work, but every step (which image, which env vars, which port, which master_key to align with the platform) is something a contributor has to figure out before their first session call works. Shipping the compose file collapses 30 minutes of trial-and-error into one optional command.

Test plan

• docker compose -f docker-compose.yml -f docker-compose.litellm.yml up — both services healthy
• Sandbox session against anthropic/claude-haiku-4-5 round-trips a message
• Misconfig (ANTHROPIC_API_KEY unset) fails at compose-parse with a clear message, not at first request

[greptile-apps]

Greptile Summary

This PR adds an optional docker-compose.litellm.yml overlay and a litellm-config.yaml.example so contributors can run a self-hosted LiteLLM proxy alongside the platform without a hosted gateway, and updates README.md with the corresponding setup steps.

• docker-compose.litellm.yml brings up ghcr.io/berriai/litellm:main-stable on port 4000 with a Python-based healthcheck and uses ${VAR:?...} to fail fast on a missing ANTHROPIC_API_KEY; the error message is in German and the master key is hardcoded rather than env-var-overridable.
• litellm-config.yaml.example wires up two Anthropic models via env-var key references; litellm-config.yaml (the user's working copy) is not covered by the existing .gitignore, a gap the author flagged but did not close.
• The README instructs users to set LITELLM_API_BASE=http://host.docker.internal:4000, but since both compose files share one project network the intra-compose service name http://litellm:4000 is available and more direct.

Confidence Score: 3/5

Safe to merge after the gitignore gap is closed; the other findings are cosmetic or low-risk defaults.

The missing .gitignore entry for litellm-config.yaml is the one concrete gap — the author identified it in the PR description but did not add the line. A contributor who copies the example, customises the master key, and runs git add . will commit the file without any warning. Everything else (German message, hardcoded default key, host.docker.internal vs service name) is style or a minor ergonomics issue that does not affect correctness.

docker-compose.litellm.yml for the hardcoded master key and German error string; .gitignore for the missing litellm-config.yaml exclusion.

Important Files Changed

Filename	Overview
docker-compose.litellm.yml	New optional compose service for the LiteLLM proxy; contains a German-language error message and a hardcoded master key that should be an env-var default instead.
litellm-config.yaml.example	Well-structured example config; uses env-var references for provider keys correctly, but the hardcoded master_key should align with any env-var approach adopted in the compose file.
README.md	New optional LiteLLM setup section; instructs users to use host.docker.internal when the direct service name (litellm:4000) is available and more idiomatic within the compose project.

_{Reviews (1): Last reviewed commit: "feat(dev): self-hosted LiteLLM proxy com..." | Re-trigger Greptile}

[BjoernSchotte]

Thanks for the review — addressed in the latest commit:

• litellm-config.yaml gitignore gap (the concrete blocker): added an explicit litellm-config.yaml entry to .gitignore. .env* didn't cover it, so a git add . after copying the example would have committed the operator's keys. Fixed.
• German error message: translated the missing-key guard to English.
• Hardcoded master key: now ${LITELLM_MASTER_KEY:-sk-litellm-local-master} — overridable via env with the same default, and a comment noting it must stay in sync with litellm-config.yaml's master_key and the platform's LITELLM_API_KEY.

On the host.docker.internal:4000 vs litellm:4000 point — keeping host.docker.internal intentionally. LITELLM_API_BASE isn't only consumed by the compose web/worker containers; it's injected into the sandbox harness pods running in the kind cluster (outside the compose network). Those pods reach the host-published proxy port via host.docker.internal, and can't resolve the litellm compose service name. Using litellm:4000 would work for web/worker but break the in-cluster harness path, which is the one that actually makes the model calls.

[BjoernSchotte]

Follow-up — pushed ab48145:

• .gitignore: added litellm-config.yaml (the concrete gap — .env* didn't cover it).
• LITELLM_API_BASE: documented both forms instead of switching outright. host.docker.internal:4000 is the value for the kind sandbox backend (the default self-hosting path), because LITELLM_API_BASE is injected into the harness pods — which run on the kind Docker network, separate from this compose project, and can't resolve the litellm service name. litellm:4000 is noted as valid only when running without kind (LOCAL_SANDBOX_URL / brain-inline, all inside compose).

Verified from the kind node:

$ getent hosts litellm                 → (unresolvable)
$ curl http://litellm:4000/...         → Could not resolve host: litellm
$ getent hosts host.docker.internal    → 0.250.250.254
$ curl http://host.docker.internal:4000/health/liveliness → HTTP 200