Add API-key LLM backends

[larstalian]

What this is

Today OpenRange ships one builder LLM backend: CodexBackend, which shells out to the local codex exec CLI (src/openrange/llm.py). That means the only way to build a world right now is to install OpenAI's Codex CLI locally and have it authenticated.

Goal: let anyone install OpenRange, set one env var, and build a world — with whatever LLM provider they already have.

Plan

Two backends, not four:

1. LiteLLMBackend (primary)

Wrap LiteLLM behind an optional extra. One backend covers ~100 providers — OpenAI, Anthropic, Gemini, Bedrock, Vertex, Cohere, OpenRouter, Groq, Together, Fireworks, DeepSeek, Ollama, vLLM, llama.cpp server, and so on. LiteLLM handles the per-provider quirks for structured output (Anthropic tool-use, Gemini responseSchema, OpenAI response_format) so we don't end up maintaining four parallel code paths.

[project.optional-dependencies]
litellm = ["litellm>=1.50"]

@dataclass(frozen=True, slots=True)
class LiteLLMBackend:
    model: str  # e.g. "anthropic/claude-sonnet-4-5", "gpt-4o-mini", "ollama/llama3"
    api_key: str | None = None
    timeout: float = 120.0
    extra_params: Mapping[str, object] | None = None

    def complete(self, request: LLMRequest) -> LLMResult:
        # 1. Build messages from request.system / request.prompt
        # 2. If request.json_schema, pass response_format={"type": "json_schema", ...}
        #    — LiteLLM normalizes this per provider
        # 3. litellm.completion(..., timeout=self.timeout)
        # 4. Map LiteLLM exceptions to LLMBackendError; return LLMResult
        ...

2. OpenAICompatibleBackend (zero-dep escape hatch)

Stdlib-only (urllib), no new dependencies. For folks who don't want LiteLLM's footprint and just need to point at OpenAI, OpenRouter, Groq, Ollama, vLLM, or any other server that speaks the Chat Completions API.

#220 is in flight for this — review feedback over there.

Current contract

The full backend surface is one Protocol and two dataclasses, in src/openrange/llm.py:

class LLMBackend(Protocol):
    def complete(self, request: LLMRequest) -> LLMResult: ...

@dataclass(frozen=True, slots=True)
class LLMRequest:
    prompt: str
    system: str | None = None
    json_schema: Mapping[str, object] | None = None

@dataclass(frozen=True, slots=True)
class LLMResult:
    text: str
    parsed_json: Mapping[str, object] | None = None

Behavior contract:

• If request.json_schema is None, return the raw model text in LLMResult.text and leave parsed_json as None.
• If request.json_schema is not None, the backend MUST return a JSON object that conforms to that schema, populating both text (raw string) and parsed_json (parsed mapping). Use the provider's structured-output / JSON-schema feature where available; the existing parse_json_object helper handles the parse step and raises a clear LLMBackendError on bad output.
• system and prompt are joined with two newlines for plain-text providers (see LLMRequest.as_prompt); native message-based APIs should map them to system/user roles instead.
• Surface HTTP, auth, and timeout failures as LLMBackendError with a useful message.

Non-obvious thing to know

complete_with_pack_dir in src/openrange/core/builder.py:566 special-cases CodexBackend to mount the pack directory into a writable sandbox so the CLI can read pack files. New backends do not need to do anything for this — the dispatch falls through to a plain llm.complete(request) for any non-CodexBackend instance, which is correct for HTTP APIs (the relevant pack content is already in the prompt).

A note on contributing

If you'd like to take one of these on, drop a quick comment here first so we don't accidentally end up with two attempts at the same backend. One backend per PR keeps reviews tractable, and please keep changes off the LLMBackend Protocol itself (streaming, tool calls, multimodal etc. belong in a separate discussion).

And if you're setting up git for this — please make sure your user.email is a verified email on your GitHub account, so the commits link back to you in the contributor graph.

Acceptance criteria

• New backend class implements complete(LLMRequest) -> LLMResult and is exported from the openrange public API.
• Dependency declared as a named optional extra; default install does not pull it in.
• Unit tests covering at minimum: plain text completion, structured JSON completion against a schema, HTTP/provider error mapped to LLMBackendError, timeout mapped to LLMBackendError. Mock the network — do not hit real APIs in CI. See tests/test_llm_and_dashboard.py for the existing pattern.
• One short note added to the README listing the supported backends and how to install the extra.
• An end-to-end smoke test against a live API is welcome but must be guarded behind an env var so CI does not require provider credentials.

[snakefood3232]

I see the goal to add API-key LLM backends to eliminate the need for local CLI installations. I'd handle this by implementing additional backends using major LLM providers' API-key authentication, ensuring they adhere to the existing LLMBackend protocol for seamless integration. Expect a PR within 3 days. I've previously developed similar backend integrations for LLMs, ensuring robust API-key handling and schema compliance.