You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
add OpenAICompatibleBackend for Chat Completions-compatible HTTP providers
support text and JSON-schema requests, API key/env auth, extra headers, timeouts, and HTTP error mapping
export the backend from the public API and document the openai-compatible optional install target
add mocked HTTP tests for text, structured JSON, HTTP errors, and timeout handling
Verification
uv lock --check
uv run ruff format src\openrange\llm.py src\openrange\__init__.py tests\test_openai_compatible_backend.py
uv run ruff check README.md pyproject.toml src\openrange\llm.py src\openrange\__init__.py tests\test_openai_compatible_backend.py
uv run mypy src\openrange\llm.py tests\test_openai_compatible_backend.py
uv run pytest tests\test_openai_compatible_backend.py -q
git diff --check
Note: I also tried uv run pytest tests\test_openai_compatible_backend.py tests\test_llm_and_dashboard.py -q; the new tests passed, but existing test_llm_and_dashboard.py cases fail on Windows because Python scripts are executed directly as Win32 binaries and os.getpgid is unavailable on Windows.
Thanks for putting this together — the test setup with a real local HTTPServer is nicer than the mock-heavy version I'd had in mind.
Quick heads-up on scope: I'm updating #188 to lead with a LiteLLMBackend (one library covers ~100 providers and handles the structured-output quirks for each), with this OpenAICompatibleBackend kept around as a zero-dep escape hatch — for folks who don't want to pull in pydantic / httpx / tiktoken and just need to point at Ollama, vLLM, or any other Chat Completions–shaped server. So this PR is still in scope, just framed a bit differently. The README copy should probably reflect that it's the lightweight stdlib option rather than the catch-all.
A few things while you're in there:
openai-compatible = [] in pyproject.toml doesn't actually install anything, so uv sync --extra openai-compatible is a no-op. Either drop the extra entirely (the backend is stdlib, no install step needed) or populate it.
The strict: True in the response_format payload — fine for OpenAI itself, but Ollama, vLLM, llama.cpp server, Together, and several OpenRouter models either reject it or silently ignore it. Since the README lists those as supported, either flip the default to False or expose it as a constructor option.
The api_key_env fallback path isn't covered by a test — every test passes api_key= explicitly. A small monkeypatch.setenv case would close that gap.
urlopen() accepts any scheme, including file:// and ftp://. Worth a scheme check in preflight() (http/https only), just so a typo in base_url doesn't quietly read a local file.
extra_headers can clobber Authorization — probably intentional for Azure-style api-key headers, but worth a docstring note.
The default OPENAI_COMPATIBLE_DEFAULT_MODEL = "gpt-4o-mini" is fine for api.openai.com but won't exist on most compatible servers. Either drop the default or document the caveat.
One small housekeeping thing: the commit on this branch is coming through as mac <mac@apple.com> rather than your GitHub identity, so it isn't linked to your profile in the contributor graph. If you don't mind amending the author (git commit --amend --author="Your Name <verified-email>" + force-push), it just makes git-blame / attribution cleaner down the line.
Happy to merge once these are sorted. Thanks again for picking this up.
removed the empty openai-compatible extra and refreshed uv.lock
made JSON schema strictness opt-in with json_schema_strict=True
added a non-http scheme guard in preflight()
added coverage for api_key_env
documented the lightweight stdlib positioning, auth header override behavior, and model caveat
amended the commit author to pntech20 <pntech20@gmail.com>
Focused checks passed locally:
uv run pytest tests/test_openai_compatible_backend.py
uv run ruff check src/openrange/llm.py tests/test_openai_compatible_backend.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
OpenAICompatibleBackendfor Chat Completions-compatible HTTP providersopenai-compatibleoptional install targetVerification
uv lock --checkuv run ruff format src\openrange\llm.py src\openrange\__init__.py tests\test_openai_compatible_backend.pyuv run ruff check README.md pyproject.toml src\openrange\llm.py src\openrange\__init__.py tests\test_openai_compatible_backend.pyuv run mypy src\openrange\llm.py tests\test_openai_compatible_backend.pyuv run pytest tests\test_openai_compatible_backend.py -qgit diff --checkNote: I also tried
uv run pytest tests\test_openai_compatible_backend.py tests\test_llm_and_dashboard.py -q; the new tests passed, but existingtest_llm_and_dashboard.pycases fail on Windows because Python scripts are executed directly as Win32 binaries andos.getpgidis unavailable on Windows.Refs #188