Generalize LLM post-processing to any OpenAI-API-compatible server#57
Draft
JRufer wants to merge 7 commits into
Draft
Generalize LLM post-processing to any OpenAI-API-compatible server#57JRufer wants to merge 7 commits into
JRufer wants to merge 7 commits into
Conversation
Rename the settings tab from "Ollama" to "OpenAI API" and switch the LLM client from Ollama's native API to the OpenAI-compatible /v1/models and /v1/chat/completions endpoints, so any compatible server (Ollama, LM Studio, OpenAI, etc.) can be used. The server URL still defaults to localhost but remains user-editable, and a new optional API key field is sent as an Authorization bearer token for non-local servers. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PThnjsJtj5TRZNBzCLECoi
…_DISPLAY test_clipboard_target_linux_cli temporarily mutates the process-wide PATH and WAYLAND_DISPLAY env vars to mock wl-copy. Running concurrently with test_clipboard_target_success let the latter observe the mock during deliver() but not during the follow-up test() call once the other test had restored PATH, causing a flaky "assertion failed: test_res.reachable" failure in CI. Serialize both tests with a mutex. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PThnjsJtj5TRZNBzCLECoi
Lets a hotkey deliver transcribed text to any OpenAI-compatible chat completions endpoint and route the model's response (e.g. a summary or rewrite) to the target's destination, instead of just the raw transcript. Reuses the global OpenAI API connection settings (endpoint/api_key/model) with optional per-target overrides for model, max_tokens, and timeout. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PThnjsJtj5TRZNBzCLECoi
Config is wrapped (state.config.lock().await.data.*), not flat; the callback registered for the new OpenAI API target referenced .ollama directly instead of .data.ollama, breaking the build. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PThnjsJtj5TRZNBzCLECoi
cargo test failed to compile voxctrl-app because two OutputTarget struct literals in the sequential-delivery test predated the new openai_* fields. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PThnjsJtj5TRZNBzCLECoi
Each openai_api target now keeps an in-memory chat log (last 20 messages / 10 exchanges) and replays it with every request, so follow-up dictations like "what did I just ask you?" resolve correctly instead of being treated as an isolated one-shot prompt. History lives only in the running process and resets on hot-reload or restart. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PThnjsJtj5TRZNBzCLECoi
The earlier openai_api target history fix didn't help users whose Q&A flow goes through the legacy ollama_enabled pre-processing pipeline (OllamaClient::process), which was completely stateless. Add a process_with_history method and a per-binding in-memory history map on InferenceEngine so follow-up questions like "what did I just ask you" resolve correctly for that pipeline too.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Renames the Ollama settings tab to "OpenAI API" and generalizes the LLM client to use OpenAI-compatible endpoints (/v1/models, /v1/chat/completions) plus an optional API key field. See commit message for details.
🤖 Generated with Claude Code
Generated by Claude Code