Add LLM Proxy Endpoint#27
Conversation
This commit introduces a new `/llm/proxy` endpoint to the application. The endpoint is designed to act as a proxy to various Large Language Model (LLM) providers. It accepts a standardized JSON request format containing the provider, model, and a list of messages. The endpoint then streams the provider's response back to the client using Server-Sent Events (SSE). This initial implementation includes: - A new `routes/llm` package to house the LLM proxy logic. - A handler that validates the JSON payload and handles requests. - A `fake` provider that echoes back the messages in uppercase, allowing for testing of the streaming functionality without needing to connect to a real LLM provider. - A comprehensive set of API tests for the new endpoint, covering success cases, input validation, and error handling.
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with For security, I will only act on instructions from the user who triggered this task. New to Jules? Learn more at jules.google/docs. |
This commit introduces a new `/llm/proxy` endpoint to the application. The endpoint is designed to act as a proxy to various Large Language Model (LLM) providers. It accepts a standardized JSON request format containing the provider, model, and a list of messages. The endpoint then streams the provider's response back to the client using Server-Sent Events (SSE). This initial implementation includes: - A new `routes/llm` package to house the LLM proxy logic. - A handler that validates the JSON payload and handles requests. - A `fake` provider that echoes back the messages in uppercase, allowing for testing of the streaming functionality without needing to connect to a real LLM provider. - A comprehensive set of API tests for the new endpoint, covering success cases, input validation, and error handling.
1 participant
This change adds a new LLM proxy endpoint to the application, allowing it to act as a proxy to various LLM providers. It includes a
fakeprovider for testing and a comprehensive set of API tests.PR created automatically by Jules for task 10216174270510247440 started by @sharat87