BYOK RX — Bring Your Own Key

One interface client, multiple providers. 😎

Route AI requests across multiple LLM providers with automatic fallback, all through a unified IChatClient interface.

Highlights

• Unified interface — single IChatClient that routes to OpenAI, Anthropic, Groq, DeepSeek, Ollama, OpenRouter, and any OpenAI-compatible provider
• Automatic fallback — define fallback chains per model; if a provider fails, BYOK tries the next one
• Model aliases — friendly names like "fast" or "cheap" that resolve to real model paths
• Per-provider key rotation — each provider gets its own API key provider; keys can rotate at runtime without restart
• Function calling — built-in support through Microsoft.Extensions.AI.FunctionInvokingChatClient
• OpenTelemetry ready — providers expose ChatClientMetadata for observability instrumentation
• Dependency injection — AddBYOKClient() extension for IServiceCollection

Install

dotnet add package BYOK.Core
dotnet add package BYOK.OpenAI
dotnet add package BYOK.Anthropic

Packages coming soon to NuGet. For now, clone and build locally.

Quick Start

using Microsoft.Extensions.AI;
using BYOK.Core;
using BYOK.OpenAI;
using BYOK.Anthropic;

var openAiKey = Environment.GetEnvironmentVariable("OPENAI_API_KEY") ?? "";
var anthropicKey = Environment.GetEnvironmentVariable("ANTHROPIC_API_KEY") ?? "";
var groqKey = Environment.GetEnvironmentVariable("GROQ_API_KEY") ?? "";

// ──────────────── Providers Setup ────────────────
var byok = BYOKClient.Create(options =>
{
    // built-in providers
    options.UseOpenAI(openAiKey);
    options.UseAnthropic(anthropicKey);
    options.UseGroq(groqKey);
    // ...

    // optional routing & alias
    options.Routing.ForModelId("openai:gpt-4o-mini")
        .FallbackTo("groq:groq/compound");

    options.Routing.ForAlias("mygroq")
        .Try("groq:groq/compound");

    // optional global timeout
    options.DefaultTimeout = TimeSpan.FromSeconds(100);
});

// ──────────────── Send a request ────────────────
var messages = new List<ChatMessage>
{
    new(ChatRole.System, "You are a helpful assistant."),
    new(ChatRole.User, "Hello!")
};

var chatOptions = new ChatOptions
{
    /* --- Change provider/model per request — no rebuild needed --- */
    ModelId = "openai:gpt-4o-mini" // "{providerName}:{modelId}" or "{alias}"
};

var response = await byok.GetResponseAsync(messages, chatOptions);

Console.WriteLine($"Result: {response.Text}");

Dependency injection

services.AddBYOKClient(options =>
{
    options.UseOpenAI(openAiKey);
    options.UseAnthropic(anthropicKey);

    options.Routing.ForModelId("openai:gpt-4o-mini")
        .FallbackTo("groq:groq/compound");
});

Then inject IChatClient anywhere:

public class MyService(IChatClient chatClient)
{
    public async Task RunAsync()
    {
        var response = await chatClient.GetResponseAsync(messages, new ChatOptions
        {
            ModelId = "openai:gpt-4o-mini" // ← providerName:modelId
        });
    }
}

Usage

Route format

Set ModelId to "{providerName}:{modelId}" and BYOK routes to the right provider:

ModelId = "openai:gpt-4o-mini"

Route	Provider	Model
openai:gpt-4o-mini	openai	gpt-4o-mini
groq:groq/compound	groq	groq/compound
deepseek:deepseek-flash	deepseek	deepseek-flash
ollama:llama3	ollama	llama3
anthropic:claude-sonnet-4-20250514	anthropic	claude-sonnet-4-20250514

The provider name must match one registered via UseOpenAICompatible(), UseAnthropicCompatible(), or one of the built-in extensions.

Built-in providers

Extension	Provider name (default)	Notes
UseOpenAI()	openai	api.openai.com/v1
UseOpenAICompatible()	custom	Any OpenAI-compatible endpoint
UseGroq()	groq	api.groq.com/openai/v1
UseOpenRouter()	openrouter	openrouter.ai/api/v1
UseOllama()	ollama	localhost:11434/v1 (no API key needed)
UseDeepSeek()	deepseek	api.deepseek.com/v1
UseAnthropic()	anthropic	api.anthropic.com
UseAnthropicCompatible()	custom	Any Anthropic-compatible endpoint
UseDeepSeekAnthropic()	deepseek-anthropic	api.deepseek.com/anthropic
UseOllamaAnthropic()	ollama-anthropic	localhost:11434 (Anthropic-compatible)

Fallback

options.Routing.ForModelId("openai:gpt-4o-mini")
    .FallbackTo("groq:groq/compound")
    .FallbackTo("anthropic:claude-sonnet-4-20250514");

If the primary fails, BYOK tries fallbacks in order.

Aliases

ForAlias() sets an alias you can use as ModelId:

options.Routing.ForAlias("fast").Try("groq:groq/compound");
options.Routing.ForAlias("cheap").Try("groq:llama-3-70b");

Use anywhere a route is expected: ModelId = "fast".

Custom providers

options.UseOpenAICompatible(
    providerName: "myprovider"
    apiKey: openAiKey,
    configure: config =>
    {
        config.BaseUrl = "https://my-custom-endpoint.com/v1";
        config.AddHeader("X-Custom", "value");
        config.NativeOptions.ClientLoggingOptions.LogLevel = OpenAI.Diagnostics.ClientLogLevel.Informational;
    },);

options.UseAnthropicCompatible(
    providerName: "myanthropic"
    apiKey: anthropicKey,
    configure: config =>
    {
        config.BaseUrl = "https://my-custom-anthropic.com";
    },);

Then use: ModelId = "myprovider:some-model" or ModelId = "myanthropic:some-model".

Or register any custom IChatClient with a factory:

options.UseProvider("my-custom", () => new MyChatClient(apiKey));

Streaming

await foreach (var update in byok.GetStreamingResponseAsync(messages, chatOptions))
{
    Console.Write(update.Text);
}

Function calling

var fcMessages = new List<ChatMessage>
{
    new(ChatRole.System, "You are a helpful assistant that can use tools."),
    new(ChatRole.User, "What time is it?")
};

var fcOptions = new ChatOptions
{
    ModelId = "groq:openai/gpt-oss-120b",
    Tools = [AIFunctionFactory.Create(GetCurrentTime)]
};

var response = await byok.GetResponseAsync(fcMessages, fcOptions);
Console.WriteLine($"Result: {response.Text}"); 

// [AI]: The current time is 14:30:25...

Define your function tool:

[Description("Returns the current time")]
static string GetCurrentTime() => DateTime.Now.ToString("HH:mm:ss");

Architecture

┌──────────────────────────────────────────┐
│              BYOKClient                   │
│  Routing → Provider Factory → IChatClient │
└──────────────────────────────────────────┘
         │                     │
         ▼                     ▼
  BYOK.OpenAI           BYOK.Anthropic
  OpenAICompatibleClient AnthropicChatClient
  (IChatClient)          (IChatClient)

Each provider client caches the underlying SDK client and only rebuilds when the API key changes.

License

[MIT] © BracoZS