feat: replace provider layer with Google Genkit Go SDK by intel352 · Pull Request #5 · GoCodeAlone/workflow-plugin-agent

intel352 · 2026-04-05T10:54:51Z

Summary

Replace all 12+ hand-rolled provider implementations (Anthropic, OpenAI, Gemini, Ollama, Copilot, OpenRouter, Cohere, HuggingFace, llama.cpp, Bedrock, Vertex, Azure) with Google Genkit Go SDK v1.6.0 adapters
New genkit/ package: adapter, type conversion, factory functions — Genkit is an internal implementation detail
provider.Provider interface unchanged — executor, ratchet-cli, mesh, orchestrator all work without modification
ProviderRegistry factories updated to call Genkit-backed constructors
Deleted ~25 provider files (~3000+ lines of hand-rolled HTTP/SDK code)
Fixed TOCTOU race in ProviderRegistry cache (concurrent cold-start)
Fixed credentialsJSON passthrough for VertexAI provider

What's Kept

provider/provider.go (interface + types)
provider/models.go, provider/auth_modes.go
provider/llama_cpp_download.go (utility)
All test providers (test_provider*.go)
executor/ package (unchanged)
tools/ package (unchanged)
orchestrator/ (factory functions updated)

What's Gained

Unified SDK for all providers (Genkit handles auth, streaming, format conversion)
Built-in tracing/observability via Genkit dev UI
MCP client/server support
Structured output (GenerateData[T]()) available for future use
Fewer direct dependencies (SDKs become transitive via Genkit)

Test plan

go build ./... passes
All 12 test packages pass
Spec review: all 8 checks pass (interface stable, thinking traces, llama_cpp_download preserved)
Code review: TOCTOU race and credentialsJSON fixes applied
Manual: test with real Anthropic/OpenAI/Ollama API keys end-to-end

🤖 Generated with Claude Code

Replace all hand-rolled provider implementations with Google Genkit Go SDK adapters. Keeps provider.Provider interface unchanged — Genkit is an internal implementation detail. All consumers (executor, ratchet-cli, mesh) unaffected. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

8-task plan covering: dependency setup, type conversion, adapter, factory functions, registry update, file deletion, dep cleanup, tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Add github.com/firebase/genkit/go@v1.6.0 and its plugins (anthropic, googlegenai, ollama, compat_oai) as dependencies. Create genkit/ package with a lazily-initialized shared Genkit instance for test/mock scenarios. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Implement bidirectional conversion between provider.Message/Response/ StreamEvent types and Genkit ai.Message/ModelResponse/ModelResponseChunk. Includes thinking trace support mapping ai.PartReasoning to StreamEvent {Type:"thinking"} per the critical alignment requirement. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add genkitProvider struct implementing provider.Provider via Genkit's Generate and GenerateStream APIs. Tools are registered lazily per unique name to prevent duplicate registration panics. Streaming extracts thinking/tool calls from the final response. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Implement factory functions for Anthropic, OpenAI, Google AI, Ollama, OpenAI-compatible (OpenRouter/Copilot/Cohere/llama.cpp), Azure OpenAI, Anthropic Foundry, Vertex AI, and AWS Bedrock. Each factory creates a Genkit instance with the appropriate plugin. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace all provider.New*Provider() calls in ProviderRegistry factory functions with gkprov.New*Provider() calls that create Genkit-backed providers. OpenAI-compatible factories (openrouter, copilot, cohere, copilot_models, llama_cpp) route through NewOpenAICompatibleProvider. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Delete 26 hand-rolled provider files and replace their usage in provider_registry.go, module_provider.go, and step_model_pull.go with Genkit factory calls. Preserve anthropic_bedrock.go (still used by genkit/providers.go) alongside new anthropic_bedrock_convert.go. Add OllamaClient for Pull/ListModels utility access in step_model_pull. Move models.go constants/types/helpers from deleted files. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Demotes golang.org/x/oauth2 to indirect — no longer directly used after removing old hand-rolled provider implementations. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Integration tests behind //go:build integration tag. Each test skips if the required API key env var is unset, covering: - Anthropic, OpenAI, Google AI, Ollama, OpenRouter - AWS Bedrock, Vertex AI, Azure OpenAI, Anthropic Foundry - provider.Provider interface satisfaction - streaming thinking trace propagation Run with: go test -tags integration ./genkit/... Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Store maxTokens in genkitProvider struct and apply it to every Chat/Stream call via ai.WithConfig(&ai.GenerationCommonConfig{ MaxOutputTokens: maxTokens}) when non-zero. All provider factory functions now populate the field. Fixes spec-reviewer Issue 2: maxTokens was silently dropped. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…rovider cache - genkit/providers.go: when credentialsJSON is non-empty, write it to a temp file and set GOOGLE_APPLICATION_CREDENTIALS before calling gk.Init so Genkit's VertexAI plugin picks it up via credentials.DetectDefault(). Previously the parameter was accepted but silently ignored. - orchestrator/provider_registry.go: re-check the cache under the write lock in createAndCache before inserting, eliminating the TOCTOU race where two concurrent callers for the same uncached alias both created a provider. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

This PR migrates the agent’s provider implementation layer from multiple hand-rolled SDK/HTTP integrations to a unified set of Genkit Go SDK adapters, while keeping the existing provider.Provider interface stable for the rest of the codebase.

Changes:

Introduces a new genkit/ package implementing provider.Provider via Genkit plugins (plus unit + integration tests).
Updates provider registries and module factories to construct Genkit-backed providers and fixes a ProviderRegistry cache TOCTOU race.
Removes legacy provider implementations/tests and replaces Ollama “utility” operations (list/pull/health) with a dedicated OllamaClient.

Reviewed changes

Copilot reviewed 43 out of 44 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
step_model_pull.go	Switches Ollama model pull/list to `provider.OllamaClient`.
provider/openrouter.go	Removes legacy OpenRouter provider implementation.
provider/openrouter_test.go	Removes legacy OpenRouter provider tests.
provider/openai.go	Removes legacy OpenAI provider implementation.
provider/openai_convert.go	Removes legacy OpenAI conversion/helpers.
provider/openai_azure.go	Removes legacy Azure OpenAI provider implementation.
provider/openai_azure_test.go	Removes legacy Azure OpenAI provider tests.
provider/ollama.go	Removes legacy Ollama provider implementation.
provider/ollama_test.go	Removes legacy Ollama provider tests.
provider/ollama_convert.go	Removes legacy Ollama conversion/helpers.
provider/ollama_convert_test.go	Removes legacy Ollama conversion tests.
provider/ollama_client.go	Adds Ollama utility client (pull/list/health).
provider/models.go	Moves/shared model-listing constants and updates Ollama model listing to use `OllamaClient`; adds Gemini model listing via `genai` SDK.
provider/llama_cpp.go	Removes legacy llama.cpp provider implementation.
provider/llama_cpp_test.go	Removes legacy llama.cpp provider tests.
provider/gemini.go	Removes legacy Gemini provider implementation.
provider/gemini_test.go	Removes legacy Gemini provider tests.
provider/copilot.go	Removes legacy Copilot provider implementation.
provider/copilot_test.go	Removes legacy Copilot provider tests.
provider/copilot_models.go	Removes legacy GitHub Models provider implementation.
provider/copilot_models_test.go	Removes legacy GitHub Models provider tests.
provider/cohere.go	Removes legacy Cohere provider implementation.
provider/anthropic.go	Removes legacy Anthropic provider implementation (direct API).
provider/anthropic_vertex.go	Removes legacy Anthropic Vertex provider implementation.
provider/anthropic_vertex_test.go	Removes legacy Anthropic Vertex provider tests.
provider/anthropic_foundry.go	Removes legacy Anthropic Foundry provider implementation.
provider/anthropic_foundry_test.go	Removes legacy Anthropic Foundry provider tests.
provider/anthropic_convert_test.go	Removes legacy Anthropic conversion tests.
provider/anthropic_bedrock_convert.go	Restores `defaultAnthropicMaxTokens` constant needed by remaining Bedrock codepaths.
provider_registry.go	Updates agent-level ProviderRegistry factories to use Genkit-backed constructors.
orchestrator/provider_registry.go	Updates orchestrator ProviderRegistry factories to use Genkit-backed constructors; fixes TOCTOU cache race.
module_provider.go	Updates `agent.provider` module to use Genkit-backed constructors.
genkit/genkit.go	Adds Genkit instance helper.
genkit/providers.go	Adds Genkit-backed provider factory functions (Anthropic/OpenAI/GoogleAI/Ollama/OpenAI-compatible/Azure/Foundry/Vertex/Bedrock wrapper).
genkit/providers_test.go	Adds unit tests for Genkit provider factory validation.
genkit/convert.go	Adds provider↔Genkit message/stream conversion.
genkit/convert_test.go	Adds unit tests for conversion logic.
genkit/adapter.go	Adds `provider.Provider` adapter over Genkit `Generate`/`GenerateStream` (tool support, usage, thinking).
genkit/adapter_test.go	Adds adapter tests using in-memory Genkit models.
genkit/integration_test.go	Adds opt-in (`integration` build tag) end-to-end tests against real provider APIs.
go.mod	Adds Genkit dependency and adjusts transitive deps.
go.sum	Updates dependency checksums for Genkit and transitives.
docs/plans/2026-04-05-genkit-provider-migration.md	Adds implementation plan doc for the migration.
docs/plans/2026-04-05-genkit-provider-migration-design.md	Adds design doc for the migration.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-05T11:02:06Z

genkit/providers.go

+	// Genkit's VertexAI plugin uses credentials.DetectDefault() which reads
+	// GOOGLE_APPLICATION_CREDENTIALS. When inline JSON is provided, write it
+	// to a temp file and point the env var at it.
+	if credentialsJSON != "" {
+		f, err := os.CreateTemp("", "vertexai-creds-*.json")
+		if err != nil {
+			return nil, fmt.Errorf("vertexai: create temp credentials file: %w", err)
+		}
+		if _, err := f.WriteString(credentialsJSON); err != nil {
+			_ = f.Close()
+			_ = os.Remove(f.Name())
+			return nil, fmt.Errorf("vertexai: write credentials: %w", err)
+		}
+		_ = f.Close()
+		os.Setenv("GOOGLE_APPLICATION_CREDENTIALS", f.Name()) //nolint:errcheck
+	}


NewVertexAIProvider writes credentials JSON to a temp file and sets GOOGLE_APPLICATION_CREDENTIALS globally, but never removes the temp file nor restores the prior env var. This can leak service-account credentials on disk and introduces cross-request races when multiple providers are created concurrently in the same process. Prefer passing credentials to the underlying client/plugin without mutating process env; if that’s not possible, guard the env var change with a mutex, restore the previous value, and ensure the temp file is deleted when no longer needed (including on error paths).

Copilot · 2026-04-05T11:02:06Z

genkit/providers.go

+// NewAnthropicProvider creates a provider backed by Genkit's Anthropic plugin.
+func NewAnthropicProvider(ctx context.Context, apiKey, model, baseURL string, maxTokens int) (provider.Provider, error) {
+	if apiKey == "" {
+		return nil, fmt.Errorf("anthropic: APIKey is required")
+	}
+	p := &anthropicPlugin.Anthropic{APIKey: apiKey, BaseURL: baseURL}
+	g := initGenkitWithPlugin(ctx, gk.WithPlugins(p))
+	return &genkitProvider{
+		g:         g,
+		modelName: "anthropic/" + model,
+		name:      "anthropic",
+		maxTokens: maxTokens,
+		authInfo: provider.AuthModeInfo{
+			Mode:        "api_key",
+			DisplayName: "Anthropic",
+			ServerSafe:  true,
+		},
+	}, nil
+}
+
+// NewOpenAIProvider creates a provider backed by Genkit's OpenAI plugin.
+func NewOpenAIProvider(ctx context.Context, apiKey, model, baseURL string, maxTokens int) (provider.Provider, error) {
+	if apiKey == "" {
+		return nil, fmt.Errorf("openai: APIKey is required")
+	}
+	var extraOpts []option.RequestOption
+	if baseURL != "" {
+		extraOpts = append(extraOpts, option.WithBaseURL(baseURL))
+	}
+	p := &openaiPlugin.OpenAI{APIKey: apiKey, Opts: extraOpts}
+	g := initGenkitWithPlugin(ctx, gk.WithPlugins(p))
+	return &genkitProvider{
+		g:         g,
+		modelName: "openai/" + model,
+		name:      "openai",
+		maxTokens: maxTokens,


The factory functions build modelName as "<provider>/" + model without applying the prior defaulting behavior (e.g., OpenAI default model, Anthropic default model). Since module and DB schemas allow model to be empty (default ''), this can produce invalid model names like openai/ and fail at runtime. Consider preserving existing defaults inside these factories when model is empty to maintain backward compatibility.

orchestrator/provider_registry.go

provider_registry.go

Copilot · 2026-04-05T11:02:07Z

module_provider.go

 		case "anthropic":
-			p = provider.NewAnthropicProvider(provider.AnthropicConfig{
-				APIKey:    apiKey,
-				Model:     model,
-				BaseURL:   baseURL,
-				MaxTokens: maxTokens,
-			})
+			if prov, err := gkprov.NewAnthropicProvider(context.Background(), apiKey, model, baseURL, maxTokens); err != nil {
+				p = &errProvider{err: err}
+			} else {
+				p = prov
+			}

 		case "openai":
-			p = provider.NewOpenAIProvider(provider.OpenAIConfig{
-				APIKey:    apiKey,
-				Model:     model,
-				BaseURL:   baseURL,
-				MaxTokens: maxTokens,
-			})
+			if prov, err := gkprov.NewOpenAIProvider(context.Background(), apiKey, model, baseURL, maxTokens); err != nil {
+				p = &errProvider{err: err}
+			} else {
+				p = prov
+			}

 		case "copilot":
-			p = provider.NewCopilotProvider(provider.CopilotConfig{
-				Token:     apiKey,
-				Model:     model,
-				BaseURL:   baseURL,
-				MaxTokens: maxTokens,
-			})
+			if baseURL == "" {
+				baseURL = "https://api.githubcopilot.com"
+			}
+			if prov, err := gkprov.NewOpenAICompatibleProvider(context.Background(), "copilot", apiKey, model, baseURL, maxTokens); err != nil {
+				p = &errProvider{err: err}
+			} else {
+				p = prov
+			}

 		case "ollama":
-			p = provider.NewOllamaProvider(provider.OllamaConfig{
-				Model:     model,
-				BaseURL:   baseURL,
-				MaxTokens: maxTokens,
-			})
+			if prov, err := gkprov.NewOllamaProvider(context.Background(), model, baseURL, maxTokens); err != nil {
+				p = &errProvider{err: err}
+			} else {
+				p = prov
+			}

 		case "llama_cpp":
-			p = provider.NewLlamaCppProvider(provider.LlamaCppConfig{
-				BaseURL:   baseURL,
-				ModelPath: model,
-				ModelName: model,
-				MaxTokens: maxTokens,
-			})
+			if prov, err := gkprov.NewOpenAICompatibleProvider(context.Background(), "llama_cpp", "", model, baseURL, maxTokens); err != nil {
+				p = &errProvider{err: err}
+			} else {
+				p = prov
+			}


Module provider creation also uses context.Background() for Genkit provider factories. This prevents module users from passing cancellation/deadline semantics into any initialization work performed by the Genkit plugins. Prefer using a context associated with the module lifecycle (or at minimum TODO/option to supply one) rather than hard-coding context.Background().

Copilot · 2026-04-05T11:02:07Z

genkit/convert.go

+	// Extract tool calls
+	if msg := resp.Message; msg != nil {
+		for _, part := range msg.Content {
+			if part.ToolRequest != nil {
+				tc := provider.ToolCall{
+					ID:        part.ToolRequest.Name,
+					Name:      part.ToolRequest.Name,
+					Arguments: make(map[string]any),
+				}
+				if input, ok := part.ToolRequest.Input.(map[string]any); ok {
+					tc.Arguments = input
+				}
+				out.ToolCalls = append(out.ToolCalls, tc)
+			}


Tool request conversion currently sets ToolCall.ID and ToolCall.Name both to part.ToolRequest.Name. This loses any per-call identifier and makes multiple tool calls to the same tool indistinguishable (and tool results will all reuse the same ToolCallID). If Genkit provides a distinct call ID, map it into ToolCall.ID and keep the tool name in ToolCall.Name; otherwise generate a stable unique ID per tool request and ensure the corresponding tool result messages map back correctly.

Copilot · 2026-04-05T11:02:07Z

genkit/adapter.go

+	refs := make([]ai.ToolRef, 0, len(tools))
+	for _, t := range tools {
+		if !p.definedTools[t.Name] {
+			tool := gk.DefineTool(p.g, t.Name, t.Description,
+				func(ctx *ai.ToolContext, input map[string]any) (map[string]any, error) {
+					// Tools are executed by the executor, not Genkit.
+					return nil, fmt.Errorf("tool %s should not be called via Genkit", t.Name)
+				},
+			)


resolveToolRefs defines tools via gk.DefineTool using an input map[string]any handler, but it doesn’t incorporate provider.ToolDef.Parameters (JSON Schema). As a result, the model may not receive the expected parameter schema for tool calling, which can break or degrade tool-call generation. Consider defining tools with an explicit schema (or using a typed input struct) so tool parameters are communicated accurately to Genkit/underlying providers.

1. VertexAI credentials: guard env var with mutex, restore previous value, and delete temp file after Genkit init completes. 2. Empty model defaults: add default model constants per provider (claude-sonnet-4-6, gpt-4o, gemini-2.5-flash, qwen3:8b) applied when model param is empty. 3-5. Context propagation: change ProviderFactory signature to accept context.Context, thread caller's ctx through all factory functions in both ProviderRegistry files. Module factories use context.TODO() with comment explaining the ModuleFactory limitation. 6. ToolCall.ID uniqueness: generate uuid.New().String() per tool call instead of reusing the tool name as ID. 7. Tool schema passthrough: use ai.WithInputSchema(t.Parameters) in DefineTool so the LLM receives accurate JSON Schema for each tool. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 44 out of 45 changed files in this pull request and generated 12 comments.

Comments suppressed due to low confidence (1)

provider_registry.go:202

createAndCache writes to the cache unconditionally after provider creation, so concurrent cold-starts can race and initialize the same provider multiple times (and the last write wins). The orchestrator registry addresses this with a re-check under the write lock; consider applying the same pattern (or singleflight) here to avoid duplicate Genkit/plugin initialization under load.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-05T18:36:42Z

genkit/convert.go

+				parts = append(parts, ai.NewToolRequestPart(&ai.ToolRequest{
+					Name:  tc.Name,


Tool-call identity is not preserved across turns: assistant tool requests are created with ToolRequest.Name=tc.Name (tc.ID is discarded), but tool results are encoded as ToolResponse.Name=m.ToolCallID. Since ToolCallID is later set to tc.ID (currently generated UUIDs), the tool response won't match the original tool request, breaking multi-turn tool calling. Align the identifier used in ToolRequest/ToolResponse (and preserve the provider.ToolCall.ID across conversions) so tool results can be correlated to the right request.

Suggested change

parts = append(parts, ai.NewToolRequestPart(&ai.ToolRequest{

Name: tc.Name,

toolCallName := tc.Name

if tc.ID != "" {

toolCallName = tc.ID

}

parts = append(parts, ai.NewToolRequestPart(&ai.ToolRequest{

Name: toolCallName,

Copilot · 2026-04-05T18:36:43Z

genkit/convert.go

+		} else if len(m.ToolCalls) > 0 {
+			// Assistant message with tool calls
+			for _, tc := range m.ToolCalls {
+				parts = append(parts, ai.NewToolRequestPart(&ai.ToolRequest{
+					Name:  tc.Name,
+					Input: tc.Arguments,
+				}))
+			}
+		} else {
+			parts = []*ai.Part{ai.NewTextPart(m.Content)}
+		}


When an assistant message includes ToolCalls, the text Content is dropped entirely (only ToolRequest parts are emitted). The executor records assistant Content alongside ToolCalls, so dropping it changes the conversation history and can affect model behavior. Preserve any non-empty assistant Content by emitting a text part in addition to tool-request parts (matching the behavior of the previous provider conversions).

Copilot · 2026-04-05T18:36:43Z

genkit/convert.go

+		if m.ToolCallID != "" {
+			parts = []*ai.Part{ai.NewToolResponsePart(&ai.ToolResponse{
+				Name:   m.ToolCallID,
+				Output: map[string]any{"result": m.Content},
+			})}


Tool results are always wrapped as {result: <string>}. In this codebase, executor tool results are typically JSON-encoded strings (or error strings), so double-wrapping can distort the intended structure presented to the model. Consider attempting to JSON-decode Message.Content first (use the decoded value as ToolResponse.Output when valid), and only fall back to a string wrapper when decoding fails.

Copilot · 2026-04-05T18:36:43Z

genkit/adapter.go

+				// Extract final response for tool calls and usage
+				if result.Response != nil {
+					final := fromGenkitResponse(result.Response)
+					for i := range final.ToolCalls {
+						tc := final.ToolCalls[i]
+						ch <- provider.StreamEvent{Type: "tool_call", Tool: &tc}
+					}


Stream() emits tool_call events from chunks (via fromGenkitChunk) and then emits tool_call events again at Done by converting the final response. Because tool-call IDs are generated during conversion, the Done-phase tool_call IDs won't match earlier tool_call events and can also cause duplicate execution downstream. Prefer emitting tool_call events from a single source (either incremental chunks or final response) and ensure IDs are stable across the stream.

Suggested change

// Extract final response for tool calls and usage

if result.Response != nil {

final := fromGenkitResponse(result.Response)

for i := range final.ToolCalls {

tc := final.ToolCalls[i]

ch <- provider.StreamEvent{Type: "tool_call", Tool: &tc}

}

// Tool calls are emitted from incremental chunks only.

// On Done, only emit usage metadata to avoid duplicate tool_call

// events with unstable IDs from final response conversion.

if result.Response != nil {

final := fromGenkitResponse(result.Response)

Copilot · 2026-04-05T18:36:43Z

genkit/providers.go

+	defaultAnthropicModel = "claude-sonnet-4-6"
+	defaultOpenAIModel    = "gpt-4o"
+	defaultGeminiModel    = "gemini-2.5-flash"


The default model constants differ from the previous provider implementations (e.g., Anthropic default was claude-sonnet-4-20250514, Gemini default was gemini-2.0-flash). This changes runtime behavior for configs that omit Model. If the goal is a drop-in replacement, keep defaults aligned with the old providers (or document the behavior change explicitly).

Suggested change

defaultAnthropicModel = "claude-sonnet-4-6"

defaultOpenAIModel = "gpt-4o"

defaultGeminiModel = "gemini-2.5-flash"

defaultAnthropicModel = "claude-sonnet-4-20250514"

defaultOpenAIModel = "gpt-4o"

defaultGeminiModel = "gemini-2.0-flash"

Copilot · 2026-04-05T18:36:44Z

orchestrator/provider_registry.go

+func copilotModelsProviderFactory(ctx context.Context, apiKey string, cfg LLMProviderConfig) (provider.Provider, error) {
+	baseURL := cfg.BaseURL
+	if baseURL == "" {
+		baseURL = "https://models.inference.ai.azure.com"


The default baseURL for copilot_models is set to https://models.inference.ai.azure.com, but the model-listing path (provider.ListModels) defaults copilot_models to https://models.github.ai/inference. These should match; otherwise copilot_models will point at a different endpoint depending on which code path is used. Update the default baseURL here to the intended GitHub Models endpoint (or adjust both places consistently).

Suggested change

baseURL = "https://models.inference.ai.azure.com"

baseURL = "https://models.github.ai/inference"

Copilot · 2026-04-05T18:36:44Z

module_provider.go

 		case "anthropic":
-			p = provider.NewAnthropicProvider(provider.AnthropicConfig{
-				APIKey:    apiKey,
-				Model:     model,
-				BaseURL:   baseURL,
-				MaxTokens: maxTokens,
-			})
+			if prov, err := gkprov.NewAnthropicProvider(context.TODO() /* ModuleFactory doesn't receive ctx; TODO: thread ctx via Start() */, apiKey, model, baseURL, maxTokens); err != nil {
+				p = &errProvider{err: err}
+			} else {
+				p = prov
+			}

 		case "openai":
-			p = provider.NewOpenAIProvider(provider.OpenAIConfig{
-				APIKey:    apiKey,
-				Model:     model,
-				BaseURL:   baseURL,
-				MaxTokens: maxTokens,
-			})
+			if prov, err := gkprov.NewOpenAIProvider(context.TODO() /* ModuleFactory doesn't receive ctx; TODO: thread ctx via Start() */, apiKey, model, baseURL, maxTokens); err != nil {
+				p = &errProvider{err: err}
+			} else {
+				p = prov
+			}

 		case "copilot":
-			p = provider.NewCopilotProvider(provider.CopilotConfig{
-				Token:     apiKey,
-				Model:     model,
-				BaseURL:   baseURL,
-				MaxTokens: maxTokens,
-			})
+			if baseURL == "" {
+				baseURL = "https://api.githubcopilot.com"
+			}
+			if prov, err := gkprov.NewOpenAICompatibleProvider(context.TODO() /* ModuleFactory doesn't receive ctx; TODO: thread ctx via Start() */, "copilot", apiKey, model, baseURL, maxTokens); err != nil {
+				p = &errProvider{err: err}


Provider initialization uses context.TODO() because ModuleFactory doesn’t receive a context. This prevents cancellation/timeouts during provider setup and can hang module loading if initialization blocks (e.g., auth discovery). Prefer threading a real context from module Start()/wiring, or at minimum use context.Background() and document the expected lifecycle/timeout behavior.

Copilot · 2026-04-05T18:36:44Z

orchestrator/provider_registry.go

-	}), nil
+func llamaCppProviderFactory(ctx context.Context, _ string, cfg LLMProviderConfig) (provider.Provider, error) {
+	// llama.cpp serves an OpenAI-compatible API
+	return gkprov.NewOpenAICompatibleProvider(ctx, "llama_cpp", "", cfg.Model, cfg.BaseURL, cfg.MaxTokens)


llama_cpp is wired through NewOpenAICompatibleProvider with BaseURL passed through unchanged. If cfg.BaseURL is empty, the provider will be initialized without a usable endpoint. Add a default base URL for llama_cpp when none is provided (matching previous llama.cpp provider defaults) or return a validation error early.

Suggested change

return gkprov.NewOpenAICompatibleProvider(ctx, "llama_cpp", "", cfg.Model, cfg.BaseURL, cfg.MaxTokens)

baseURL := cfg.BaseURL

if baseURL == "" {

baseURL = "http://127.0.0.1:8080/v1"

}

return gkprov.NewOpenAICompatibleProvider(ctx, "llama_cpp", "", cfg.Model, baseURL, cfg.MaxTokens)

Copilot · 2026-04-05T18:36:44Z

provider_registry.go

+	r.Factories["anthropic"] = func(ctx context.Context, apiKey string, cfg LLMProviderConfig) (provider.Provider, error) {
+		return gkprov.NewAnthropicProvider(ctx, apiKey, cfg.Model, cfg.BaseURL, cfg.MaxTokens)
 	}
-	r.Factories["openai"] = func(apiKey string, cfg LLMProviderConfig) (provider.Provider, error) {
-		return provider.NewOpenAIProvider(provider.OpenAIConfig{
-			APIKey:    apiKey,
-			Model:     cfg.Model,
-			BaseURL:   cfg.BaseURL,
-			MaxTokens: cfg.MaxTokens,
-		}), nil
+	r.Factories["openai"] = func(ctx context.Context, apiKey string, cfg LLMProviderConfig) (provider.Provider, error) {
+		return gkprov.NewOpenAIProvider(ctx, apiKey, cfg.Model, cfg.BaseURL, cfg.MaxTokens)
 	}
-	r.Factories["openrouter"] = func(apiKey string, cfg LLMProviderConfig) (provider.Provider, error) {
-		if cfg.BaseURL == "" {
-			cfg.BaseURL = "https://openrouter.ai/api/v1"
+	r.Factories["openrouter"] = func(ctx context.Context, apiKey string, cfg LLMProviderConfig) (provider.Provider, error) {
+		baseURL := cfg.BaseURL
+		if baseURL == "" {
+			baseURL = "https://openrouter.ai/api/v1"
 		}
-		return provider.NewOpenAIProvider(provider.OpenAIConfig{
-			APIKey:    apiKey,
-			Model:     cfg.Model,
-			BaseURL:   cfg.BaseURL,
-			MaxTokens: cfg.MaxTokens,
-		}), nil
+		return gkprov.NewOpenAICompatibleProvider(ctx, "openrouter", apiKey, cfg.Model, baseURL, cfg.MaxTokens)
 	}
-	r.Factories["copilot"] = func(apiKey string, cfg LLMProviderConfig) (provider.Provider, error) {
-		return provider.NewCopilotProvider(provider.CopilotConfig{
-			Token:     apiKey,
-			Model:     cfg.Model,
-			BaseURL:   cfg.BaseURL,
-			MaxTokens: cfg.MaxTokens,
-		}), nil
+	r.Factories["copilot"] = func(ctx context.Context, apiKey string, cfg LLMProviderConfig) (provider.Provider, error) {
+		baseURL := cfg.BaseURL
+		if baseURL == "" {
+			baseURL = "https://api.githubcopilot.com"
+		}
+		return gkprov.NewOpenAICompatibleProvider(ctx, "copilot", apiKey, cfg.Model, baseURL, cfg.MaxTokens)
 	}


These ProviderRegistry factories pass user-configured BaseURL values into Genkit providers without validating them. The repo has SSRF protections via provider.ValidateBaseURL (https-only + blocks private/internal IPs) used in other code paths. Apply the same validation here for remote providers (e.g., anthropic/openai/openrouter/copilot) before creating the provider, and return an error when invalid.

Copilot · 2026-04-05T18:36:44Z

orchestrator/provider_registry.go

+	// Cache — re-check under write lock to avoid TOCTOU race
 	r.mu.Lock()
+	if existing, ok := r.cache[alias]; ok {
+		r.mu.Unlock()
+		return existing, nil
+	}


The post-create cache re-check avoids overwriting the cache entry, but it still allows multiple goroutines to concurrently create a provider on cold start (all but one instance are discarded). If provider creation is expensive (Genkit init, auth discovery), consider using a per-alias singleflight (or store a promise/future in the cache) so only one initialization runs per alias.

convert.go: - Tool-call identity: use tc.ID in ToolRequest.Name for correlation - Preserve assistant text alongside ToolCalls (don't drop Content) - Try JSON-decode tool results before wrapping (avoid double-wrap) - Remove chunk-based tool_call emission (emit only from final response) adapter.go: - Stream() emits tool_call events only from final Done response, avoiding duplicates with unstable IDs from incremental chunks providers.go: - Update default models to current: claude-sonnet-4-6, gpt-4.1, gemini-2.5-flash - Add SSRF validation (provider.ValidateBaseURL) for remote providers (Anthropic, OpenAI, OpenAI-compatible), skip for local providers - Add model default for OpenAICompatible (falls back to gpt-4.1) orchestrator/provider_registry.go: - Fix copilot_models default URL: models.github.ai/inference - Add llama_cpp default baseURL: http://127.0.0.1:8080/v1 - Use singleflight.Group to deduplicate concurrent cold-start provider creation per alias Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

intel352 and others added 12 commits April 5, 2026 06:04

docs: add Genkit provider migration implementation plan

d9d271c

8-task plan covering: dependency setup, type conversion, adapter, factory functions, registry update, file deletion, dep cleanup, tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

chore: run go mod tidy after provider cleanup (Task 7)

771cf9f

Demotes golang.org/x/oauth2 to indirect — no longer directly used after removing old hand-rolled provider implementations. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI review requested due to automatic review settings April 5, 2026 10:54

Copilot started reviewing on behalf of intel352 April 5, 2026 10:55 View session

Copilot AI reviewed Apr 5, 2026

View reviewed changes

intel352 requested a review from Copilot April 5, 2026 18:30

Copilot started reviewing on behalf of intel352 April 5, 2026 18:30 View session

Copilot AI reviewed Apr 5, 2026

View reviewed changes

intel352 merged commit 9dae1ae into master Apr 5, 2026
4 checks passed

intel352 deleted the feat/genkit-migration branch April 5, 2026 19:20

		parts = append(parts, ai.NewToolRequestPart(&ai.ToolRequest{
		Name: tc.Name,

-				parts = append(parts, ai.NewToolRequestPart(&ai.ToolRequest{
-					Name:  tc.Name,
+				toolCallName := tc.Name
+				if tc.ID != "" {
+					toolCallName = tc.ID
+				}
+				parts = append(parts, ai.NewToolRequestPart(&ai.ToolRequest{
+					Name:  toolCallName,

	baseURL = "https://models.inference.ai.azure.com"
	baseURL = "https://models.github.ai/inference"

Conversation

intel352 commented Apr 5, 2026

Summary

What's Kept

What's Gained

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants