Move integration tool descriptions out of Go and into YAML#197
Draft
arjansingh wants to merge 8 commits into
Draft
Move integration tool descriptions out of Go and into YAML#197arjansingh wants to merge 8 commits into
arjansingh wants to merge 8 commits into
Conversation
The two meta-tools, search and execute, carry long descriptions that teach the model how to find and call integration tools. Those descriptions used to live inside the Go source. Every wording change looked like a code change in review, and needed a rebuild to ship. We moved the words into markdown files, one per tool. Now the code owns behavior and the markdown owns the words. Reworking how we explain a tool to the model is a content edit you can review on its own. We tightened the wording while it moved. The model is told to ask before guessing a tool name. Every integration's main tool opens with the same "start here" hint, so they all sit at the same discoverability bar. And we pushed back on the mistakes models make most: calling execute before searching, and guessing argument shapes instead of reading the schema. Co-Authored-By: claude-opus-4-7 <noreply@anthropic.com>
…equired A tool's parameters and which ones were required used to live in two separate places: a map of parameter names to descriptions, and a separate list of the names that were required. Nothing tied the two together. A typo in the required list, like "querry" instead of "query", compiled fine and quietly changed the tool's contract. You could even mark a parameter required that did not exist. We folded the two into one. Each parameter now carries its own required flag, so a parameter that is required but never declared is no longer something you can write down. All 36 adapters moved to the new shape with the same parameters and required flags as before. Nothing the server hands out changed, and a snapshot test holds us to that. Co-Authored-By: claude-opus-4-7 <noreply@anthropic.com>
Every integration's tools were written inline in Go, about 1,400 of them across 36 adapters. Editing the wording for one tool meant a code change, reviewed next to dispatch maps and everything else in the file. Writing prose was slow and the reviews were noisy. We moved all of it into a YAML file per adapter. Each adapter's tool code is now a few lines that load the YAML. Editing a description is a content edit with no Go in the way. A misspelled key fails at startup instead of quietly shipping a tool with a blank description. The few adapters that build their tools on the fly from a remote server keep doing that; only the ones with a fixed tool set moved. Nothing the server hands out changed. The same snapshot test from the reshape holds the tool list identical across all 36 adapters. Co-Authored-By: claude-opus-4-7 <noreply@anthropic.com>
When the same required parameter showed up across several tools, search would lift it into a shared section to avoid repeating it. But lifting it lost the fact that it was required. The model saw the parameter listed once with no sign that a tool actually needed it, so it would leave it out and the call would fail. Now each tool remembers which parameters were required, and search reports from that memory instead of working it out again after the parameter has been moved. Lifting a parameter into the shared block can no longer erase that it was required. We also stopped merging a parameter into the shared block when tools disagree about whether it is required. If owner is required in some tools and optional in others, it stays on each tool, since one shared entry can only tell one of those stories. While we were here, the loader now rejects any tool or parameter with an empty description, instead of shipping a blank one that only surfaces later as a model failure. Co-Authored-By: claude-opus-4-7 <noreply@anthropic.com>
… search extraction Readability passes on a few helpers, no behavior change. A couple of functions in the YAML loader and the search code nested their real work inside a positive check. We flipped those to return early on the empty case, so the main path stays at the left margin instead of indented. One helper had two separate guards that did the same thing for a missing value and a null value. We merged them, so "nothing here means no parameters" reads as a single decision. Co-Authored-By: claude-opus-4-7 <noreply@anthropic.com>
Listing integrations or checking their health could hang forever. The check asked each integration in turn whether it was healthy, with no time limit on any one question. A single unreachable integration, like ollama on a host that never answers, would stall the whole thing behind it. We gave each check a two second budget and now run them all at once instead of one after another. Worst case, these tools take about one budget no matter how many integrations are down, instead of adding up every slow one. If the caller hangs up, the checks stop early. This is a stopgap. The real fix is to let an integration say why it is unhealthy, not just whether it is. Right now a timeout, a refused connection, and a bad token all look the same: unhealthy, no reason, go read the logs. Letting the health check return an error the way our other calls already do would put the real reason on the wire. We left that for later. We also wrote down a testing rule: a stale binary running an old build looks exactly like the new build, so any test against a live server has to confirm it is hitting the right version before trusting the result. Co-Authored-By: claude-opus-4-7 <noreply@anthropic.com>
…tools to YAML We moved every adapter's tool descriptions out of Go and into tools.yaml. Twelve adapters that landed on main after this branch started never made that move, so once the branch caught up to main, the build broke. Those twelve still declared their tools the old way. The Google Workspace suite (Calendar, Chat, Docs, Drive, Forms, Meet, People, Sheets, Slides, Tasks), Kubernetes, and Vercel each wrote their parameters as a Go map with a separate list of required names. The migration had already deleted the type that shape relied on. This commit moves all twelve to tools.yaml and the three-line loader, the same as every other adapter. We checked each one against its original Go definitions, tool for tool and field for field, before removing that check. A few existing adapters also grew tools on main while we were away. ClickHouse added connection listing, Cloudflare added a large batch (AI Gateway, Workers AI, Vectorize, Queues, Hyperdrive, Zero Trust, and more), and GitHub added runner registration tokens. The code to run those tools came across in the rebase, but their descriptions did not, so we add them back to each adapter's YAML. ClickHouse also needed one dispatch entry put back. Last, we register the new adapters in the wire-identity test and regenerate its fixture, so the test now covers the full set of tools the server ships. The fixture only gained entries. Nothing already there changed. Co-Authored-By: claude-opus-4-8 <noreply@anthropic.com>
acmacalister
approved these changes
Jun 26, 2026
acmacalister
left a comment
acmacalister
left a comment
Collaborator
There was a problem hiding this comment.
CI all green (build, test, lint, security). PR moves 1,400+ tool descriptions to YAML with byte-identity snapshot test and no Must Fix or Should Fix issues found. LGTM.
…date flag The byte-identity fixture was named for the migration that created it, tools_list_pre_migration.json, and lived under testdata like throwaway test scaffolding. It is neither anymore. It locks the exact tool list the server hands the model, the same idea as go.sum: a generated file, committed, that CI fails on when the real output drifts. We renamed it to tools_list.lock.json and moved it beside the server package it locks. We renamed the test to TestToolsList_MatchesWireLock and gave it an -update flag. Changing a description is now edit the YAML, run the test with -update, commit both. Without -update the test still fails on any drift, which is the point. An accidental change to what the model sees shows up in review instead of shipping quietly. CI already checks this. The race-test step runs the whole tree on every PR, so the lock is enforced without a dedicated step. Co-Authored-By: claude-opus-4-8 <noreply@anthropic.com>
acmacalister
approved these changes
Jun 26, 2026
acmacalister
left a comment
acmacalister
left a comment
Collaborator
There was a problem hiding this comment.
Clean PR. All CI checks passed (build, test, lint, security). The YAML migration is thorough, the wire lock enforces parity, and the architecture patterns are followed correctly. LGTM.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every integration's tool descriptions lived inline in Go, about 1,400 tools across the adapters. Editing the wording for one tool was a code change, reviewed next to dispatch maps, compaction specs, and everything else in the file. Prose was slow to write and the reviews were noisy.
This PR moves all of it into a
tools.yamlfile per adapter. Editing a description is now a content change you can review on its own, with no Go in the diff. The tool list the server hands to the model stays byte-for-byte the same as before.What was built
tools.yaml per adapter. Each adapter's tool code is now a few lines that load a YAML file. Descriptions, parameters, and required flags all live in the YAML. Every adapter with a fixed tool set moved over.
A stricter parameter type. Parameters and required-ness used to be two separate collections, cross-referenced by string name. A typo in the required list compiled fine and quietly changed the tool's contract. Now each parameter carries its own required flag, so a parameter that is required but never declared is no longer something you can write down.
Prose moved for the two meta-tools too. The long search and execute descriptions that teach the model how to discover and call tools moved into markdown files, same idea: code owns behavior, content owns the words.
A loud loader. A misspelled key fails at startup instead of quietly shipping a tool with a blank description.
The wire lock
The thing that makes this safe is a lock file on the tool list the server hands the model. It works like
go.sum. Thetools.yamlfiles are the source of truth, andserver/tools_list.lock.jsonis the generated, committed snapshot of the exact wire output they produce. A test,TestToolsList_MatchesWireLock, compares the live output against the lock byte-for-byte on every run, and CI runs it on every PR.That is what lets you trust this migration. Across all the adapters, the model sees the same tool list it saw before. Nothing was dropped or reworded by accident.
It also stays useful after the migration. The lock catches any future change to what the model sees, intended or not: a dropped description, a renamed parameter, a flipped required flag all turn the test red. When the change is intentional, you regenerate the lock and commit it with the YAML:
You don't hand-edit the lock, the same way you don't hand-edit
go.sum. An accidental change shows up as a lock diff in review instead of shipping quietly.Design decisions
We kept the wire output identical on purpose. Nothing downstream of the server had to change, and the lock holds us to it.
Adapters that build their tools at runtime from a remote server keep doing that. Only the ones with a fixed tool set moved to YAML.
Required-ness now lives on each parameter. That also fixed a real bug: when search lifts a shared parameter into a shared block, it no longer loses whether a given tool needed it.
Test plan
make build,go vet, and the fullgo test -racesuite green-updateregenerates it cleanlyNote
This branch was rebased onto the latest main, which had added twelve adapters since it diverged (the Google Workspace suite, Kubernetes, and Vercel). Those are migrated to YAML here too, so the whole adapter set lands on one shape. There is also a small unrelated fix in here for health probes hanging on an unreachable integration.
Co-Authored-By: claude-opus-4-7 noreply@anthropic.com