-
Notifications
You must be signed in to change notification settings - Fork 93
docs: surface MCP and agent skills for coding agents #3093
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,130 @@ | ||
| --- | ||
| sidebar_position: 3 | ||
| --- | ||
|
|
||
| # Agent Skills | ||
|
|
||
| Set up and operate Estuary pipelines through your AI assistant. Agent skills are step-by-step playbooks that give your assistant everything it needs to run the right commands, build the right specs, and explain the gotchas — no docs to stitch together, no flowctl commands to memorize. | ||
|
|
||
| Ask your assistant to "capture my Postgres into Snowflake" or "why is this materialization failing?", and the relevant skill guides it through the work. | ||
|
|
||
| Skills are distributed from the open-source [`estuary/agent-skills`](https://github.com/estuary/agent-skills) repository and work with [Claude Code, GitHub Copilot, Cursor, OpenAI Codex, Gemini CLI, and 30+ other tools](https://agentskills.io) via the open [SKILL.md](https://agentskills.io) standard. | ||
|
|
||
| :::tip | ||
| Pair skills with the [MCP integration](./mcp-integration.md) so your assistant's explanations stay grounded in current Estuary documentation while the skills drive the work. See [Using coding agents with Estuary](./using-coding-agents.md) for the full setup. | ||
| ::: | ||
|
|
||
| ## What's included | ||
|
|
||
| ### Captures (sources) | ||
|
|
||
| Capture data from databases, APIs, and webhooks into Estuary collections. | ||
|
|
||
| | Skill | Description | | ||
| |-------|-------------| | ||
| | `capture-postgres-create` | PostgreSQL CDC (vanilla, RDS, Aurora, Cloud SQL, Supabase, Neon) | | ||
| | `capture-mysql-create` | MySQL CDC via binlog replication (RDS, Aurora, Cloud SQL, Azure) | | ||
| | `capture-mongodb-create` | MongoDB CDC (Atlas, DocumentDB, self-hosted) | | ||
| | `capture-sqlserver-create` | SQL Server CDC (RDS, Azure SQL, Cloud SQL) | | ||
| | `capture-http-ingest-create` | HTTP webhook capture (GitHub, Shopify, Stripe, or any JSON source) | | ||
| | `capture-generic-create` | Any of 148+ source connectors via dynamic schema discovery | | ||
|
|
||
| ### Materializations (destinations) | ||
|
|
||
| Stream Estuary collections into downstream databases and warehouses. | ||
|
|
||
| | Skill | Description | | ||
| |-------|-------------| | ||
| | `materialize-postgres-create` | PostgreSQL destination | | ||
| | `materialize-snowflake-create` | Snowflake destination (JWT auth) | | ||
| | `materialize-bigquery-create` | BigQuery destination (GCS staging) | | ||
| | `materialize-redshift-create` | Amazon Redshift destination (S3 staging) | | ||
| | `materialize-databricks-create` | Databricks destination (Unity Catalog) | | ||
| | `materialize-generic-create` | Any destination connector via dynamic schema discovery | | ||
|
|
||
| ### Operations | ||
|
|
||
| Manage and troubleshoot running pipelines. | ||
|
|
||
| | Skill | Description | | ||
| |-------|-------------| | ||
| | `estuary-flowctl-setup` | Install, authenticate, and update the flowctl CLI | | ||
| | `estuary-task-health` | End-to-end health check for a task: status, data flow, errors, and history | | ||
| | `estuary-catalog-status` | Check control-plane status of a task with `flowctl catalog status` | | ||
| | `estuary-task-stats` | Inspect data volume, document counts, and hourly throughput for a task | | ||
| | `estuary-catalog-history` | Review publication history and recent spec changes | | ||
| | `estuary-logs` | Search and analyze task logs with jq filtering | | ||
| | `estuary-connector-restart` | Pause and restart connectors via shard management | | ||
| | `estuary-ssh-tunnels` | Diagnose and fix SSH tunnel connection issues | | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - An [Estuary account](https://dashboard.estuary.dev/register) | ||
| - The [flowctl](../guides/get-started-with-flowctl.md) CLI — the `estuary-flowctl-setup` skill walks you through installation and authentication | ||
|
|
||
| ## Installation | ||
|
|
||
| ### Skills CLI | ||
|
|
||
| Install all skills at once, in any supported tool: | ||
|
|
||
| ```bash | ||
| npx skills add estuary/agent-skills | ||
| ``` | ||
|
|
||
| ### Claude Code | ||
|
|
||
| Add the Estuary marketplace: | ||
|
|
||
| ```bash | ||
| /plugin marketplace add estuary/agent-skills | ||
| ``` | ||
|
|
||
| Then install by group: | ||
|
|
||
| ```bash | ||
| /plugin install estuary-captures@estuary | ||
| /plugin install estuary-materializations@estuary | ||
| /plugin install estuary-operations@estuary | ||
| ``` | ||
|
|
||
| Or run `/plugin` to browse from the Discover tab. Installed skills auto-update when the marketplace refreshes. | ||
|
|
||
| ### Manual installation | ||
|
|
||
| Clone the repository and copy the skill folders into your AI tool's skill directory: | ||
|
|
||
| ```bash | ||
| git clone https://github.com/estuary/agent-skills.git | ||
| cp -r agent-skills/skills/* your-project/.claude/skills/ | ||
| ``` | ||
|
|
||
| Common paths by tool: | ||
|
|
||
| | Tool | Path | | ||
| |------|------| | ||
| | Claude Code | `.claude/skills/` | | ||
| | Cursor | `.cursor/skills/` | | ||
| | GitHub Copilot / VS Code | `.github/skills/` | | ||
| | OpenCode | `.opencode/skills/` | | ||
| | Codex | `.codex/skills/` | | ||
|
|
||
| ## Usage | ||
|
|
||
| Once installed, ask your AI assistant in plain English: | ||
|
|
||
| > "Capture my Postgres database into Estuary." | ||
| > | ||
| > "Materialize my collections into Snowflake." | ||
| > | ||
| > "Capture from MySQL and materialize into Redshift." | ||
| > | ||
| > "Why is my materialization failing?" | ||
|
|
||
| Your assistant picks the matching skill and walks through the commands, spec, and common pitfalls. | ||
|
|
||
| ## Related pages | ||
|
|
||
| - [Using coding agents with Estuary](./using-coding-agents.md) — end-to-end setup for flowctl, MCP, and skills | ||
| - [MCP integration](./mcp-integration.md) — connect Estuary's documentation to your assistant | ||
| - [flowctl setup](../guides/get-started-with-flowctl.md) — install and authenticate the CLI | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| --- | ||
| sidebar_position: 2 | ||
| sidebar_position: 5 | ||
| slug: /reference/Configuring-task-shards/ | ||
| --- | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| --- | ||
| sidebar_position: 1 | ||
| sidebar_position: 4 | ||
| slug: /reference/time-travel/ | ||
| --- | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| --- | ||
| sidebar_position: 1 | ||
| --- | ||
|
|
||
| # Using Coding Agents with Estuary | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this page useful as-is? It mostly seems to reiterate information from other places in docs. |
||
|
|
||
| Build and operate Estuary pipelines from your AI coding assistant — Claude Code, Cursor, GitHub Copilot, OpenAI Codex, Gemini CLI, and others. Instead of stitching together docs and remembering flowctl commands, you can ask in plain English: "capture my Postgres into Snowflake" or "why is this materialization failing?" | ||
|
|
||
| Three pieces work together: | ||
|
|
||
| 1. **[flowctl](../guides/get-started-with-flowctl.md)** — Estuary's CLI. This is what your agent actually runs to discover sources, build specs, publish, and inspect tasks. | ||
| 2. **[MCP integration](./mcp-integration.md)** — connects your assistant to Estuary's documentation so its answers are backed by current docs, without leaving your editor. | ||
| 3. **[Agent skills](./agent-skills.md)** — step-by-step playbooks that tell your assistant the exact commands, spec shapes, and gotchas for each connector and operation. | ||
|
|
||
| You can use any one of these on its own, but they're most useful together: skills drive flowctl to do the work, and MCP keeps the assistant's explanations accurate. | ||
|
|
||
| ## Setup | ||
|
|
||
| ### 1. Install and authenticate flowctl | ||
|
|
||
| flowctl is the CLI your agent drives. Install it: | ||
|
|
||
| ```bash | ||
| # Mac (Homebrew) | ||
| brew tap estuary/flowctl | ||
| brew install flowctl | ||
| ``` | ||
|
|
||
| For Linux, direct download, or Windows (WSL), see the [flowctl setup guide](../guides/get-started-with-flowctl.md). | ||
|
|
||
| Then authenticate: | ||
|
|
||
| ```bash | ||
| flowctl auth login | ||
| ``` | ||
|
|
||
| This opens the Estuary dashboard's CLI-API tab. Copy the access token and paste it back into the terminal. For CI/CD and other non-interactive environments, use a refresh token via `FLOW_AUTH_TOKEN` — see the [flowctl setup guide](../guides/get-started-with-flowctl.md). | ||
|
|
||
| ### 2. Connect the MCP server | ||
|
|
||
| Point your assistant at Estuary's MCP server so it can answer documentation questions in context: | ||
|
|
||
| ```bash | ||
| # Claude Code | ||
| claude mcp add --transport http estuary https://estuary.mcp.kapa.ai | ||
| ``` | ||
|
|
||
| Setup differs per tool (Cursor, VS Code, ChatGPT Desktop, Claude Desktop, and others). See [MCP integration](./mcp-integration.md) for the full instructions and authentication steps. | ||
|
|
||
| ### 3. Install the agent skills | ||
|
|
||
| Add the [Estuary agent skills](./agent-skills.md) so your assistant knows the right commands and spec shapes for each connector: | ||
|
|
||
| ```bash | ||
| # Skills CLI (works across tools) | ||
| npx skills add estuary/agent-skills | ||
| ``` | ||
|
|
||
| ```bash | ||
| # Claude Code (plugin marketplace) | ||
| /plugin marketplace add estuary/agent-skills | ||
| ``` | ||
|
|
||
| See [Agent skills](./agent-skills.md) for the Claude Code plugin groups, manual installation, and per-tool skill directories. | ||
|
|
||
| ## What you can do | ||
|
|
||
| Once set up, ask your assistant in plain English. For example: | ||
|
|
||
| - "Capture my PostgreSQL database into Estuary." | ||
| - "Materialize my collections into Snowflake." | ||
| - "Capture from MySQL and materialize into Redshift." | ||
| - "Why is my materialization failing?" | ||
| - "What's the difference between captures and materializations?" | ||
|
|
||
| The skills tell your assistant which flowctl commands to run and how to build the specs; the MCP integration keeps its explanations grounded in Estuary's documentation. | ||
|
|
||
| ## Next steps | ||
|
|
||
| - [flowctl setup](../guides/get-started-with-flowctl.md) — install, authenticate, and update the CLI | ||
| - [MCP integration](./mcp-integration.md) — connect Estuary's docs to your assistant | ||
| - [Agent skills](./agent-skills.md) — the full catalog of capture, materialization, and operations skills | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| --- | ||
| sidebar_position: 3 | ||
| sidebar_position: 6 | ||
| slug: /reference/working-logs-stats/ | ||
| --- | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -64,7 +64,17 @@ Once you've installed flowctl and are ready to begin working, authenticate your | |
|
|
||
| The token will expire after a predetermined duration. Repeat this process to re-authenticate. | ||
|
|
||
| ## Drive flowctl from an AI coding agent | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This section sounds a little "marketing"-y. My suggestion would be to tone down the hype and simply present it as an alternate option. Something like: |
||
|
|
||
| You don't have to run flowctl commands by hand. AI assistants like Claude Code, Cursor, and GitHub Copilot can build and operate your pipelines for you: | ||
|
|
||
| - **[Agent skills](../features/agent-skills.md)** give your assistant step-by-step playbooks for each connector and operation, so it knows the exact flowctl commands and spec shapes to use. Ask it to "capture my Postgres into Snowflake" and it does the work. | ||
| - **[MCP integration](../features/mcp-integration.md)** connects your assistant to Estuary's documentation, so its answers stay accurate while you code. | ||
|
|
||
| See [Using coding agents with Estuary](../features/using-coding-agents.md) for the full setup. | ||
|
|
||
| # Next steps | ||
|
|
||
| 1. [flowctl concepts](../concepts/flowctl.md): Learn more about using flowctl. | ||
| 2. [User guides](../guides/flowctl/README.md): Check out some of the detailed user guides to see flowctl in action. | ||
| 3. [Using coding agents with Estuary](../features/using-coding-agents.md): Let an AI assistant build and operate your pipelines. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is already an Agent Skills page here: https://docs.estuary.dev/guides/agent-skills/
It's what the dashboard "Agent Skills" button links to.
See also conversation from #2961 for additional notes. Re: yes, we could bundle the MCP integration and agent skills pages under one subsection to make the related pages easier to find. Maybe we could keep it under
/featuresfor now./guidesis getting a little unwieldy and I'm not convinced we'd have enough content for a top-level section.