Skip to content

feat: add agent-skills discovery release pipeline#66

Closed
gregnr wants to merge 2 commits into
mainfrom
feat/agent-skills-discovery
Closed

feat: add agent-skills discovery release pipeline#66
gregnr wants to merge 2 commits into
mainfrom
feat/agent-skills-discovery

Conversation

@gregnr
Copy link
Copy Markdown
Member

@gregnr gregnr commented Apr 14, 2026

Implements part 1 of Cloudflare's agent-skills discovery RFC - builds per-skill archives and an index.json that are meant to live in /.well-known/agent-skills/.... Part 2 is getting our www site to pull these release artifacts at build time and serve them from https://supabase.com/.well-known/agent-skills/....

Artifacts live in GitHub releases within this repo. Releases will auto-generate upon any change to ./skills in main. Since each skill has their own independent version, these bundled releases just use the current date+commit hash as the tag.

GitHub Release (e.g. 2026-04-14-a3f9c2)
├── index.json               # discovery index per RFC v0.2.0
├── supabase.tar.gz
│   ├── SKILL.md
│   └── references/
└── supabase-postgres-best-practices.tar.gz
    ├── SKILL.md
    └── references/

@Rodriguespn
Copy link
Copy Markdown
Collaborator

Rodriguespn commented Apr 15, 2026

Thanks for taking care of this @gregnr! I went through the PR and also compared our build-release.ts with jonathanhefner/agentskills-build-for-well-known — the GitHub Action Jonathan built for this (referenced in agentskills/agentskills#254). The Action isn't published yet (no releases/tags, hasn't been transferred to the agentskills org), so we can't use it directly. But it has a few improvements worth pulling into our script so we're closer to spec and aligned for when it ships.

Suggested improvements to build-release.ts

1. Deterministic archives (source)

Our current tar czf includes timestamps, uid/gid, and gzip header timestamps — so identical skill content produces different hashes on each build. The Action fixes this with a two-step process that zeroes all metadata:

// Two-step: uncompressed tar with zeroed metadata, then gzip without header timestamp
await execFileAsync("tar", [
  "--mtime=@0",
  "--owner=0",
  "--group=0",
  "--numeric-owner",
  "--no-recursion",
  "-cf",
  tarPath,
  "-C",
  skillDir,
  ...sortedEntries,
]);

// gzip -n: omit original name and timestamp from header
// gzip -f: force overwrite, replaces tarPath with tarPath.gz
await execFileAsync("gzip", ["-nf", tarPath]);

This means digests only change when content actually changes, which is the whole point of the digest field for client-side caching.

2. File listing with exclude patterns (walker, exclude matching)

Our archives include everything in the skill directory. The Action walks the dir, applies .gitignore-style excludes, and returns sorted paths. It also collects parent directories (here) since tar with --no-recursion needs explicit dir entries:

// Patterns without "/" match basenames at any depth; with "/" match relative paths
function matchesExclude(relPath: string, patterns: string[]): boolean {
  for (const pattern of patterns) {
    const target = pattern.includes("/") ? relPath : relPath.split("/").pop()!;
    if (matchesGlob(target, pattern)) return true;
  }
  return false;
}

// Default excludes
const DEFAULT_EXCLUDES = [".git", ".gitignore", ".gitattributes", ".DS_Store",
  "Thumbs.db", ".vscode", ".idea", "*.swp", ".*.swp", "*~", ".*~"];

Not a problem today since our skill dirs are clean, but good hygiene.

3. Auto-detect skill-md vs archive type (source)

We currently hardcode type: "archive" for everything. Both our skills today have references/, so it's correct, but per the spec single-file skills should use type: "skill-md". The Action checks after listing files:

if (isSingleFile) {
  // Single-file skill: copy SKILL.md directly
  type = "skill-md";
  const destDir = join(outputDir, skill.name);
  await mkdir(destDir, { recursive: true });
  artifactPath = join(destDir, "SKILL.md");
  await copyFile(skillMdPath, artifactPath);
  url = `${skill.name}/SKILL.md`;
} else {
  // Multi-file skill: create archive
  type = "archive";
  artifactPath = join(outputDir, `${skill.name}.tar.gz`);
  url = `${skill.name}.tar.gz`;
  await createTarGz(skill.directory, files, artifactPath);
}

4. Relative URLs instead of absolute (skill-md, archive)

We emit /.well-known/agent-skills/supabase.tar.gz (path-absolute). The Action emits supabase.tar.gz (relative). Both are valid per RFC 3986 Section 5, but relative is more portable — the consumer (supabase/supabase#44878) controls the serving path.

5. Validate both name and description in frontmatter (source)

We only validate description. The Action validates both fields exist and are strings:

function parseFrontmatter(content: string, filePath: string): SkillFrontmatter {
  const { data } = matter(content);

  if (!data.name || typeof data.name !== "string") {
    throw new Error(`Missing or invalid 'name' in frontmatter: ${filePath}`);
  }
  if (!data.description || typeof data.description !== "string") {
    throw new Error(`Missing or invalid 'description' in frontmatter: ${filePath}`);
  }

  return { name: data.name, description: data.description };
}

What's good as-is

  • The release workflow and {DATE}-{SHA} tagging — the Action doesn't handle releases, so our workflow stays either way.
  • The overall approach of building artifacts in this repo and consuming them from www at build time is solid and matches how the RFC expects things to work.

Happy to pick up any of these changes if you want!

Rodriguespn added a commit that referenced this pull request May 6, 2026
Replaces the inline bash tar loop with a TypeScript build script that
produces both skill tarballs and a discovery index.json conforming to
the agent-skills discovery spec (RFC v0.2.0). Applies improvements
identified in code review of #66:

- Deterministic archives: zero timestamps/uid/gid so digests only change
  when content changes (requires GNU tar; falls back to BSD tar with a
  warning on macOS)
- Exclude patterns: .gitignore-style walk skips .DS_Store, .vscode, etc.
- Auto-detect skill-md vs archive: single-file skills emit type "skill-md"
  and copy SKILL.md directly instead of creating a tarball
- Relative URLs in index.json: portable across serving paths
- Validate both name and description frontmatter fields

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@gregnr gregnr closed this May 9, 2026
@gregnr
Copy link
Copy Markdown
Member Author

gregnr commented May 9, 2026

Closing in favor of #77.

Rodriguespn added a commit that referenced this pull request May 11, 2026
…ne (#77)

* feat: build index.json for .well-known/agent-skills in release pipeline

Replaces the inline bash tar loop with a TypeScript build script that
produces both skill tarballs and a discovery index.json conforming to
the agent-skills discovery spec (RFC v0.2.0). Applies improvements
identified in code review of #66:

- Deterministic archives: zero timestamps/uid/gid so digests only change
  when content changes (requires GNU tar; falls back to BSD tar with a
  warning on macOS)
- Exclude patterns: .gitignore-style walk skips .DS_Store, .vscode, etc.
- Auto-detect skill-md vs archive: single-file skills emit type "skill-md"
  and copy SKILL.md directly instead of creating a tarball
- Relative URLs in index.json: portable across serving paths
- Validate both name and description frontmatter fields

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* docs: add .well-known discovery section and GitHub Action migration note to README

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* feat: simplify release build script using tar npm library

Replace shell-out to system tar/gzip binaries with the tar npm package.
Drops the custom file-walker, exclude-pattern matching, GNU tar detection,
and BSD tar fallback — none of which are needed in a clean CI checkout.
Determinism is preserved via portable: true and mtime: new Date(0).
Always emits type: "archive" for all skills, removing the isSingleFile
branch since the spec recommendation is not a hard requirement.
Renames release-please.yml to release.yml.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Rodriguespn added a commit to supabase/supabase that referenced this pull request May 15, 2026
…45641)

## Summary

This PR makes `fetchAgentSkills.mjs` a spec-compliant client of the
[agent-skills `.well-known` URI
spec](agentskills/agentskills#254), and updates
the script to match the current release structure in
[`supabase/agent-skills`](https://github.com/supabase/agent-skills).

---

## 1. Spec-compliant URL resolution and digest verification

`fetchAgentSkills.mjs` acts as a client consuming the `.well-known`
discovery index. The [agent-skills `.well-known`
spec](agentskills/agentskills#254) is explicit
on two points:

**URL resolution** — skill artifact URLs in `index.json` must be
resolved per [RFC 3986
§5.2.2](https://datatracker.ietf.org/doc/html/rfc3986#section-5.2.2)
using the index URL as the base URI:

> "The `url` field specifies where to fetch the skill artifact. URLs are
resolved per RFC 3986 Section 5 using the index URL as the base URI."

This means `skill.url` can be relative (`supabase.tar.gz`),
path-absolute (`/.well-known/agent-skills/supabase.tar.gz`), or fully
absolute (e.g. a CDN URL like
`https://cdn.example.com/supabase.tar.gz`). The previous implementation
extracted a filename with `.split('/').pop()` which happened to work for
bare relative URLs but was not doing RFC 3986 resolution.

**Digest verification** — clients must verify artifact integrity before
use:

> "Clients **must** verify downloaded content against the `digest` in
the index. A mismatch indicates the content is corrupted or tampered
with — clients **must not** use unverified content."

The updated script uses `new URL(skill.url, githubReleaseIndexUrl)` for
compliant resolution, verifies each artifact's SHA-256 digest from the
in-memory buffer before any disk writes, and only writes to
`public/.well-known/agent-skills/` once all digests pass.

**Acknowledged overhead**: since Supabase owns both the publisher
([`scripts/build-release.ts`](https://github.com/supabase/agent-skills/blob/main/scripts/build-release.ts)
in `supabase/agent-skills`) and this consumer, the practical risk of
non-compliant URL handling is currently low — the publisher always emits
bare relative filenames. However, being spec-compliant here gives us
full flexibility to change how skills are packaged or hosted in
`supabase/agent-skills` in the future (e.g. moving artifacts to a CDN)
without needing to update this script.

---

## 2. Semver release tags

#44878 referenced `supabase/agent-skills#66` (date+SHA tags).
[supabase/agent-skills#77](supabase/agent-skills#77)
has since merged, moving releases to semver tags managed by Release
Please. `/releases/latest` works for both formats — no code change
needed, just a rebase.

---------

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants