feat(skills): native Anthropic Skills upload + sync (closes #35)

[teslashibe]

Closes #35.

Wires the template into Anthropic's Beta Skills API so any fork can drop a SKILL.md folder (e.g. `~/.claude/skills/astrology-skill/`) and have it uploaded, persisted, and attached to per-user agents.

Summary

• Backend (`internal/skills/`): one package, three files — `skills.go` (Service: List/UploadDir/UploadZip/SyncDirs/Delete + filesystem walker that preserves relative paths in the multipart upload), `zip.go` (single-skill zip extractor with path-traversal hardening), `handler.go` (Fiber CRUD).
• Migration `00005_skills.sql` — `skills(id, anthropic_skill_id, name UNIQUE, description, source, version, timestamps)`. UNIQUE(name) makes re-syncs idempotent (UPSERT updates the existing row's Anthropic ID + version).
• Provisioner: both `cmd/provision` (bootstrap, optional `SKILLS_AGENT_IDS` env) and `internal/agent/provision.go` (per-user) attach skill IDs. The agent toolset (bash + code-execution) is added only when at least one skill is configured — keeps the default permission surface unchanged for forks not using skills.
• CLI `cmd/skills-sync`: walks `SKILLS_SOURCES` (comma-separated dirs), uploads every immediate child containing `SKILL.md`. Folders without `SKILL.md` are skipped with a warning, not aborted.
• Mobile: Settings → Skills card → Skills screen with upload (zip via DocumentPicker), delete (with confirm), and sync. Empty state CTAs upload.
• Docs: `docs/SKILLS.md` covers SKILL.md format, folder layout, sandbox constraints (no network → bundle wheels), and the REST surface.

Constraints worth re-flagging in review

• Anthropic's API sandbox has no network and no `pip install`. Skills with native deps must bundle wheels (the astrology skill already does).
• Per-agent 20-skill cap — `AnthropicIDs` selects the oldest 20 by `created_at`. Documented in `docs/SKILLS.md`.
• Skills are workspace-wide (shared across everyone using the same `ANTHROPIC_API_KEY`); no per-team isolation in this PR.
• Per request: no tests in this PR. Listed under "Out of scope" in feat(skills): native Anthropic Skills upload + sync #35; will land in follow-up.

Test plan

• Backend builds (`go build ./...`).
• Mobile typechecks (`tsc --noEmit`).
• `make db-reset && make up` — confirm migration runs.
• Set `SKILLS_SOURCES=~/.claude/skills`, run `make skills-sync`, confirm `astrology-skill` shows up at `GET /api/skills/`.
• In the mobile UI, open Settings → Skills, confirm the list, hit delete, confirm row is gone from both Anthropic and the DB.
• Start a new chat, ask the agent to "use the astrology skill to draft a chart for someone born today" — verify Claude reads SKILL.md via bash.

Made with Cursor

[teslashibe]

Forensic Audit — Skills MVP vs #35

Per .cursor/rules/issue-audit-user-stories.mdc. Done after the implementation, before review.

1. Findings (severity-ordered, evidence first)

HIGH

H1. Re-uploading a changed skill orphans the old Anthropic skill.
backend/internal/skills/skills.go:149 calls Beta.Skills.New(...) every time UploadDir runs and then UPSERTs by name. When a skill with the same name already exists in our DB, the row's anthropic_skill_id is rotated to point at the new skill — but the old skill is never deleted from Anthropic. Over time, every re-sync orphans one skill per change. The SDK exposes Beta.Skills.Versions.New(skillID, ...) for the correct flow (push a new version onto the existing skill).
Fix: on UPSERT collision, fetch the existing anthropic_skill_id first; if present, call Versions.New instead of Skills.New. Fall back to Skills.New only when no prior row exists.

H2. Newly uploaded skills don't reach already-provisioned per-user agents.
internal/agent/provision.go:EnsureForUser caches the per-user anthropic_agent_id in users and createAgent (which reads SkillIDsFn) only runs on the FIRST call. After the first chat, uploading a skill via the UI has zero effect for that user — the cached agent has the old skill list forever. The SDK exposes Beta.Agents.Update(agentID, ...) with a Skills field (betaagent.go:1977 "Skills. Full replacement.") that solves this.
Fix: when Service.UploadDir/Delete succeed, broadcast a "skills changed" signal that triggers Agents.Update for every cached user. Cheaper alternative: add a skills_revision column to users, compare on each session create, and re-create-or-update if stale.

H3. Upload endpoint inherits Fiber's 4 MiB default body limit.
cmd/server/main.go:78 constructs Fiber without setting BodyLimit. Skills with bundled wheels (the astrology skill ships pyswisseph wheels — ~5 MiB) will 413 before even reaching the handler. AC2 ("uploads via API") will fail on the canonical example skill.
Fix: raise BodyLimit to ~64 MiB, or set BodyLimit per-route via Fiber's per-route config on /api/skills.

MEDIUM

M1. AnthropicIDs silently truncates at 20.
skills.go:84 selects LIMIT 20 to honour Anthropic's per-agent skill cap, but never logs when it truncates. A workspace with 25 skills will quietly attach the oldest 20 to every new agent and the operator gets no signal. Documented in docs/SKILLS.md but not surfaced at runtime.
Fix: when len(rows) == 20 and a SELECT count(*) returns more, log a warning with the truncated names.

M2. Brittle 404 detection on Anthropic delete.
skills.go:117 does strings.Contains(err.Error(), "404") to decide whether to swallow the Anthropic error and clean the local row. The SDK error type carries a structured status code (*anthropic.Error.StatusCode); string-matching the message is fragile against locale/i18n or SDK upgrades.
Fix: type-assert to *anthropic.Error and check StatusCode == 404.

M3. make skills-sync env loading.
Makefile:skills-sync does source backend/.env 2>/dev/null || true — but bash will still fail on lines with empty values like RESEND_API_KEY= if the user's env file has any quoting quirks. Reliable in zsh, brittle elsewhere.
Fix: rely on godotenv.Load() inside cmd/skills-sync/main.go (already present at L21) and drop the shell-level source from the Makefile.

LOW

L1. Sync handler logs nothing.
handler.go:94 passes a no-op log function, so server-side bulk syncs from the API leave no trail. The CLI logs to stdout. Inconsistent observability.

L2. Upload handler reads entire file into memory.
handler.go:53 does io.ReadAll(f) then passes the byte slice to UploadZip. For a 50 MiB skill bundle that's a transient ~150 MiB allocation (file → bytes → zip reader). archive/zip accepts an io.ReaderAt so we could write to a temp file and stream.
Trade-off: minor; "lean" beats "optimal" in MVP.

L3. Custom YAML frontmatter parser is reinventing a wheel.
skills.go:222 readSkillMD walks the --- delimiters by hand. gopkg.in/yaml.v3 was added to go.mod for the actual parse — goldmark-meta or even the yaml.v3 decoder against a 2-document stream would be standard.
Trade-off: acceptable; the hand-roll is 20 lines and works.

L4. expo-document-picker lacks a peer-version constraint.
mobile/package.json got expo-document-picker from npm install --silent which writes the latest. Expo SDK 55 wants a specific version — npx expo install would have respected that. Dev experience hazard, not a runtime bug.

2. Gaps vs issue intent

Issue intent	Status	Notes
make skills-sync from local paths	✅ done	cmd/skills-sync + Makefile target.
Re-sync is idempotent	⚠️ partial	UPSERT on name ✓; but H1 orphans the old Anthropic skill so it's idempotent in our DB only.
Folder w/o SKILL.md is skipped with warning	✅ done	Service.SyncDirs + os.Stat check.
Upload via multipart .zip	✅ done	handler.upload + unzipSingleSkill.
Missing SKILL.md → 400, no Anthropic call	✅ done	unzipSingleSkill errors before UploadDir runs.
Provisioner attaches code-execution + skill IDs	✅ done — for NEW agents	H2: existing per-user agents never get updated.
Settings → Skills lists name/description/source/delete	✅ done	mobile/app/(app)/skills.tsx.
Delete removes from Anthropic AND DB	✅ done	Best-effort, swallows 404 (see M2).
ANTHROPIC_API_KEY missing → CLI exits early	✅ done	cmd/skills-sync/main.go:23 log.Fatal.
Anthropic 4xx → no DB row + error surfaced	✅ done	UploadDir errors before INSERT.

The two real gaps are H1 and H2. Everything else either passed or has a low-severity caveat documented above.

3. User stories (audited)

• As a template forker, I want to point SKILLS_SOURCES at a directory of skill folders and run make skills-sync, so that all my custom skills are uploaded to Anthropic. — MET (with H1 caveat for re-runs).
• As an end-user, I want to see which Skills my agent has installed and remove ones I don't need. — MET.
• As an end-user, I want to upload a new Skill folder/zip from the UI, so that I can add capabilities without redeploying. — PARTIALLY MET; H3 will reject any skill with bundled wheels until BodyLimit is raised, and H2 means even successful uploads don't reach my existing agent until I sign out and back in.
• As a backend operator, I want skill IDs persisted in our DB so the UI doesn't round-trip Anthropic on every page load. — MET.

4. Acceptance criteria (re-stated, testable)

Sync from filesystem

• AC-1.1 Given SKILLS_SOURCES=<dir>; when make skills-sync runs; then every <dir>/*/SKILL.md is uploaded and GET /api/skills/ returns those rows.
• [⚠️] AC-1.2 Given a skill with the same name already exists; when re-running; then the local row UPDATES in place. Pass for our DB; FAIL on Anthropic side per H1 — old skill not cleaned up.
• AC-1.3 Given a folder lacks SKILL.md; when sync walks it; then it's skipped with a logged warning and the run continues.

Upload via API

• [⚠️] AC-2.1 Given an authenticated user; when they POST a multipart <5 MiB .zip; then the response is 201 with the new skill JSON. For >4 MiB zips → fails per H3.
• AC-2.2 Given the zip lacks SKILL.md at root; then the server returns 400 and does NOT call Anthropic.
• AC-2.3 Given the zip has more than one top-level folder; then the server returns 400 with "exactly one top-level folder".
• AC-2.4 Given a path-traversal entry (e.g. ../etc/passwd); then the server returns 400 with "escapes root".

Provisioner attaches skills

• AC-3.1 Given ≥1 skill in DB; when a NEW user creates their first session; then the Anthropic Agent has the agent toolset (bash + code-exec) AND those skill IDs.
• [❌] AC-3.2 Given a user has already chatted (cached agent_id); when a skill is uploaded; then the user's NEXT session uses an agent that knows about the new skill. FAIL per H2 — cached agent is never refreshed.

Mobile UI

• AC-4.1 Given I open Settings → Skills; then I see name, description, source, and a delete affordance per row.
• AC-4.2 Given I tap delete and confirm; then the row is removed from Anthropic + DB and disappears from the list.

Negative paths

• AC-5.1 Given ANTHROPIC_API_KEY=""; when make skills-sync runs; then exit code is non-zero with a clear log line and no Anthropic calls were made.
• AC-5.2 Given Anthropic returns 4xx on upload; then no DB row is written and the caller sees the error.

5. Risks and follow-up actions

Risk	Likelihood	Mitigation
Anthropic Skills API beta header changes	Med	SDK upgrade absorbs it; pinned at v1.37.0 in go.mod.
Skill bundle exceeds Fiber body limit (H3)	High for canonical astrology skill	Raise BodyLimit; ship as part of this PR or a follow-up before users hit it.
Orphaned Anthropic skills on re-sync (H1)	High after first re-sync	Implement Versions.New flow before announcing the feature externally.
Stale per-user agents miss new skills (H2)	High after first upload	Add Agents.Update call from Service.UploadDir.
Path-traversal regression	Low (covered)	unzipSingleSkill guards; consider a regression test in the follow-up tests PR.

Recommended follow-up tickets

1. fix(skills): use Versions.New on re-upload — H1.
2. fix(skills): refresh cached per-user agents on skill change — H2.
3. fix(skills): raise Fiber BodyLimit for /api/skills uploads — H3.
4. chore(skills): structured 404 detection on Anthropic delete — M2.
5. chore(skills): log truncation when AnthropicIDs hits the 20-cap — M1.
6. test(skills): table-driven coverage — explicitly deferred per spec; opens after the H-level fixes land.

---

Bottom line: the MVP meets 8/10 ACs as written. The two failing ACs (1.2 partial, 2.1 and 3.2) are real and high-severity but each is a targeted ~20-line fix using SDK methods we already pull in. None of them require redesign — they're bugs introduced by writing the simplest version first.

[teslashibe]

Live Test Results — All Audit Findings Resolved

Tested every API + Web route on the live stack. All audit findings (H1/H2/H3 + M1/M2/M3) plus three additional Anthropic API constraints surfaced during testing are now fixed and verified.

Audit fixes

Finding	Before	After
H1 display_title uniqueness on re-upload	Re-running make skills-sync returned 400 `Skill cannot reuse an existing display_title`	UploadDir looks up the prior anthropic_skill_id and uses Beta.Skills.Versions.New; first uploads still use Beta.Skills.New. Verified: same skill ID, new version on each re-sync.
H2 cached per-user agents missed new skills	Uploading a skill only reached new users; existing users had a stale cached anthropic_agent_id	refreshAgentsAsync fires after every upload/delete, enumerates users.anthropic_agent_id, pushes the new list via Beta.Agents.Update. Fire-and-forget; failures logged.
H3 Fiber 4 MiB body limit	Skill bundles >4 MiB returned 413 before reaching the handler	BodyLimit: 64 * 1024 * 1024. Verified with a 6 MiB upload — HTTP 201.
M1 silent 20-skill truncation	No log when AnthropicIDs capped at 20	Logs the names of dropped skills with the cap value.
M2 brittle strings.Contains(err.Error(), "404")	Fragile across SDK upgrades / locales	New is404 helper uses errors.As(*anthropic.Error{}) then checks StatusCode == 404. Used in both Delete and deleteAllVersions.
M3 brittle source backend/.env in Makefile	Failed on env files with quoting quirks	Dropped — CLI uses godotenv.Load() already; Makefile target is one line now.

Additional Anthropic API constraints surfaced (live testing)

Constraint	Symptom	Fix
Cannot delete skill with existing versions	DELETE on any re-uploaded skill returned 500	deleteAllVersions enumerates via Versions.ListAutoPaging and deletes each before calling Skills.Delete.
SDK helper BetaManagedAgentsSkillParamsOfCustom omits the required Type field	Beta.Agents.New returned 400 `skills[0].type: Field required`	Wrapped in skills.SkillParams that sets Type: BetaManagedAgentsCustomSkillParamsTypeCustom. Provisioner uses it.
Anthropic cloud rejects loopback MCP URLs (localhost/127.0.0.1/::1)	Local-dev session creation hung 750ms then 500'd with a confusing 400 from Anthropic	createAgent now fails fast with: MCP_PUBLIC_URL/APP_URL resolves to loopback (localhost); for local dev, expose via ngrok or cloudflared and set MCP_PUBLIC_URL to the public URL
Anthropic API sandbox rejects bundled .whl/.zip/.tar/.tgz/.gz	"Skill cannot contain nested zip files"	openSkillFiles skips these extensions silently. (Already in earlier commit.)
Symlinked source dirs (~/.claude/skills/foo) silently skipped	e.IsDir() returns false for symlinks	Switched to os.Stat + filepath.EvalSymlinks before walking. (Already in earlier commit.)

Live test results

=== AUTH ===
  POST /auth/login → JWT (201 chars)

=== API READ ROUTES (auth) ===
  200  /api/me                         (195 bytes)
  200  /api/teams/                     (303 bytes)
  200  /api/skills/                    (633 bytes)
  200  /api/agent/sessions             (346 bytes)
  200  /api/platforms                  (634 bytes)

=== UNAUTH GUARDS (expect 401) ===
  401  /api/me
  401  /api/skills/
  401  /api/agent/sessions
  401  /api/teams/

=== SKILL UPLOAD VALIDATION (expect 400) ===
  400  empty POST          → "missing 'file' upload (zip of the skill folder)"
  400  notazip.txt         → "upload must be a .zip"
  400  empty-skill.zip     → "zip root empty-skill does not contain SKILL.md"

=== EXPO WEB ROUTES ===
  200  /  /chat  /settings  /skills  /platforms  /teams  /capture

Skills-specific E2E

• Sync ~/.claude/skills/astrology-skill/ (symlinked to ~/teslashibe/astrology-skill) → uploaded on first run, Versions.New on re-run (same skill_01... ID, new version timestamp). ✓
• Multipart upload of a 3.5 KiB skill zip → HTTP 201, UPSERT updates existing row. ✓
• 6 MiB body → HTTP 201 (would have been 413 before H3 fix). ✓
• Delete a versioned skill → HTTP 204 (was 500 with "Cannot delete skill with existing versions"). ✓
• Direct MCP tool call astrology_birth_summary({date: 1990-08-15, time: 14:30, timezone: America/New_York, place: NYC}) → Sun in Leo · Fire · Fixed · Sun-ruled. ✓ Engine works end-to-end.

What was NOT tested live (with reason)

• Agent → MCP tool round-trip from a real chat: Anthropic's cloud cannot reach localhost. To test this end-to-end locally, run ngrok http 8080 and set MCP_PUBLIC_URL=https://<id>.ngrok.app before bringing up the stack. The fail-fast hint added in this commit makes the constraint visible immediately instead of mid-session. In production this is a non-issue.

Files changed in this push

backend/cmd/server/main.go                   |   8 +- (BodyLimit 64MiB)
backend/cmd/skills-sync/main.go              |   8 ++ (drain refresh goroutine)
backend/internal/agent/provision.go          |  20 ++ (loopback guard, SkillParams)
backend/internal/mcp/platforms/astrology.go  |   7 +  (NoCredentials: true)
backend/internal/skills/skills.go            | 196 +++ (Versions.New + delete cascade + refreshAgentsAsync + is404 + SkillParams + truncation log)
Makefile                                     |   3 +- (drop bash source)