Skip to content

feat(registry): tag-based versioning with # ref syntax and CDN-bypass#151

Merged
jrob5756 merged 2 commits intomainfrom
feat/registry-tag-versioning
May 6, 2026
Merged

feat(registry): tag-based versioning with # ref syntax and CDN-bypass#151
jrob5756 merged 2 commits intomainfrom
feat/registry-tag-versioning

Conversation

@jrob5756
Copy link
Copy Markdown
Collaborator

@jrob5756 jrob5756 commented May 5, 2026

Summary

Reworks the workflow registry to behave more like a plugin system — versions are auto-discovered from git tags instead of being explicitly listed, refs can pin to any tag/branch/SHA, and stale CDN content is no longer a problem.

Closes the two pain points raised in discussion:

  1. `conductor registry update` doesn't reflect changes immediately — root cause was Fastly's CDN cache on `raw.githubusercontent.com`. Index fetches now resolve the ref to a commit SHA via the GitHub API and fetch from `…//index.yaml`. Unique URL per commit → CDN can't serve stale data. No `--force` flag needed.

  2. Versioning was awkward — required maintainers to enumerate every release in `index.yaml`. Now versions are auto-discovered from the registry repo's git tags, semver-sorted.

New ref grammar

```
[@][#]
```

Example Resolves to
`qa-bot` default registry, latest tag (or default branch HEAD if no tags)
`qa-bot#v1.2.3` default registry, ref `v1.2.3`
`qa-bot#main` default registry, branch `main`
`qa-bot@team` registry `team`, latest
`qa-bot@team#v1.2.3` registry `team`, tag `v1.2.3`
`qa-bot@#abc123` default registry, commit `abc123`

Hard errors: empty ref after `#`, multiple `#` or `@`, empty workflow name, path registry with a ref.

Behavior changes

  • `versions: [...]` field dropped from `index.yaml` schema. Existing indexes with this field still parse (Pydantic ignores it) — no migration required.
  • `latest` = newest semver-sorted tag (with leading `v` stripped for parsing). Falls back to default branch HEAD if no tags exist.
  • Cache key is commit SHA (`//<sha[:12]>/`). Mutable branch refs re-resolve fresh on every fetch; cache hit only when SHAs match.
  • Atomic cache writes: fetch into `tempfile.mkdtemp`, then `os.replace` into the final dir. No partial-state visibility for concurrent runs.
  • Index and workflow always come from the same commit (`load_index(entry, ref=sha)`) — fixes a subtle bug where `latest` could resolve to a tag that doesn't contain a workflow listed in the default-branch index.
  • `registry list` / `registry show` drop the per-workflow "Versions" column and add a "Latest tags:" footer (up to 5 newest, with `, ...` if more) for github registries.
  • `registry update` help text updated to explain the SHA-based CDN bypass.

Implementation

Module Change
`registry/github.py` Added `get_default_branch`, `resolve_ref_to_sha`; `list_tags` now paginates (cap 1000)
`registry/version_resolver.py` New module: `resolve_ref` (latest tag → default branch fallback) and `materialize_to_sha`
`registry/index.py` `load_index(entry, ref=None)` accepts an optional ref; github path always resolves to SHA before fetching
`registry/resolver.py` New `#`-based parser; `ResolvedRef.version` → `ResolvedRef.ref`
`registry/cache.py` Ref-aware, SHA-keyed, atomic fetch_workflow
`cli/registry.py` Drop Versions column, add Latest tags footer
`cli/app.py` Update 4 callers from `.version` → `.ref`

Tests

  • 153 registry tests pass (including 14 new tests in `test_version_resolver.py`).
  • Full suite: 2324 pass, 9 skipped, 0 regressions.
  • New coverage: tag pagination, semver sort with prereleases, mixed parseable/non-parseable tags, default-branch fallback, atomic-write failure cleanup, branch ref re-resolution, cache-hit skip, race-on-rename handling, path-registry-with-ref error.

Docs updated

  • `docs/design/registry.md` — full rewrite of Reference syntax, Versioning, and Caching sections; updated Implementation table.
  • `docs/cli-reference.md` — new syntax examples; `registry update`/`show`/`list` descriptions.
  • `README.md` — quick-start example uses `#ref` form.

Notes for reviewers

  • The `#` separator may need quoting in shells if surrounded by spaces (e.g. `conductor run 'qa-bot @ team # v1'`). Mid-token use (the typical case) is fine unquoted.
  • Branch refs are re-resolved to SHAs on every `fetch_workflow` call, which means an extra GitHub API roundtrip when running with a branch ref. Acceptable trade-off for correctness — explicit tag refs hit the cache directly.

jrob5756 and others added 2 commits May 5, 2026 19:14
- New ref grammar: <workflow>[@<registry>][#<ref>] (was @@Version)
- Auto-discover versions from git tags; semver-sort with v-prefix strip
- Fall back to default branch HEAD when no tags exist
- Any tag/branch/SHA can be pinned via #ref
- Drop versions: field from index.yaml (silently ignored for back-compat)
- Cache by commit SHA (sha[:12]); mutable refs re-resolve fresh
- Atomic cache writes (tempfile.mkdtemp + os.replace)
- Index loads always resolve ref->SHA, fetching at /<sha>/index.yaml
  to bypass raw.githubusercontent.com Fastly CDN
- registry list/show display 'Latest tags:' footer
- Path registries hard-error on non-empty refs
- Empty # and multiple @/# are hard errors

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Address quality issues from comprehensive PR review:

- Declare packaging>=21.0 explicitly in pyproject.toml; remove the
  silent ImportError fallback in version_resolver._sort_tags that
  could return commit-date-ordered tags as 'latest'.
- Promote _sort_tags -> sort_tags (was imported across module
  boundaries from cli/registry.py).
- Add RegistryNotFoundError(RegistryError) subclass; _raise_for_status
  raises it on HTTP 404. _load_github_index now catches only
  RegistryNotFoundError in the per-filename loop, so auth, rate-limit,
  and network errors propagate clearly instead of being collapsed into
  a misleading 'no index file found' message.
- _format_latest_tags now catches only RegistryError and httpx.HTTPError
  (was bare Exception); surfaces the actual error message in the
  '(unavailable: ...)' display.
- New prune_temp_dirs() helper cleans orphaned .tmp-* directories
  (created by the atomic-write pattern) on registry update.
- Tighten misleading docstrings: list_tags clarifies 'commit-date order'
  vs semver; fetch_workflow Raises documents silent sibling-fetch
  failures.

10 new tests covering the new error propagation, tmp-dir cleanup,
and surfaced suggestions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jrob5756 jrob5756 merged commit 33bc758 into main May 6, 2026
7 checks passed
@jrob5756 jrob5756 deleted the feat/registry-tag-versioning branch May 6, 2026 00:56
@jrob5756 jrob5756 mentioned this pull request May 6, 2026
jrob5756 added a commit that referenced this pull request May 6, 2026
* chore: release 0.1.12

Bumps version to 0.1.12 and updates CHANGELOG with the four PRs
merged since v0.1.11:

- #149: Windows install diagnostics
- #151: Tag-based registry versioning with # ref syntax
- #152: Unified reasoning.effort configuration
- #153: Dashboard layout fix for human_gate options + loop-backs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: add #155 (Windows update reliability) to 0.1.12 changelog

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant