LCR-85: Add multi-harness skill system with --tools flags by aboucaud · Pull Request #76 · LightconeResearch/lightcone-cli

aboucaud · 2026-04-22T17:23:18Z

Summary

Extract Claude-Code-specific skill installation into a harness-agnostic system
New src/lightcone/cli/harness.py registry maps tool IDs to per-tool install paths
Add --tools flag to lc init and lc update --sync
Skills, agents, and guides distributed to all selected harnesses
Hooks and settings remain Claude Code only
21 unit tests + 11 CLI integration tests

Test plan

uv run pytest tests/test_harness.py tests/test_harness_install.py -v — 32/32 passed
uv run ruff check src/ tests/ — clean
uv run mypy src/ — no new errors
Manual: lc init --tools claude --tools codex . and verify both .claude/ and .codex/ populated
Manual: lc update --tools codex and verify only .codex/ affected

🤖 Generated with Claude Code

Extract Claude-Code-specific skill installation into a harness-agnostic system. New harness registry maps tool IDs (Claude Code, Codex, Cursor, GitHub Copilot, OpenCode) to per-tool install paths. Skills, agents, and guides are copied to all selected harnesses; hooks and settings remain Claude Code only. - Add src/lightcone/cli/harness.py (registry module) - Add --tools flag to lc init and lc update --sync - Refactor _install_harnesses to loop over selected harnesses - Add _display_install_summary with post-install output - Add tests/test_harness.py (unit tests for registry) - Add tests/test_harness_install.py (CLI integration tests) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

linear · 2026-04-22T17:23:21Z

LCR-85 Reformat the skill system to support multiple harnesses (Claude Code, Codex, Cursor, ...)

Today the skill system is Claude-Code-specific: .claude/skills/lc-*/SKILL.md files dropped into the project during lc init. To land at launch as an "open standard for science," it should not force a specific harness — users on Codex, Cursor, OpenCode, Copilot, etc. should get equivalent agent guidance.

Current state

The CLI ships these skills today (see dev/*/.claude/skills/):

lc-new — scaffold a new analysis
lc-build — build/iterate loop
lc-verify — verification pass
lc-migrate — move an existing project into the ASTRA/Lightcone CLI format
lc-feedback — capture telemetry
authoring — docs authoring skill

Plus the lc-extractor agent (.claude/agents/lc-extractor.md).

All of these are copied into each target project by lc init / lc update --sync.

Design direction — OpenSpec-style per-tool delivery

OpenSpec solves the same problem by keeping a single canonical skill set and mapping it to each tool's native format at install time:

Canonical skills (openspec-propose, openspec-apply, openspec-archive, ...) live in one source location.
openspec init --tools <id>[,<id>...] (where <id> ∈ claude, cursor, codex, github-copilot, opencode, ...) stamps out the right files per tool:
- Claude Code → .claude/skills/openspec-*/SKILL.md
- Cursor → .cursor/skills/openspec-*/SKILL.md
- GitHub Copilot → .github/prompts/opsx-<id>.prompt.md
- etc.
Slash commands are mapped analogously per tool.
--tools all / --tools none also supported; users can re-run init to add/remove harness support.

It's a directory-structure lookup table, not transpilation — the skill content stays identical; only the file location and filename convention changes per harness.

Proposed scope for the `lightcone-cli` repo

Extract the current Claude-Code skill files into a canonical source inside the lightcone-cli package (e.g. src/lightcone_cli/skills/<skill-name>/SKILL.md + any scripts/assets alongside).
Add a small harness registry — one entry per supported tool mapping skill/command/agent content to install paths.
Rework lc init / lc update --sync to accept --tools and stamp out the right files. Default --tools claude preserves current behavior for beta users.
Launch-day supported harnesses: Claude Code (primary) + at least one more (propose Codex since that's the second most-requested among beta users). Others can be added post-launch.
Docs page covering the harness matrix and how to add a new harness.

Coordinates with LCR-86 (split core vs. research-assistance skills) — decisions there determine which skills flow through the harness map and which ship separately.
Blocks most skill-authoring work that lands in the last weeks before launch — avoid writing skill content twice by doing the restructure early.

References

OpenSpec supported tools table: supported tools
Superpowers skill framework (another reference for composable skill design): superpowers

github-actions · 2026-04-22T17:23:47Z

✅ Eval Results

Metric	Value
Score	1.00
Build complete	❌
Cost	$1.05
Turns	47
Duration	298s
lightcone-cli	`0.1.dev1+g5b6b40804` (`5b6b4080`)
Results	Download

Graders

✅ spec_valid (1.00)
✅ all_materialized (1.00)

Full output

6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:39:02 httpx HTTP Request: POST https://proxy.app.daytona.io/toolbox/bc72d5f9-4349-46c9-8d6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:39:03 httpx HTTP Request: POST https://proxy.app.daytona.io/toolbox/bc72d5f9-4349-46c9-8d6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:39:03 httpx HTTP Request: POST https://proxy.app.daytona.io/toolbox/bc72d5f9-4349-46c9-8d6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:39:03 httpx HTTP Request: POST https://proxy.app.daytona.io/toolbox/bc72d5f9-4349-46c9-8d6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:39:04 httpx HTTP Request: POST https://proxy.app.daytona.io/toolbox/bc72d5f9-4349-46c9-8d6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:39:04 httpx HTTP Request: POST https://proxy.app.daytona.io/toolbox/bc72d5f9-4349-46c9-8d6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:39:05 httpx HTTP Request: POST https://proxy.app.daytona.io/toolbox/bc72d5f9-4349-46c9-8d6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:39:05 httpx HTTP Request: POST https://proxy.app.daytona.io/toolbox/bc72d5f9-4349-46c9-8d6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:39:05 httpx HTTP Request: POST https://proxy.app.daytona.io/toolbox/bc72d5f9-4349-46c9-8d6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:39:06 httpx HTTP Request: POST https://proxy.app.daytona.io/toolbox/bc72d5f9-4349-46c9-8d6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:39:06 httpx HTTP Request: POST https://proxy.app.daytona.io/toolbox/bc72d5f9-4349-46c9-8d6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:39:06 httpx HTTP Request: POST https://proxy.app.daytona.io/toolbox/bc72d5f9-4349-46c9-8d6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:39:07 httpx HTTP Request: POST https://proxy.app.daytona.io/toolbox/bc72d5f9-4349-46c9-8d6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:39:07 httpx HTTP Request: POST https://proxy.app.daytona.io/toolbox/bc72d5f9-4349-46c9-8d6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:39:07 httpx HTTP Request: POST https://proxy.app.daytona.io/toolbox/bc72d5f9-4349-46c9-8d6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:39:08 httpx HTTP Request: POST https://proxy.app.daytona.io/toolbox/bc72d5f9-4349-46c9-8d6d-b4e1f9104382/files/bulk-upload "HTTP/1.1 200 OK"
21:44:09 lightcone.eval.sandbox Deleted sandbox for trial build-snae-0
  snae trial 0: score=1.00 incomplete

lightcone-cli: 0.1.dev1+g5b6b40804 (HEAD 5b6b4080)

  Eval Results: Scores  
┏━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Task ┃     Score     ┃
┡━━━━━━╇━━━━━━━━━━━━━━━┩
│ snae │ 1.00 +/- 0.00 │
│      │  pass@k: 0%   │
└──────┴───────────────┘

   Eval Results: Cost &   
         Duration         
┏━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Task ┃ Cost / Duration ┃
┡━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ snae │      $1.05      │
│      │      298s       │
└──────┴─────────────────┘

Total: 1 trials, $1.05, 298s

Results saved to: eval-results/build-5b6b4080/results.json

Update TestSyncProjectPlugins to reflect that sync only copies skills, agents, and guides. Hooks and scripts are init-time only. - test_sync_copies_plugin_dirs: assert agents synced, hooks/scripts not - test_sync_scripts_executable: renamed to test_sync_no_scripts - _make_plugin_source fixture: add agents/ dir Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

aboucaud · 2026-04-22T19:43:56Z

@claude please review this PR

claude · 2026-04-22T19:44:15Z

Claude finished @aboucaud's task in 2m 3s —— View job

PR Review: LCR-85 Multi-Harness Skill System

Tasks

Read CLAUDE.md for repo guidelines
Review src/lightcone/cli/harness.py (new harness registry)
Review src/lightcone/cli/commands.py (modified init/update with --tools)
Review tests/test_harness.py and tests/test_harness_install.py
Review tests/test_cli.py changes
Review docs/adr/0001-adopt-dagster-slurm.md
Compile and post review feedback

Overall this is a clean, well-structured refactor. The registry abstraction is the right call and the test coverage is solid. A few issues to address before merging:

Bugs / Correctness

1. Type annotation bug in _display_install_summary

commands.py:1064 declares global_notices: list[str] = [] but line 1070 appends (h.tool_name, gpath) tuples to it. mypy strict mode will catch this.

# current (wrong)
global_notices: list[str] = []
...
global_notices.append((h.tool_name, gpath))   # tuple, not str

# fix
global_notices: list[tuple[str, str]] = []

Fix this →

2. _install_harnesses ignores has_skills / has_agents / has_guides flags

commands.py:898-912 always copies skills, agents, and guides unconditionally. _sync_project_plugins (line 2270-2275) correctly checks these flags. A hypothetical harness with has_skills=False would get skills installed at init time but not at sync time — a silent inconsistency.

# _install_harnesses should mirror _sync_project_plugins's guards:
if h.has_skills:
    _copy_dir(plugin_source / "skills", prefix / "skills")
if h.has_agents:
    _copy_dir(plugin_source / "agents", prefix / "agents")
    ...
if h.has_guides:
    _copy_dir(plugin_source / "guides", prefix / "guides")

Fix this →

3. _sync_project_plugins accesses HARNESS_REGISTRY[tid] without validation

commands.py:2264 uses a bare dict lookup that raises KeyError on an unknown ID. resolve_harnesses() already does proper validation with a descriptive error. Prefer reusing it here for consistency:

# current
for tid in tool_ids:
    harness = HARNESS_REGISTRY[tid]   # KeyError on unknown id

# fix: reuse existing validator
for harness in resolve_harnesses(tool_ids):
    ...

Fix this →

Design Issues

4. --tools Choice list duplicated across three locations

commands.py declares the valid choice list twice (for init and update), independently of the registry. Adding a new harness requires updating three places. Reference ALL_TOOL_IDS directly:

# commands.py — both @click.option("--tools") decorators
type=click.Choice(list(ALL_TOOL_IDS)),

Fix this →

5. ensure_dir in harness.py uses bare print() instead of Rich

harness.py:147 uses print() while the rest of the codebase uses console.print(). Since harness.py is intentionally a leaf module (no imports from commands), this is structurally correct — but it means the warning bypasses Rich formatting and won't respect the console's capture mode in tests. Consider accepting a warn_fn callback or just letting the caller handle the OSError.

6. Duplicate tool IDs silently double-install

resolve_harnesses(("claude", "claude")) produces two entries and _install_harnesses will copy each harness's content twice. The test test_duplicate_tools_preserved confirms this is accepted behavior. Since Click multiple=True options can produce duplicates from the CLI, it may be worth de-duplicating at the resolve_harnesses level (preserving order with dict.fromkeys):

ids = list(dict.fromkeys(tool_ids)) if tool_ids else ["claude"]

Minor Issues

7. _display_install_summary omits agents and guides

The summary (line 1076) only shows {prefix}/skills for all harnesses, and adds {prefix}/hooks and {prefix}/settings.json for Claude. Agents and guides are also installed but never mentioned. This will confuse users checking where things landed.

8. Wrong ADR in this PR

docs/adr/0001-adopt-dagster-slurm.md references src/prism/... paths and describes replacing SLURM execution backends — unrelated to multi-harness skills. It appears to be from a different branch. Should this be in this PR?

Tests

The test structure is clean and covers the important cases well. A few notes:

test_update_sync_default_only_claude (line 137 of test_harness_install.py) asserts exit_code == 0 or exit_code == 1 or exit_code is None — this is effectively a no-op assertion. Consider mocking _prompt_sync_projects to test the tools plumbing without the interactive prompt.
TestResolveGlobalCommandsPath::test_env_expansion_unresolved and test_codex_unresolved_when_env_unset are testing the same thing (both assert the result is non-None and contains CODEX_HOME). Could be merged.

Summary: Four issues worth fixing before merge — the type annotation bug (#1), the has_* flag inconsistency between init and sync (#2), missing validation in _sync_project_plugins (#3), and the ADR document that appears misplaced (#8). Issues #4–#7 are polish.

- Add harness.py to repository structure - Add harness registration to extending table Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

aboucaud · 2026-04-22T19:56:33Z

@claude please fix the reported issues

claude · 2026-04-22T19:56:50Z

- Fix type annotation: global_notices list[str] -> list[tuple[str,str]] - Add has_skills/has_agents/has_guides guards to _install_harnesses to match the same guards already present in _sync_project_plugins - Replace raw HARNESS_REGISTRY[tid] lookups with resolve_harnesses() in _sync_project_plugins and _prompt_sync_projects for consistent validation - Replace hardcoded click.Choice tool lists with ALL_TOOL_IDS so adding a new harness requires only one change in the registry - Add agents/guides entries to _display_install_summary output - Replace print() in ensure_dir with warnings.warn() (leaf module) - De-duplicate tool IDs in resolve_harnesses() using dict.fromkeys - Remove misplaced docs/adr/0001-adopt-dagster-slurm.md (Prism/SLURM ADR unrelated to this multi-harness PR) Co-authored-by: Alexandre Boucaud <aboucaud@users.noreply.github.com>

aboucaud · 2026-04-22T20:15:26Z

@claude please fix the test errors you created

claude · 2026-04-22T20:15:40Z

Claude finished @aboucaud's task in 2m 9s —— View job

Tasks

Run tests to identify failures — found 2 failures in tests/test_harness.py
Investigate root causes
Fix test_duplicate_tools_preserved → renamed test_duplicate_tools_deduplicated, updated assert len(result) == 2 → assert len(result) == 1 to match the de-duplication added in fix Add tiered Claude Code permission presets to prism init #6
Fix test_warns_on_permission_error — removed capsys stdout check, replaced with pytest.warns(UserWarning, match="Cannot create directory") to properly capture warnings.warn() output from fix Add prism setup command, replace prism remote #5
Committed and pushed → b0975c4

Both failures were in tests/test_harness.py and were directly caused by the previous round of fixes: fix #6 de-duplicated tool IDs in resolve_harnesses but the test still expected length 2, and fix #5 switched print() to warnings.warn() but the test was reading from stdout via capsys.
| Branch

…warn - test_duplicate_tools_preserved → test_duplicate_tools_deduplicated: update assertion to len==1 after fix #6 introduced dict.fromkeys de-duplication - test_warns_on_permission_error: replace capsys stdout check with pytest.warns(UserWarning) after fix #5 changed print() to warnings.warn() Co-authored-by: Alexandre Boucaud <aboucaud@users.noreply.github.com>

1. CLAUDE.md: update Plugin system section to reflect multi-harness install/sync behaviour; add convention to keep CLAUDE.md current during PRs. 2. harness.py: add comments clarifying that has_hooks/has_settings are Claude Code-specific flags (no other tool has per-project agent automation hooks or settings files). 3. commands.py: make pip upgrade failure non-fatal in lc update — warn and continue to sync instead of raising SystemExit(1), so lc update --tools X works from inside a project venv. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

aboucaud · 2026-04-22T21:07:47Z

Still missing

documentation update
a verification whether hooks do or do not exist on other CLI tools
a fix for the last manual check to pass

- docs/architecture.md: rename plugin section to "Agent harness plugin"; show per-harness vs Claude-only content; list harness registry - docs/cli/init.md: add --tools option; update modes and internal helpers - docs/cli/update.md: add --tools option; clarify what is/isn't synced; note non-fatal pip upgrade; update examples - docs/skills/index.md: update install paths; add multi-harness table Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

The plugin source directory was named after Claude Code. Now that skills, agents, and guides are distributed to all harnesses, rename to plugin/ to reflect its actual purpose. - git mv claude/ plugin/ (history preserved) - pyproject.toml: update force-include and sdist include paths - src/lightcone/cli/plugin.py: update bundled + dev path constants - CLAUDE.md, docs/: replace all claude/lightcone and lightcone/cli/claude/lightcone path references Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

AGENTS.md is the harness-agnostic convention read by Codex and other agent tools. CLAUDE.md is retained as a one-line pointer for Claude Code session compatibility. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

claude-md: Reflect harness.py in project docs (LCR-85)

057f23b

- Add harness.py to repository structure - Add harness registration to extending table Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

aboucaud marked this pull request as draft April 22, 2026 21:01

aboucaud and others added 3 commits April 22, 2026 23:09

aboucaud marked this pull request as ready for review April 22, 2026 21:39

aboucaud requested a review from EiffL April 22, 2026 21:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LCR-85: Add multi-harness skill system with --tools flags#76

LCR-85: Add multi-harness skill system with --tools flags#76
aboucaud wants to merge 9 commits into
mainfrom
lcr-85-multi-harness

aboucaud commented Apr 22, 2026 •

edited

Loading

Uh oh!

linear Bot commented Apr 22, 2026

Current state

Design direction — OpenSpec-style per-tool delivery

Proposed scope for the `lightcone-cli` repo

Related

References

Uh oh!

github-actions Bot commented Apr 22, 2026 •

edited

Loading

Uh oh!

aboucaud commented Apr 22, 2026

Uh oh!

claude Bot commented Apr 22, 2026 •

edited

Loading

Uh oh!

aboucaud commented Apr 22, 2026

Uh oh!

claude Bot commented Apr 22, 2026 •

edited

Loading

Uh oh!

aboucaud commented Apr 22, 2026

Uh oh!

claude Bot commented Apr 22, 2026 •

edited

Loading

Uh oh!

aboucaud commented Apr 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

aboucaud commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

linear Bot commented Apr 22, 2026

Current state

Design direction — OpenSpec-style per-tool delivery

Proposed scope for the lightcone-cli repo

Related

References

Uh oh!

github-actions Bot commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Eval Results

Graders

Uh oh!

aboucaud commented Apr 22, 2026

Uh oh!

claude Bot commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review: LCR-85 Multi-Harness Skill System

Bugs / Correctness

Design Issues

Minor Issues

Tests

Uh oh!

aboucaud commented Apr 22, 2026

Uh oh!

claude Bot commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Fixed all reported issues

Uh oh!

aboucaud commented Apr 22, 2026

Uh oh!

claude Bot commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aboucaud commented Apr 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

aboucaud commented Apr 22, 2026 •

edited

Loading

Proposed scope for the `lightcone-cli` repo

github-actions Bot commented Apr 22, 2026 •

edited

Loading

claude Bot commented Apr 22, 2026 •

edited

Loading

claude Bot commented Apr 22, 2026 •

edited

Loading

claude Bot commented Apr 22, 2026 •

edited

Loading