Skip to content

feat(coding): complete auditor suite with performance-auditor and security-auditor#73

Merged
gupsammy merged 2 commits into
mainfrom
feat/coding-perf-security-auditors
May 24, 2026
Merged

feat(coding): complete auditor suite with performance-auditor and security-auditor#73
gupsammy merged 2 commits into
mainfrom
feat/coding-perf-security-auditors

Conversation

@gupsammy
Copy link
Copy Markdown
Owner

Summary

Completes the claude-coding auditor suite by adding the final two agents — performance-auditor and security-auditor — migrating the last two first-generation global agents they replace. The suite now covers all four review dimensions: architecture, code quality, performance, and security.

What changed

  • performance-auditor (new) — static-analysis performance agent. Dual-mode (advisor/auditor), language-agnostic, evidence-grounded. Replaces the old global agent's fabricated Performance Score: X/100 and invented latency tables with complexity/scaling reasoning that cites file:line. Recommends profiling rather than guessing, and never reports a metric it didn't measure.
  • security-auditor (new) — static-analysis security agent. Traces untrusted input source→sink, rates by exploitability × impact on the CVSS Critical/High/Medium/Low scale, and emits an inline report (the old global agent told a read-only auditor to write a security_analysis.md file). Keeps WebSearch for CVE confirmation and permits read-only SCA scanners (npm audit / pip-audit / osv-scanner) for the dependency step only.
  • Version: claude-coding bumped to 0.2.53 via the auto-version hook, with README badges and marketplace.json synced.

Design notes

  • Both agents mirror the existing code-auditor / architecture-auditor structure: dual-mode selection rules, read-only Bash, project memory, and severity-tagged findings grounded in file:line evidence.
  • Two deliberate, suite-aware divergences: security uses the CVSS Critical/High/Medium/Low scale (not the suite's Major/Minor) because every developer security tool speaks CVSS; and the auditor Bash grant stays unscoped + prose-constrained for suite consistency rather than being scoped on one agent alone.

Review

Both agents went through the full authoring pipeline: /create-agent/repair-agent (7-dimension structural audit) → /counselors multi-model review (Opus + Codex 5.5-high) → applied fixes. Counselors-driven tightenings included cross-mode evidence discipline (require file:line in advisor mode), a reachable-path requirement across all security audit steps, and distinguishing confirmed-reachable dependency exposure from advisory-only matches.

Follow-ups (not in this PR)

  • After merge: delete the retired global agents ~/.claude/agents/performance-auditor.md and ~/.claude/agents/security-auditor.md.
  • Optional suite polish noted in review: qualify architecture-auditor's "scale" wording to fully close a minor trigger overlap with performance-auditor.

Migrate the final two first-generation global auditors into the
claude-coding plugin, completing the 4-agent auditor suite alongside
architecture-auditor and code-auditor.

- performance-auditor: static-only, evidence-grounded, language-agnostic
  dual-mode agent; replaces the old version's fabricated Performance
  Score and invented latency metrics with complexity/scaling reasoning
- security-auditor: traced source->sink discipline, CVSS Critical/High/
  Medium/Low severity, inline report (no file creation), read-only SCA
  for the dependency step

Both reviewed via /repair-agent and /counselors (Opus + Codex 5.5).
@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 21, 2026

Claude finished @gupsammy's task in 3m 25s —— View job


PR Review: feat(coding): complete auditor suite with performance-auditor and security-auditor

Todo List

  • Gather context (fetch diff, read changed files)
  • Review agent files against CLAUDE.md conventions
  • Check manifest version sync
  • Post final review

What looks good

Version sync — all four locations updated consistently: plugin.json, marketplace.json, both README badges. Auto-version hook working correctly.

Dual-mode structure — both agents cleanly mirror code-auditor's Advisor/Auditor split with matching output templates, mode-selection rules, and "Done when…" step terminators. The structural consistency across the suite will make it easy to reason about which agent to reach for.

Evidence discipline — the core invariant ("if you can't point to a file and line, it's not a finding") is applied in both agents with the same rigor as their peers, and the counselors-driven cross-mode tightening (requiring file:line in Advisor mode too) is a real improvement over the first-generation agents.

performance-auditor:

  • The "you measure nothing — never fabricate a score or latency" principle is exactly right and directly addresses the old agent's invented Performance Score: X/100 anti-pattern.
  • Scale-calibrated findings ("O(n²) over a user-supplied list — fine at dozens, quadratic at tens of thousands") are much more useful than generic complexity warnings.
  • Bash constraint is clear: read-only structural commands only, no benchmark/build execution.

security-auditor:

  • Reachable-path requirement before any finding is promoted ("confirmed-reachable vs advisory-only") directly closes the false-positive hole that most static security tools have.
  • Defense-in-depth vs exploitable-today distinction in the output template is excellent — conflating them erodes trust in security reports.
  • WebSearch scoped narrowly to CVE/advisory confirmation rather than generic best-practice padding.
  • SCA scanner exception (step 5 only, read-only, manifest-vs-advisory) is well-reasoned and correctly isolated.
  • Exploit output restriction ("a data-flow trace or a single illustrative payload is enough — never produce copy-paste-ready, multi-stage, or obfuscated exploits") is appropriate and aligns with responsible disclosure norms.
  • CVSS severity scale (Critical/High/Medium/Low) over the suite's Major/Minor — the PR documents this as deliberate and the reasoning is sound: every developer security tool speaks CVSS.

Must fix before merge

README agent counts are stale (inline comments posted):

File Current text After this PR
plugins/claude-coding/README.md:3 "Eight skills and one command" "Nine skills and five agents"
README.md:82 "Nine skills and two agents" "Nine skills and five agents"

The plugin now ships five agents (code-auditor, architecture-auditor, test-engineer, performance-auditor, security-auditor). Neither README was updated to reflect the new additions.


Optional / nits

| vs > scalar (inline comments posted on both agent files): CLAUDE.md specifies description: > (folded scalar) for all agent descriptions. Both new agents use description: | (literal block), following code-auditor's pattern. Semantically equivalent, but worth a coordinated cleanup across the suite to match the documented convention.

model: sonnet vs model: inherit: architecture-auditor uses model: inherit, which respects the user's configured default and benefits from future model upgrades automatically. Both new agents hardcode model: sonnet, consistent with code-auditor. Not a blocker — just worth considering whether the suite should standardize on inherit.


Summary

Two clean, well-engineered agents with strong evidence discipline and scope hygiene. The only required fix is updating the agent-count text in both READMEs before merge. All other notes are optional.

Comment thread plugins/claude-coding/README.md Outdated
# claude-coding ![v0.2.52](https://img.shields.io/badge/v0.2.52-blue?style=flat-square)
# claude-coding ![v0.2.53](https://img.shields.io/badge/v0.2.53-blue?style=flat-square)

Coding workflow skills for Claude Code. Eight skills and one command covering the commit loop, project maintenance, documentation, and CI setup: stage and commit with conventional format, push and open a PR with smart branch handling, safely prune merged or stale branches, keep your CLAUDE.md accurate and concise, generate professional READMEs through a structured interview, create or update a changelog from git history, refresh an existing README against current codebase state, and configure production-ready Claude Code GitHub Actions workflows.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description says "Eight skills and one command" but the plugin now ships five agents (code-auditor, architecture-auditor, test-engineer, performance-auditor, security-auditor). This line needs updating to reflect the actual agent count — e.g. "Nine skills and five agents covering…"

Suggested change
Coding workflow skills for Claude Code. Eight skills and one command covering the commit loop, project maintenance, documentation, and CI setup: stage and commit with conventional format, push and open a PR with smart branch handling, safely prune merged or stale branches, keep your CLAUDE.md accurate and concise, generate professional READMEs through a structured interview, create or update a changelog from git history, refresh an existing README against current codebase state, and configure production-ready Claude Code GitHub Actions workflows.
Coding workflow skills for Claude Code. Nine skills and five agents covering the commit loop, project maintenance, documentation, and code quality.

Comment thread README.md Outdated
### 💻 claude-coding   ![v0.2.52](https://img.shields.io/badge/v0.2.52-blue?style=flat-square)
### 💻 claude-coding   ![v0.2.53](https://img.shields.io/badge/v0.2.53-blue?style=flat-square)

Coding workflow skills for Claude Code. Nine skills and two agents covering the commit loop, project maintenance, documentation, and code quality.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still reads "Nine skills and two agents" after adding two new agents. With code-auditor, architecture-auditor, test-engineer, performance-auditor, and security-auditor, the count is now five.

Suggested change
Coding workflow skills for Claude Code. Nine skills and two agents covering the commit loop, project maintenance, documentation, and code quality.
Coding workflow skills for Claude Code. Nine skills and five agents covering the commit loop, project maintenance, documentation, and code quality.

@@ -0,0 +1,169 @@
---
name: performance-auditor
description: |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLAUDE.md convention for agent descriptions is the > folded scalar, not | literal block. Both formats render identically in YAML but > is the documented standard. code-auditor also uses |, so this follows its lead — worth a coordinated cleanup across the suite.

Suggested change
description: |
description: >

@@ -0,0 +1,196 @@
---
name: security-auditor
description: |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same | vs > note as performance-auditor — CLAUDE.md specifies > folded scalar for agent descriptions.

Suggested change
description: |
description: >

The auditor suite now ships five agents (code-auditor, architecture-auditor,
test-engineer, performance-auditor, security-auditor). Update both the plugin
README and the root README section to reflect the new count. Addresses the
must-fix items from the PR #73 review.
@gupsammy gupsammy merged commit e0345da into main May 24, 2026
1 check passed
@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 24, 2026

Claude finished @gupsammy's task in 1m 31s —— View job


Todo List

  • Fetch PR diff and read changed files
  • Review agent files against CLAUDE.md conventions
  • Check manifest version sync
  • Post final review

PR Review: feat(coding): complete auditor suite with performance-auditor and security-auditor

This PR was already merged. Notes below are for the record and to flag any follow-up items.


What looks good

Version sync — All four locations updated consistently: plugin.json, marketplace.json, and both README badges all show 0.2.53. Auto-version hook working correctly.

README agent-count fix — Both README updates from the previous review were included before merge:

  • plugins/claude-coding/README.md:3 → "Nine skills and five agents" ✅
  • README.md:82 → "Nine skills and five agents" ✅

Dual-mode structure — Both agents cleanly mirror code-auditor's Advisor/Auditor split with matching output templates, mode-selection rules, and "Done when…" step terminators. Suite is now internally consistent across all four dimensions.

Evidence discipline — The "if you can't point to file:line, it's not a finding" invariant is consistently applied across both agents, including in Advisor mode (the counselors-driven tightening).

performance-auditor specifics:

  • "You measure nothing — never fabricate a score or latency" is the exact right principle to counter the old agent's invented Performance Score: X/100 anti-pattern.
  • Scale-calibrated findings ("O(n²) over a user-supplied list — fine at dozens, quadratic at tens of thousands") are concrete and actionable.
  • Bash constraint is correctly limited to read-only structural commands.

security-auditor specifics:

  • Reachable-path requirement before promoting any finding directly addresses the false-positive problem in most static tools.
  • Exploitable-today vs defense-in-depth distinction in the output template is well-designed and trust-preserving.
  • SCA scanner exception is properly isolated to step 5 only.
  • Exploit output restriction ("a data-flow trace or a single illustrative payload is enough — never produce copy-paste-ready, multi-stage, or obfuscated exploits") is appropriate.
  • CVSS severity scale over the suite's Major/Minor — rationale is documented and sound.

Optional / nits (not blocking, post-merge polish)

| vs > scalar — CLAUDE.md specifies description: > (folded scalar) for agent descriptions. Both new agents use description: | (literal block), following code-auditor's existing pattern. Semantically identical — worth a coordinated pass across the suite (code-auditor, performance-auditor, security-auditor) to match the documented convention.

Description length — CLAUDE.md targets 50–70 tokens for agent descriptions. Both new descriptions run ~80–90 tokens (performance: ~85, security: ~80), slightly over the budget. Compare architecture-auditor at ~60 tokens. Minor — the descriptions are still within reason and clearly routed — but worth tightening if the suite gets a cleanup pass.

model: sonnet vs model: inheritarchitecture-auditor uses model: inherit (picks up user's configured default and benefits from future model upgrades automatically). Both new agents hardcode model: sonnet, consistent with code-auditor. Optional standardization question for the suite.


Summary

Clean, well-engineered pair of agents. The required README fixes from the previous review were included. All four auditor-suite agents are now present, internally consistent, and evidence-grounded. The remaining notes are optional polish items for a future cleanup pass.

@gupsammy gupsammy deleted the feat/coding-perf-security-auditors branch May 24, 2026 04:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant