Skip to content

Troubleshooting

Jump to bottom
Pengfei Hu edited this page May 31, 2026 · 2 revisions

Troubleshooting

Common errors, their root causes, and the fastest way out. Cross-referenced to relevant pages.

Exit codes

0    Pass (advisory mode, or strict mode with no fail_on hits)
2    Config error  — manifest is missing, malformed, or has typos
3    Input parse error — a tool source file is malformed, too large, or escapes the manifest dir
4    Other AgentsShipgateError (also: --strict-plugins on a plugin/adapter failure)
6    Baseline integrity failure — `agents-shipgate baseline verify --strict` detected SHIP-BASELINE-INTEGRITY-MISMATCH
20   Strict-mode gate failure — ≥1 unsuppressed finding hit ci.fail_on, or ≥1 active unbaselined finding set blocks_release

Exit code 6 is emitted only by the standalone baseline verify command; scan/verify always use 20 for gate failure. Anything else is an unhandled exception — please file a bug. Run with --verbose to surface the traceback.

Reading the result, not the exit code. The release gate is agents-shipgate-reports/report.json → release_decision.decision (blocked | review_required | insufficient_evidence | passed), not the exit code. In advisory mode (the default) the process exits 0 even when the decision is blocked. Lead with the decision; treat the exit code as the CI fail signal only.

CLI errors

Config file not found: shipgate.yaml

Config error: Config file not found: shipgate.yaml in /home/me/repo. 
Run `agents-shipgate init --workspace . --write` to create one.

You're not in the right directory, or the file doesn't exist yet. The hint is real — init --write produces a starter manifest.

Invalid shipgate.yaml: ... Did you mean X?

Config error: Invalid shipgate.yaml:
- agent.declaire_purpose: Extra inputs are not permitted. Did you mean declared_purpose?

The manifest is validated strictly. Typo suggestions are derived programmatically from the schema — if the suggestion equals what you typed, it means you put a real field at the wrong nesting level (e.g. declared_purpose at the root instead of inside agent). Move it to the right place.

Unsupported manifest version

Config error: Unsupported manifest version '0.2'; this Agents Shipgate build supports version '0.1'.

The manifest schema is version: "0.1" even though the CLI is at v0.10.0 and the report schema is at v0.22. The three counters are independent — the manifest grammar bumps on its own deprecation cycle. Keep version: "0.1" in shipgate.yaml.

Input path '...' resolves outside manifest directory

Input parsing error: Input path '../../shared/openapi.yaml' resolves outside manifest directory: 
/home/me/shared/openapi.yaml

A path under tool_sources[].path (or any openai_api.*.path, anthropic.*.path, etc.) escaped the manifest's directory. Out-of-tree paths are rejected by containment (exit 3). Three workarounds:

  1. Move the spec inside the manifest tree.
  2. Symlink it: ln -s ../shared/openapi.yaml openapi.yaml then point path: openapi.yaml.
  3. Copy it as part of CI prep: 
    - run: cp ../shared/openapi.yaml .
- uses: ThreeMoonsLab/agents-shipgate@v0.10.0

Input file too large: … is X bytes; maximum is 10485760 bytes

The 10 MB limit is per file and intentional. If your OpenAPI spec is genuinely larger than 10 MB, split it (paths per service) or open an issue with your use case.

Unable to parse OpenAPI operation … RecursionError

OpenAPI specs with deeply nested or pathological $ref cycles will trip the schema-resolver bounds (MAX_SCHEMA_RESOLVE_DEPTH=32, MAX_SCHEMA_RESOLVE_NODES=5000). The truncated branch is annotated with x-agents-shipgate-resolution-truncated: max_depth or max_nodes rather than crashing — but if you see a RecursionError reach the CLI, the spec is genuinely deeper than the bounds. File an issue with the spec.

Unsupported report format(s): …

The report (--format) supports markdown, json, and sarif. HTML and PDF are packet formats, not report formats — request them through the Release Evidence Packet instead:

agents-shipgate scan -c shipgate.yaml --packet-format md,json,html
pipx install 'agents-shipgate[pdf]'                         # adds packet.pdf
agents-shipgate scan -c shipgate.yaml --packet-format md,json,html,pdf
# Re-render from an existing packet at full fidelity:
agents-shipgate evidence-packet --from agents-shipgate-reports/packet.json --format html,pdf

The packet lands at agents-shipgate-reports/packet.{md,json,html} (plus packet.pdf with the [pdf] extras). packet.pdf requires weasyprint via the optional extras.

Agent-mode structured errors

When you set AGENTS_SHIPGATE_AGENT_MODE=1, every error prints a structured JSON line on stderr with an error kind plus next_action (single string, back-compat) and next_actions (ranked list). Pattern-match the error field instead of scraping the human message:

$ AGENTS_SHIPGATE_AGENT_MODE=1 agents-shipgate scan -c missing.yaml
Config error: Config file not found: missing.yaml
{"error": "config_error", "message": "...", "next_action": "agents-shipgate detect --workspace . --json", "next_actions": [{"kind": "command", "command": "agents-shipgate detect --workspace . --json", "why": "..."}]}

The full set of error kinds: config_error, config_already_exists, input_parse_error, unknown_check_id, unknown_fingerprint, other_error, internal_error, malformed_patch. unknown_fingerprint is emitted by explain-finding when the fingerprint matches no entry in the supplied report (the payload includes a close-match suggestion when one exists). The machine-readable catalog — exit codes, causes, per-kind fields, recovery hints — lives at docs/errors.json.

config_error splits two ways in agent mode: a missing manifest routes to agents-shipgate detect … --json (then init --write); an invalid manifest (bad YAML, schema failure, unsupported version) routes to edit <path> — do not re-run init, which refuses to overwrite an existing file.

Three read-only JSON diagnostics help you triage without a full scan:

agents-shipgate doctor --json        # resolves tool_sources; reports unresolved_sources[]
agents-shipgate contract --json      # installed CLI contract: schema versions + gating signal
agents-shipgate self-check --json    # returns {"ready": true} when the install is healthy

doctor --json exits 0 even when a required tool_sources[].path is missing or escapes the manifest dir — it reports a SHIP-DIAG-MISSING-SOURCE-FILE diagnostic and an unresolved_sources[] field so an agent can route to a fix. (doctor without --json exits 3 to preserve loud failure for humans; scan always raises InputParseError (exit 3) on missing sources.)

Findings that look wrong

"Why is this GET tool flagged for missing approval?"

GETs without write hints are tagged read_only at HIGH confidence and short-circuited from policy checks. The remaining cases that fire are typically:

  • The operationId contains a destructive keyword (e.g. get_v2_*_destroy_with_associated_resources). The destroy token correctly maps to the destructive tag. Suppress per-tool with checks.ignore or use risk_overrides.tools.{tool}.remove_tags: [destructive] to override the heuristic.
  • The OpenAPI method isn't GET. Some specs use x-agents-shipgate.httpMethod or other annotations to indicate a method that contradicts the actual REST verb. Inspect the tool in report.json's tool_inventory.

"Why isn't delete_* flagged?"

If a tool with a delete keyword in its name isn't getting the destructive tag, check:

  1. Is it from an SDK source where the function name was overridden via name_override? The override name is what's tokenized.
  2. Did risk_overrides.tools.{tool}.remove_tags: [destructive] accidentally suppress it?
  3. Is is_effectively_read_only returning true because of an readOnlyHint: true MCP annotation?

Run with --verbose and pipe to AGENTS_SHIPGATE_LOG_FORMAT=json to see the full risk-hint trace per tool.

"My SDK function isn't being detected"

The static AST extractor only sees direct decorators with these names:

  • @function_tool
  • @agents.function_tool
  • @openai_agents.function_tool
  • Aliased imports of those (e.g. from agents import function_tool as ft → @ft)

It does NOT detect:

  • Tools added via Agent(tools=[...]) from a list built at runtime
  • Factory functions like make_tool(...)
  • Tool.from_fn(...) constructions

For higher-confidence inventory, declare those tools through MCP exports or OpenAPI. See Real-World Examples § OpenAI Agents SDK.

"My description was set via description_override but flagged missing"

The OpenAI Agents SDK extractor reads name_override / description_override from decorator kwargs (it does not fall back to the docstring when the override is present). If you still see this on a current build, file a false-positive issue with the minimal decorator — it's a bug.

"False positive that I can't override with risk_overrides"

risk_overrides.tools.{tool}.remove_tags removes hints by tag. If the heuristic is producing the wrong fingerprint identity (different tool, same evidence shape), suppress the specific finding via checks.ignore instead. Always include a reason — it shows up in the audit trail.

CI integration issues

Action runs but PR comment doesn't appear

Check the workflow's permissions::

permissions:
  contents: read
  pull-requests: write     # required for pr_comment

Without pull-requests: write, the action logs a warning and skips the comment. The artifact upload still happens.

Job ends with exit code 20 but I'm in advisory mode

The ci.fail_on setting in shipgate.yaml overrides the CLI mode. If the manifest declares ci.fail_on: [critical, high], the gate fires regardless of --ci-mode advisory. Either remove the fail_on from the manifest or pass --fail-on "" on the CLI.

Duplicate tool name 'foo'; kept openapi source 'bar' and ignored mcp source 'baz'

When a tool name appears in multiple sources, the higher-priority source wins (openai_api > openapi > mcp > sdk_function). The losing source's risk hints are dropped. If both sources had useful metadata, you have three options:

  1. Rename one of the tools. Use an alias in your MCP export or add a prefix.
  2. Remove the duplicate source. If the OpenAPI fully covers the MCP, drop the MCP entry.
  3. Use risk_overrides on the surviving tool to add back any tags the dropped source contributed.

This warning is informational, not an error — but a noisy one usually indicates a config drift.

Performance

Scan takes >5 seconds

Profile with:

AGENTS_SHIPGATE_LOG_FORMAT=json agents-shipgate scan -c shipgate.yaml --verbose 2>&1 | \
  grep -E 'loaded sources|risk hints generated|checks completed'

Each timestamped entry tells you where time is being spent. Typical bottlenecks:

  • OpenAPI specs with thousands of $ref references — bounded by MAX_SCHEMA_RESOLVE_NODES, but still walks the tree
  • Multi-config workspace mode with many manifests — runs sequentially today

For the 591-tool DigitalOcean spec, expected wall-clock is ~290 ms.

Report file size is huge

A 600-tool scan can produce a 5-10 MB JSON report. Use --format markdown if you only need the human-readable form, or pipe the JSON to jq -c . for a flat one-line variant.

Still stuck?

Agents Shipgate · Apache-2.0 · maintained by Three Moons Lab · Report a false positive

🏠 Home

Getting started

Reference

Workflows

Extending

Project

Clone this wiki locally