[spark-compete] Narrow bare except in inspect_spawner_prd_auto_trace JSONL scanner to (json.JSONDecodeError) + (OSError)

[4gjnbzb4zf-sudo]

{
"schema": "spark-compete-hotfix-v1",
"event": "spark-compete-first-event",
"submission_mode": "public_repo_pr",
"submission_target_url": "#533",
"team": {
"name": "SparkThisUp",
"members": ["ValHallaBuilder", "Baz707", "DanFireDash"],
"github_accounts": ["4gjnbzb4zf-sudo"],
"llm_device_holder": "ValHallaBuilder",
"device_holder_github": "4gjnbzb4zf-sudo"
},
"target_repo": {
"id": "vibeforge1111/spark-cli",
"source": "https://github.com/vibeforge1111/spark-cli",
"owner_surface": "spark-cli"
},
"issue": {
"type": "bug",
"severity": "low",
"title": "inspect_spawner_prd_auto_trace JSONL scanner swallows programming bugs as bare Exception twice (per-line parse and outer file open)",
"actual_behavior": "src/spark_cli/system_map.py:inspect_spawner_prd_auto_trace walks a JSONL trace file line-by-line, parses each line with json.loads(line), and collects identifier fields (requestId, missionId, traceRef) into the system-map join-keys block. The per-line parse uses except Exception: continue at L872 and the outer file-open block uses except Exception as exc: out['join_error'] = ... at L887. Both swallow any unrelated runtime error: a typo on payload.get(...) raises AttributeError that gets silently skipped at the per-line layer, while a NameError or KeyboardInterrupt during the scan gets repackaged as a synthetic join_error: AttributeError: ... diagnostic that lies about what actually failed.",
"expected_behavior": "Per-line skip only happens for the documented fallback (a malformed JSONL line that cannot be parsed). The outer join_error diagnostic is preserved for the documented disk-layer failures (missing file, permission denied, unreadable encoding). Programming bugs propagate naturally so the operator sees them as a real traceback.",
"repro_steps": [
"gh pr checkout ",
"PYTHONPATH=src python -c "from spark_cli.system_map import inspect_spawner_prd_auto_trace; from pathlib import Path; import tempfile, json; d = Path(tempfile.mkdtemp()); p = d/'trace.jsonl'; p.write_text(json.dumps({'requestId':'r1','missionId':'m1','traceRef':'t1'}) + '\n' + 'not json\n' + json.dumps({'requestId':'r2','missionId':'m2'}) + '\n'); r = inspect_spawner_prd_auto_trace(p, builder_home=d); print('join_keys:', r['join_keys'], 'join_error:', r.get('join_error'))"",
"Output before AND after: join_keys: {'request_id_count': 2, 'mission_id_count': 2, 'trace_ref_count': 1, 'derived_trace_ref_count': 2} join_error: None -- the malformed JSONL line is still skipped, the valid lines are still counted, no spurious join_error.",
"Inverse: edit a typo into the per-line loop (e.g. payload.get_foo_typo) -- before the change, the AttributeError gets silently swallowed and the JSONL scanner reports zero matches; after the change, the AttributeError propagates so the operator sees the real bug."
],
"affected_workflow": "system_map.py is the system-map compile module driven by spark report system-map. inspect_spawner_prd_auto_trace produces the join-keys block that the system-map pipeline uses to correlate spawner PRD traces with builder-side state. When the scanner silently swallows a programming bug, downstream join-overlap statistics come back understated and the operator never finds out the scanner itself broke."
},
"evidence": {
"safe_links_only": true,
"before_after_proof": "Two stacked try/except patterns. Before: per-line except Exception: continue and outer except Exception as exc: out['join_error'] = f'{type(exc).__name__}: {exc}'. After: except json.JSONDecodeError: continue and except OSError as exc: out['join_error'] = f'{type(exc).__name__}: {exc}'. The except bodies are byte-identical -- only the catch classes change. The per-line try body is one statement (json.loads of a single line); JSONDecodeError is the exhaustive class. The outer try body is a file-open + a per-line read loop; OSError covers FileNotFoundError, PermissionError, UnicodeDecodeError. Diff: 2 insertions, 2 deletions, one file.",
"links": ["https://github.com//pull/533"],
"forbidden": ["pdf","zip","exe","unknown downloads","shortened links","archives","binaries","tokens","browser cookies","wallet material","raw logs","raw conversations","raw memory","raw patches","private repo maps","private scoring details"]
},
"proposed_fix": {
"approach": "Replace the per-line except Exception: with except json.JSONDecodeError: and the outer except Exception as exc: with except OSError as exc:. Both narrow classes are exhaustive for their respective try bodies. The except bodies are byte-identical pre/post -- the per-line layer still skips malformed lines, the outer layer still records join_error with the same {type(exc).__name__}: {exc} format string.",
"files_expected": ["src/spark_cli/system_map.py"],
"tests_or_smoke": "Manual smoke verified: mixed valid/invalid JSONL input (two valid records + one malformed line) returns the expected join_keys counts (request_id_count: 2, mission_id_count: 2, trace_ref_count: 1, derived_trace_ref_count: 2) with join_error: None. AST parse of system_map.py is clean. Diff is 2 insertions, 2 deletions."
},
"pr": {
"branch": "sentinel/exception-narrow/spawner-prd-auto-trace-jsonl",
"title_prefix": "[spark-compete]",
"author_github": "4gjnbzb4zf-sudo",
"body_must_include": ["packet","team","pr_author","repo","actual_behavior","expected_behavior","repro_steps","before_after_proof","tests_or_smoke","duplicate_notes","risk_notes","review_claim"],
"url": "#533"
},
"review_claim": {
"impact_claim": "low",
"evidence_types": ["redacted_terminal_excerpt"],
"duplicate_notes": "Pre-flight gh pr list --search 'spawner_prd_auto_trace narrow' returned zero hits. Our own #526 narrowed six different sqlite-introspection helpers in this same file (L651/L941/L983/L2576/L2684/L2742) and our pending #532 narrows the two top-of-file read_json/read_toml helpers (L306/L314). inspect_spawner_prd_auto_trace is at L848-L887, distinct from all of those. No competitor PR touches this function.",
"risk_notes": "Local scope: one file, two lines changed. The except bodies are byte-identical pre/post. The per-line skip semantics for malformed JSONL lines are unchanged. The outer join_error diagnostic is preserved for the documented disk-layer failures. The only observable behavior change is that programming bugs in the scanner no longer get mis-attributed to file corruption or silently dropped -- which is the goal of the fix.",
"review_state_requested": "pr_review"
}
}

[4gjnbzb4zf-sudo]

QA write-up — for reviewer eyes

TL;DR. inspect_spawner_prd_auto_trace in src/spark_cli/system_map.py walks a JSONL trace file and parses each line with json.loads(line). Both the per-line parse (L872) and the outer file-open block (L887) use bare except Exception: — so an AttributeError on a payload.get(...) typo gets silently dropped at the per-line layer, while a NameError or KeyboardInterrupt during the scan gets repackaged into a synthetic join_error: ... diagnostic. Narrow the inner to json.JSONDecodeError, the outer to OSError.

Before

if path.exists():
    try:
        with path.open("r", encoding="utf-8") as handle:
            for line in handle:
                if not line.strip():
                    continue
                try:
                    payload = json.loads(line)
                except Exception:
                    continue
                if not isinstance(payload, dict):
                    continue
                ...
    except Exception as exc:
        out["join_error"] = f"{type(exc).__name__}: {exc}"

If a typo lands on payload.get(...), the AttributeError gets silently skipped and the JSONL scanner reports zero matches with no signal that the scanner itself broke.

After

                try:
                    payload = json.loads(line)
                except json.JSONDecodeError:
                    continue
                ...
    except OSError as exc:
        out["join_error"] = f"{type(exc).__name__}: {exc}"

Same skip semantics for malformed JSONL lines. Same join_error diagnostic for the documented disk-layer failures (missing file, permission denied, unreadable encoding). Programming bugs propagate so the operator gets a real traceback.

Smoke

$ PYTHONPATH=src python3 -c "
from spark_cli.system_map import inspect_spawner_prd_auto_trace
from pathlib import Path
import tempfile, json

d = Path(tempfile.mkdtemp())
p = d / 'trace.jsonl'
p.write_text(
    json.dumps({'requestId':'r1','missionId':'m1','traceRef':'t1'}) + '\n'
    + 'not json\n'
    + json.dumps({'requestId':'r2','missionId':'m2'}) + '\n'
)
r = inspect_spawner_prd_auto_trace(p, builder_home=d)
print('join_keys:', r['join_keys'])
print('join_error:', r.get('join_error'))
"
join_keys: {'request_id_count': 2, 'mission_id_count': 2, 'trace_ref_count': 1, 'derived_trace_ref_count': 2}
join_error: None

Two valid records counted, malformed line skipped, no spurious join_error.

What didn't change

One file, two lines changed. Except bodies byte-identical pre/post at both sites. The success path through the JSONL loop is unchanged. The per-line malformed-line skip and the outer join_error diagnostic are preserved for their documented failure conditions. The only observable behavior change is that programming bugs in the scanner no longer get silently swallowed or mis-attributed — the goal of the fix.

Pattern-match: same shape as our own #163, #526, and pending #532 in the same file.

[vibeforge1111]

Thanks, this was useful. We maintainer-adopted this exception-boundary improvement in #609 and merged it into master at 5e60f9a12c40438779a4aa1e0b9293e1fed7ce72.

What shipped:

• JSON/TOML helpers now catch parse and OS errors instead of broad Exception.
• Spawner PRD trace scanning catches line JSON decode failures and file OS failures at the right boundary.
• git helper subprocess failures catch expected subprocess/OS errors.
• revoke-all active mission pause write failures catch expected OS write errors.

Verification on the adopted PR:

• focused system-map and revoke-all tests passed
• full tests/test_system_map.py passed
• full tests/test_cli.py passed
• compileall, git diff --check, registry-pin verification, CodeQL, scorecard, secret-scan, and test-and-audit all passed

Source credit is recorded for this PR, and #609 itself should not receive participant credit. Points are still gated because this PR body had spark-compete-hotfix-v1 schema text but not a clean fenced JSON packet that the validator can extract. To unlock points review, add a clean fenced JSON packet in a PR comment or body update, then validate it at https://compete.sparkswarm.ai/api/packet/validate.

Closing this PR as adopted into #609. No code action is needed unless you believe #609 missed part of the issue.

[vibeforge1111]

R22/R23 final scoring update: this PR did receive final R22/R23 public leaderboard credit. Earlier gate, credit-review, or points-lock wording described the pre-final review state, not a final rejection or permanent zero. Final R22/R23 credit for this PR: 48 points. Thanks for the contribution.