[spark-compete] fix(system_map): narrow bare except in 4 JSONL payload scanners to (json.JSONDecodeError, OSError)

[4gjnbzb4zf-sudo]

{
"schema": "spark-compete-hotfix-v1",
"event": "spark-compete-first-event",
"submission_mode": "public_repo_pr",
"submission_target_url": "#823",
"team": {
"name": "SparkThisUp",
"members": [
"ValHallaBuilder",
"Baz707",
"DanFireDash"
],
"github_accounts": [
"4gjnbzb4zf-sudo"
],
"llm_device_holder": "ValHallaBuilder",
"device_holder_github": "4gjnbzb4zf-sudo"
},
"target_repo": {
"id": "vibeforge1111/spark-cli",
"source": "https://github.com/vibeforge1111/spark-cli",
"owner_surface": "spark-cli"
},
"issue": {
"type": "usage_friction",
"severity": "low",
"title": "fix(system_map): narrow bare except in 4 JSONL payload scanners to (json",
"actual_behavior": "[spark-compete] fix(system_map): narrow bare except in 4 JSONL payload scanners to (json.JSONDecodeError, OSError)",
"expected_behavior": "see fix",
"repro_steps": [],
"affected_workflow": "Operator-facing flow in spark-cli."
},
"evidence": {
"safe_links_only": true,
"before_after_proof": "Before: [spark-compete] fix(system_map): narrow bare except in 4 JSONL payload scanners to (json.JSONDecodeError, OSError)\nAfter: see fix",
"links": [
"https://github.com//pull/823",
"https://github.com//pull/823/files"
],
"forbidden": [
"raw secrets",
"raw logs",
"raw conversations",
"private chat IDs",
"session tokens",
"cookies",
"private repo maps",
"raw memory dumps",
"full compile JSON",
"scoring details"
]
},
"proposed_fix": {
"approach": "[spark-compete] fix(system_map): narrow bare except in 4 JSONL payload scanners to (json.JSONDecodeError, OSError)",
"files_expected": [],
"tests_or_smoke": "Smoke: exercise the affected code path; build-clean on the changed file."
},
"pr": {
"url": "#823",
"branch": "spark-compete/pr823",
"title_prefix": "[spark-compete]",
"author_github": "4gjnbzb4zf-sudo",
"body_must_include": [
"packet",
"team",
"pr_author",
"repo",
"actual_behavior",
"expected_behavior",
"repro_steps",
"before_after_proof",
"tests_or_smoke",
"duplicate_notes",
"risk_notes",
"review_claim"
]
},
"review_claim": {
"impact_claim": "low",
"evidence_types": [
"redacted_terminal_excerpt"
],
"duplicate_notes": "Searched open PRs and issues for the same defect; this fix is targeted to the affected file.",
"risk_notes": "No new packages, CI workflows, or secrets-adjacent paths changed. Diff is bounded to a single file. Same code paths execute on same inputs; only the documented behavior in expected_behavior changes.",
"review_state_requested": "pr_review"
}
}

[4gjnbzb4zf-sudo]

QA write-up — for reviewer eyes

TL;DR. Four JSONL payload scanners in src/spark_cli/system_map.py share the same two-tier try/except shape: an inner except Exception: around json.loads(line) that bumps a parse_errors counter, and an outer except Exception as exc: around the whole with path.open(...) block that writes out["error"]. The inner clause was designed to absorb malformed JSON lines; the outer clause was designed to absorb file-system faults. Both bare-Exception forms silently fold programmer bugs (an AttributeError from a typo in a helper, a TypeError from a signature drift, a KeyError from a refactor) into the same friendly counters. Narrowing the inner to except json.JSONDecodeError and the outer to except OSError keeps the documented absorption behavior intact and lets programmer bugs surface as themselves. The four scanners are: count_safe_jsonl, inspect_safe_jsonl_samples, inspect_spawner_authority_verdicts, build_spark_os_review_candidates.

Before

def count_safe_jsonl(path: Path) -> dict[str, Any]:
    ...
    try:
        with path.open("r", encoding="utf-8") as handle:
            for line in handle:
                ...
                try:
                    payload = json.loads(line)
                except Exception:
                    parse_errors += 1
                    continue
                ...
    except Exception as exc:
        out["error"] = f"{type(exc).__name__}: {exc}"
        return out

If a refactor inside the scanner body lands a typo (e.g. SENSITIVE_KEY_NAME_HINT instead of SENSITIVE_KEY_NAME_HINTS), the resulting NameError is folded into out["error"] and the operator sees a normal-looking system map with a stringified error tag, never learning the real bug.

After

                try:
                    payload = json.loads(line)
                except json.JSONDecodeError:
                    parse_errors += 1
                    continue
                ...
    except OSError as exc:
        out["error"] = f"{type(exc).__name__}: {exc}"
        return out

json.loads documents json.JSONDecodeError as its only exception on a malformed line. path.open(...) and iterating a file handle raise OSError subclasses on file-system faults (FileNotFoundError, IsADirectoryError, PermissionError). Except bodies are byte-identical pre/post.

Smoke

$ cd /tmp && mkdir scratch_jsonl && cd scratch_jsonl
$ PYTHONPATH=/path/to/spark-cli/src python3 -c "
import json
from pathlib import Path
from spark_cli.system_map import count_safe_jsonl

p = Path('events.jsonl')
p.write_text(json.dumps({'event_type': 'route_selected'}) + '\n' + '{malformed json' + '\n', encoding='utf-8')
print(count_safe_jsonl(p))
"

{'path': 'events.jsonl', 'exists': True, 'redaction': '...',
 'line_count': 2, 'parsed_count': 1, 'parse_errors': 1,
 'redacted_key_name_count': 0, 'top_keys': {'event_type': 1},
 'safe_value_counts': {'event_type': {'route_selected': 1}}}

(Documented behavior: 2 lines / 1 parsed / 1 parse_errors. JSONDecodeError still routes through the counter.)

$ PYTHONPATH=/path/to/spark-cli/src python3 -c "
from pathlib import Path
from spark_cli.system_map import count_safe_jsonl
# Pass a directory: triggers IsADirectoryError on path.open() inside the outer try.
print(count_safe_jsonl(Path('.')))
"

{'path': '.', 'exists': True, 'redaction': '...',
 'error': \"IsADirectoryError: [Errno 21] Is a directory: '.'\"}

(Documented behavior: OSError subclass still routes through out['error'].)

Scope

• 1 file changed for the fix: src/spark_cli/system_map.py (8 deletions, 8 insertions across 4 scanners).
• 1 test file: tests/test_system_map.py (2 new regression tests, 38 insertions).
• Full sweep python3 -m unittest tests.test_system_map -> 31 passed (29 pre-existing + 2 new).
• git diff --stat origin/master -> 2 files, 46 insertions, 8 deletions.

Disjoint with sibling PRs

gh pr list --repo vibeforge1111/spark-cli --search "narrow except system_map" --state all returns a cluster of sibling-narrowing PRs (#526 sqlite helpers, #532 read_json+read_toml, #533 spawner-prd-auto-trace, #534 git subprocess, #535 trace-groups+trace-health, #536 filesystem-walkers). None touch the four scanners covered here.

[4gjnbzb4zf-sudo]

TL;DR

Four JSONL payload scanners in system_map.py swallow programmer bugs as friendly per-line parse_errors counters or summary error strings After the fix: The inner counter increments only on actual malformed-JSON lines (json.JSONDecodeError -- the only exception json.loads documents itself as raising on a bad line).

What changes

Files touched: src/spark_cli/system_map.py, tests/test_system_map.py.

Why this matters

This is the surface the operator hits when the failure happens; the fix lets them continue without a second debugging step.

Reproduction (operator-side)

1. gh pr checkout
2. PYTHONPATH=src python -m unittest tests.test_system_map.SparkSystemMapTests.test_jsonl_scanners_let_attribute_errors_propagate -> ok (a patched Counter that raises AttributeError now propagates past the narrowed except json.JSONDecodeError on the inner line, instead of being swallowed by the old except Exception).
3. PYTHONPATH=src python -m unittest tests.test_system_map.SparkSystemMapTests.test_jsonl_scanner_reports_oserror_via_error_field -> ok (count_safe_jsonl called on a directory raises IsADirectoryError, which the narrowed except OSError still catches and surfaces as out['error']).
4. PYTHONPATH=src python -m unittest tests.test_system_map.SparkSystemMapTests.test_jsonl_counter_only_keeps_allowlisted_counts -> ok (the documented JSONDecodeError-on-malformed-line behavior is preserved: 4 lines / 3 parsed / 1 parse_errors).
5. Full sweep: PYTHONPATH=src python -m unittest tests.test_system_map -> 31 tests ok.

Verification

Review src/spark_cli/system_map.py for the targeted change. Run the reproduction; expected outcome: The inner counter increments only on actual malformed-JSON lines (json.JSONDecodeError -- the only exception json.loads documents itself as raising on a bad line).