[spark-compete] make providers status/test/doctor agree on openai+codex auth mode (stop false [OK] READY)

[banse]

[spark-compete] make providers status/test/doctor agree on openai+codex auth mode (stop false [OK] READY)

team

The Dudes — members: His Dudeness, El Duderino, Duder. LLM device holder: His Dudeness (github: banse).

pr_author

banse

repo

vibeforge1111/spark-cli (owner surface: spark-cli). Base branch: master. Branched from upstream baseline fc49c16.

actual_behavior

With the openai chat role configured against the default OpenAI base_url, no OPENAI_API_KEY set, and the codex CLI present/signed in, the three derivations of the openai chat-role auth_mode disagree:

• provider_auth_mode (used by build_llm_env at setup time) has no codex fallback for the openai role, so spark setup persists llm.roles.chat.auth_mode = "not_configured" into setup.json.
• provider_status_payload and build_llm_repair_hints re-derive auth_mode = "codex_oauth" via a codex fallback that setup never applied. So spark providers status prints [OK] chat provider=openai ... auth=codex_oauth, ready=True, zero repair hints, exit 0. spark doctor llm inherits the same false-OK via collect_llm_doctor_context.
• resolve_llm_doctor_target reads the raw persisted auth_mode ("not_configured"), so its openai+codex branch is never taken and it raises SystemExit → spark providers test --role chat reports [FIX], exit 1.

Net: a verify/health check (providers status, doctor llm) reports the chat provider fully READY while the actual configured runtime is unauthenticated and providers test fails — a verify check that passes when it should fail (diagnostic-trust / silent-wrong-result).

expected_behavior

providers status and doctor llm must agree with providers test for the same role. With openai+default-base+no-key the chat role is not authenticated, so status should report it not ready and surface the existing OPENAI_API_KEY-not-configured repair hint. Concretely: provider_status_payload(...).roles.chat.ready == provider_test_payload(...).ok (both False here).

repro_steps

1. Configure the openai chat role with the default OpenAI base_url and no OPENAI_API_KEY (auth_mode persisted as not_configured by spark setup).
2. Ensure the codex CLI is present/signed in (detect_codex_cli present).
3. Run spark providers status (or spark doctor llm): chat shows [OK] ready=True auth=codex_oauth, exit 0, no repair hints.
4. Run spark providers test --role chat: it reports [FIX] and exits 1 because resolve_llm_doctor_target sees the raw not_configured auth_mode and raises SystemExit.
5. Observe the contradiction: status/doctor say READY, test says not configured.

before_after_proof

Reporting-side, behavior-preserving fix (no runtime routing change): remove the openai+default-base codex_oauth over-promotion in provider_status_payload (cli.py) and build_llm_repair_hints (cli.py) so both reporting derivations leave the openai role auth_mode as not_configured — matching what spark setup/provider_auth_mode persists and what resolve_llm_doctor_target reads. This makes provider_status_payload(ready) == provider_test_payload(ok) and re-emits the existing OPENAI_API_KEY-not-configured repair hint.

The reporting-side fix is correct because resolve_llm_doctor_target gates its openai+codex branch on the raw persisted auth_mode (not_configured), so codex routing is not actually wired for the openai chat role at runtime.

Deterministic fail-pre / pass-post (source-only revert, redacted terminal excerpt):

# BEFORE fix (src/spark_cli/cli.py reverted to baseline fc49c16; test present):
E       AssertionError: True != False
FAILED tests/test_cli.py::SparkCliTests::test_provider_status_ready_matches_provider_test_for_openai_default_base_no_key
1 failed in 0.44s

# AFTER fix (fix restored):
1 passed in 0.26s

(status ready=True vs test ok=False diverge on baseline → True != False; after the fix both are False.)

tests_or_smoke

tests/test_cli.py::SparkCliTests::test_provider_status_ready_matches_provider_test_for_openai_default_base_no_key — deterministic unit test that monkeypatches detect_codex_cli present (no real codex CLI needed) and stubs codex_cli_auth_payload, builds the openai+default-base+no-key role state, and asserts provider_status_payload(...).roles.chat.ready == provider_test_payload(...).ok (both False after the fix; they diverge on baseline). Mirrors the existing test_provider_test_explicit_codex_uses_codex_oauth_defaults style.

Run targeted (Python 3.12 venv; full suite has pre-existing macOS TCC/reparse-point env failures unrelated to this change):

.venv-test/bin/python -m pytest 'tests/test_cli.py::SparkCliTests::test_provider_status_ready_matches_provider_test_for_openai_default_base_no_key' -q

120 provider/codex/doctor/status/llm tests pass with the fix; the only 2 failures (telegram relay secret repair, save_codex_client_config) fail identically on the stashed baseline and are pre-existing macOS env issues, not regressions.

duplicate_notes

Swept ~500 open upstream PRs by file+symbol+mechanism. None touch the provider_status_payload / build_llm_repair_hints openai+default-base codex_oauth over-promotion vs provider_test/provider_auth_mode consistency. Closest neighbors confirmed distinct:

• [codex] fix Codex OAuth provider status model #756 surfaces the codex_client model for roles already in codex_oauth (codex_client block ~11664-11672), not the openai default-base derivation.
• [spark-compete] fix(providers): live API probe stops false [OK] on failing providers #494 adds a live API probe for failing providers and threads secret_keys into repair hints; its openai branch handles local base_url, not the default-base codex over-promotion.
• Report unconfigured provider roles #180 adds unconfigured-role hints to the not-configured early-return path.
• [spark-compete] fix(cli): spark providers status, live status, and verify --onboarding report contradictory readiness #742 is README-only; docs: surface mission provider mismatch — uses OpenAI Codex when Anthropic configured #816 is docs-only.

Also distinct from our own CLI-1 #619 (system_map TCC), CLI-REVOKE #751 (revoke-all SystemExit), CLI-SETUP #713 (setup restart-to-apply).

risk_notes

Minimal and surgical: removes two identical codex_oauth over-promotion branches in the reporting derivations only; no change to runtime routing (build_llm_env, provider_auth_mode, resolve_llm_doctor_target). Effect is limited to the openai role with default base_url and no API key when the codex CLI is present — previously a false [OK]; status now matches providers test and emits the pre-existing repair hint. Genuine codex-provider configuration (provider == codex) is unaffected.

review_claim

Impact: medium (diagnostic-trust / silent-wrong-result in a verify/health check). Evidence types: failing_test, passing_test, redacted_terminal_excerpt. Review state requested: pr_review.

packet

Schema spark-compete-hotfix-v1. Validated at POST /api/packet/validate → status: "pass" (0 errors, 0 warnings).

{
  "schema": "spark-compete-hotfix-v1",
  "event": "spark-compete-first-event",
  "team": {
    "name": "The Dudes",
    "members": ["His Dudeness", "El Duderino", "Duder"],
    "llm_device_holder": "His Dudeness",
    "device_holder_github": "banse",
    "github_accounts": ["banse"]
  },
  "submission_mode": "public_repo_pr",
  "submission_target_url": "https://github.com/vibeforge1111/spark-cli/pull/1249",
  "target_repo": {
    "id": "vibeforge1111/spark-cli",
    "source": "https://github.com/vibeforge1111/spark-cli",
    "owner_surface": "spark-cli"
  },
  "issue": {
    "type": "bug",
    "severity": "medium",
    "title": "providers status/doctor report chat provider [OK] READY while providers test fails for openai+codex (default base, no API key)",
    "actual_behavior": "With the openai chat role configured against the default OpenAI base_url, no OPENAI_API_KEY set, and the codex CLI present/signed in, `spark setup` persists llm.roles.chat.auth_mode=\"not_configured\" (provider_auth_mode applies no codex fallback for the openai role). But provider_status_payload and build_llm_repair_hints re-derive auth_mode=\"codex_oauth\" via a codex fallback that setup never applied, so `spark providers status` and `spark doctor llm` print `[OK] chat provider=openai ... auth=codex_oauth`, ready=True, zero repair hints, exit 0. Meanwhile resolve_llm_doctor_target reads the RAW persisted auth_mode (\"not_configured\"), never takes its openai+codex branch, and raises SystemExit, so `spark providers test --role chat` reports [FIX] and exits 1. A verify/health check passes while the configured runtime is actually unauthenticated.",
    "expected_behavior": "providers status and doctor llm must agree with providers test for the same role: with openai+default-base+no-key, the chat role is not authenticated, so status should report the role as not ready and surface the existing OPENAI_API_KEY-not-configured repair hint (provider_status_payload(...).ready == provider_test_payload(...).ok). A health check must not report READY when the runtime cannot actually be called.",
    "repro_steps": [
      "Configure the openai chat role with the default OpenAI base_url and no OPENAI_API_KEY (auth_mode persisted as not_configured by spark setup).",
      "Ensure the codex CLI is present/signed in (detect_codex_cli present).",
      "Run `spark providers status` (or `spark doctor llm`): chat shows [OK] ready=True auth=codex_oauth, exit 0, no repair hints.",
      "Run `spark providers test --role chat`: it reports [FIX] and exits 1 because resolve_llm_doctor_target sees the raw not_configured auth_mode and raises SystemExit.",
      "Observe the contradiction: status/doctor say READY, test says not configured."
    ],
    "affected_workflow": "LLM provider verification / health diagnostics (`spark providers status`, `spark providers test`, `spark doctor llm`)."
  },
  "evidence": {
    "safe_links_only": true,
    "before_after_proof": "Deterministic unit test test_provider_status_ready_matches_provider_test_for_openai_default_base_no_key (tests/test_cli.py) monkeypatches detect_codex_cli present and stubs codex_cli_auth_payload, builds the openai+default-base+no-key role state, and asserts provider_status_payload(...).roles.chat.ready == provider_test_payload(...).ok. BEFORE fix (source stashed): FAIL with `AssertionError: True != False` at tests/test_cli.py (status ready=True, test ok=False diverge). AFTER fix: PASS (both False; repair hints present). Source-only stash used to prove fail-pre/pass-post; the 2 unrelated macOS TCC/reparse-point failures (telegram relay secret, save_codex_client_config) fail identically on the stashed baseline and are pre-existing env issues, not regressions.",
    "links": [
      "https://github.com/vibeforge1111/spark-cli/pull/1249"
    ],
    "forbidden": [
      "No real secrets, API keys, OAuth tokens, or OPENAI_API_KEY values are included; the test uses synthetic state and stubs the codex auth payload.",
      "No real usernames, personal data, or real filesystem paths of real people are included; inputs are synthetic fixtures."
    ]
  },
  "proposed_fix": {
    "approach": "Reporting-side, behavior-preserving fix (no runtime routing change). Remove the openai+default-base codex_oauth over-promotion in provider_status_payload and build_llm_repair_hints so both reporting derivations leave the openai role auth_mode as not_configured, matching what spark setup/provider_auth_mode persists and what resolve_llm_doctor_target reads. This makes provider_status_payload(ready) == provider_test_payload(ok) for openai+default-base+no-key and re-emits the existing OPENAI_API_KEY-not-configured repair hint. Confirmed reporting-side is correct because resolve_llm_doctor_target gates its openai+codex branch on the RAW persisted auth_mode (not_configured), so codex routing is not actually wired for the openai chat role at runtime.",
    "files_expected": [
      "src/spark_cli/cli.py",
      "tests/test_cli.py"
    ],
    "tests_or_smoke": "tests/test_cli.py::SparkCliTests::test_provider_status_ready_matches_provider_test_for_openai_default_base_no_key — deterministic; monkeypatches detect_codex_cli (no real codex needed) and stubs codex_cli_auth_payload. FAILS on baseline (AssertionError: True != False), PASSES after the fix. Run targeted (Python 3.12 venv): `.venv-test/bin/python -m pytest 'tests/test_cli.py::SparkCliTests::test_provider_status_ready_matches_provider_test_for_openai_default_base_no_key' -q`."
  },
  "pr": {
    "branch": "fix/providers-status-codex-auth-consistency",
    "title_prefix": "[spark-compete]",
    "author_github": "banse",
    "body_must_include": [
      "packet",
      "team",
      "pr_author",
      "repo",
      "actual_behavior",
      "expected_behavior",
      "repro_steps",
      "before_after_proof",
      "tests_or_smoke",
      "duplicate_notes",
      "risk_notes",
      "review_claim"
    ],
    "url": "https://github.com/vibeforge1111/spark-cli/pull/1249"
  },
  "review_claim": {
    "impact_claim": "medium",
    "evidence_types": ["failing_test", "passing_test", "redacted_terminal_excerpt"],
    "duplicate_notes": "Swept ~500 open upstream PRs by file+symbol+mechanism. None touch the provider_status_payload / build_llm_repair_hints openai+default-base codex_oauth over-promotion vs provider_test/provider_auth_mode consistency. Closest neighbors confirmed distinct: #756 surfaces the codex_client model for roles already in codex_oauth (codex_client block ~11664-11672, not the openai default-base derivation); #494 adds a live API probe for failing providers and threads secret_keys into repair hints (its openai branch handles local base_url, not the default-base codex over-promotion); #180 adds unconfigured-role hints to the not-configured early-return path; #742 is README-only; #816 is docs-only. Also distinct from our own CLI-1 #619 (system_map TCC), CLI-REVOKE #751 (revoke-all SystemExit), CLI-SETUP #713 (setup restart-to-apply).",
    "risk_notes": "Minimal and surgical: removes two identical codex_oauth over-promotion branches in the REPORTING derivations only; no change to runtime routing (build_llm_env, provider_auth_mode, resolve_llm_doctor_target). Effect is limited to the openai role with default base_url and no API key when the codex CLI is present, which previously produced a false [OK]; status now matches providers test and emits the pre-existing repair hint. Genuine codex-provider configuration (provider==codex) is unaffected. 120 provider/codex/doctor/status/llm tests pass; the only 2 failures are pre-existing macOS TCC/reparse-point env issues unrelated to this change.",
    "review_state_requested": "pr_review"
  }
}