Skip to content

test(2275): add 429-retried allowlist contract + fix CHANGELOG citations [skip-runtime-e2e]#44

Merged
saurabhjain1592 merged 1 commit into
mainfrom
fix-2275-followup-changelog-and-429-test
May 20, 2026
Merged

test(2275): add 429-retried allowlist contract + fix CHANGELOG citations [skip-runtime-e2e]#44
saurabhjain1592 merged 1 commit into
mainfrom
fix-2275-followup-changelog-and-429-test

Conversation

@saurabhjain1592
Copy link
Copy Markdown
Member

Summary

Follow-up PR addressing two HIGH findings + two MEDIUM-priority asks from the hostile review of PR #42 (the regression-test PR for issue #2275).

Defensive / documentation only. No SDK code paths modified. No retry semantics changed. No version bump — under [Unreleased] in CHANGELOG.

HIGH findings fixed

HIGH 1 — CHANGELOG line-citation drift (visible on crates.io / docs.rs)

PR #42's [Unreleased] entry cited src/client.rs:517-525 for the retry-allowlist. The actual allowlist block sits at lines 560-565 in current main post-merge (further shifted by this PR's rustdoc addition). Lines 517-525 in current main are part of the check_status helper + execute_with_retry signature — not the allowlist at all. The cited code snippet was correct; just the line numbers were stale. The drift ships to crates.io docs.

Fix: updated CHANGELOG entry's src/client.rs:517-525src/client.rs:560-565.

HIGH 2 — No positive coverage that 429 IS retried (allowlist not regression-locked)

The new test_401_not_retried_issue_2275 only asserts 401 → terminal. There's no test for the other direction — that 429 still triggers retry (since 429 is in the allowlist {429, 402, 403}). A future refactor that drops *status != 429 from execute_with_retry would make every 4xx terminal, silently breaking the rate-limit retry contract, and the existing tests wouldn't catch it (the test_retry_logic test uses 500, not 429).

Fix: added test_429_is_retried_allowlist_contract in tests/integration_test.rs (next to test_401_not_retried_issue_2275, NOT a separate file). Mounts wiremock with .respond_with(ResponseTemplate::new(429)).expect(3), configures the client with max_attempts=3 + initial_delay=1ms, calls proxy_llm_call, asserts result.is_err(), asserts (implicitly via .expect(3)) that the mock was called exactly 3 times.

MEDIUM-priority asks addressed

MEDIUM 1 — Rustdoc on execute_with_retry

src/client.rs:532 (post-rustdoc) — added /// block describing the retry contract:

  • Retried statuses: 5xx, 429, transport-level errors (non-ApiError variants of AxonFlowError)
  • Terminal statuses: 401 (cites #2275) + every 4xx not in the {429, 402, 403} allowlist (enumerated)
  • Caveat block explaining the 402/403 dead-clause situation, citing execute_request and CHANGELOG.md for history

MEDIUM 2 — Allowlist clarity on 402/403

src/client.rs:586 returns 402 + 403 as Ok(client_resp) from execute_request — they're SUCCESS responses with policy/quota data. They never propagate as Err to execute_with_retry, so the *status != 402 and *status != 403 clauses on lines 563-564 are functionally dead.

Fix: kept the dead clauses (defense-in-depth + reads as intent for future hackers who don't realize 402/403 are success) and added a comment block above the allowlist explaining the situation. Don't delete — would be a behavior change if someone ever refactors execute_request to error on 402/403.

Mutation test — confirms the new 429 test isn't tautological

Per the brief's instruction, temporarily deleted the *status != 429 clause from execute_with_retry:

// MUTATION (temporary):
if *status >= 400
    && *status < 500
    && *status != 402
    && *status != 403
{
    return Err(e);
}

Re-ran the test:

test test_429_is_retried_allowlist_contract ... FAILED

failures:
    test_429_is_retried_allowlist_contract

	Expected range of matching incoming requests: == 3
	Number of matched incoming requests: 1

The mock was called exactly 1 time (immediate-terminal under the mutation) instead of 3 (full retry exhaustion under correct code). Wiremock's .expect(3) panicked on Drop. Restored the clause; test passes.

Result: assertion is independent of the SUT — failing for the right reason (no retry happened) under the mutation; passing for the right reason (3 retries happened) under correct code.

Hostile R3 self-review

  • Adversarial matcher review — wiremock's .expect(3) is checked at MockServer Drop, which happens at the end of the #[tokio::test] function scope. The assert!(result.is_err(), ...) assert before drop won't mask a mock-call-count violation (the panic on Drop fires regardless).
  • Cross-read with test_retry_logic — that test uses 500 + expect(2), doesn't probe 429. The new test fills the gap by mounting 429 + expect(3). Two complementary tests; no overlap, no contradiction.
  • Cross-read with test_401_not_retried_issue_2275 — that test uses 401 + expect(1). Together with the new test (429 + expect(3)), the pair brackets the allowlist boundary: out → terminal, in → retried. Either direction breaking is now caught.
  • Line citations re-verified against current source post-rustdocgrep -n "async fn execute_with_retry" src/client.rs returns 532; grep -n "if \*status >= 400" returns 560; grep -n "if status.is_success() || status.as_u16() == 402" returns 586. All three numbers cited in the CHANGELOG + in-code comment match the actual source.
  • fmt + clippy cleancargo fmt --all -- --check clean, cargo clippy --all-targets --all-features -- -D warnings clean.
  • Full test suite green — 19 integration tests + 4 x_client_id tests + 2 doctests pass on the worktree.

Skip-runtime-e2e justification

This PR is defensive / documentation only:

  • One new mock-based unit test in tests/integration_test.rs (uses wiremock, not the live community-stack agent)
  • One rustdoc block on execute_with_retry
  • One in-code comment block above the retry-allowlist
  • CHANGELOG line-number correction

No new feature surface, no wire contract changes, no SDK code paths modified, no retry semantics changed. The retry contract itself is unchanged from PR #42; this PR only adds a complementary test + documentation around behavior that already shipped.

runtime-e2e/x-client-id/ (the only runtime-e2e/ directory in this repo) is the wire-level companion to tests/x_client_id_header_test.rs for the v9 identity headers — completely orthogonal to the retry-contract surface this PR documents. No reason to mount the live agent to verify a mock-based retry-allowlist regression test.

Issue references

…ons [skip-runtime-e2e]

Addresses two HIGH findings from the hostile review of PR #42 + two
MEDIUM asks.

HIGH 1 — CHANGELOG line-citation drift. PR #42 cited
`src/client.rs:517-525` for the retry-allowlist. After the PR merged,
the actual allowlist block sits at lines 560-565 (further shifted by
this PR's rustdoc addition). Citation now points to the correct range.

HIGH 2 — No positive coverage that 429 IS retried. The new
`test_401_not_retried_issue_2275` only locked one direction of the
allowlist. A future refactor dropping `*status != 429` from
`execute_with_retry` would silently make every 4xx terminal — breaking
the rate-limit retry contract — and the existing tests wouldn't catch
it (`test_retry_logic` uses 500, not 429). New
`test_429_is_retried_allowlist_contract` brackets the other side via
`.expect(3)` against `max_attempts=3`.

MEDIUM 1 — Rustdoc on `execute_with_retry` describing the retry
contract: which statuses retry (5xx + 429 + transport errors), which
are terminal (401 + everything else 4xx outside the allowlist), with a
cross-reference to issue #2275 and CHANGELOG.md.

MEDIUM 2 — Clarifying comment above the retry-allowlist explaining
that 402/403 are returned as `Ok(client_resp)` from `execute_request`
(line 586) — they never reach `execute_with_retry` as `Err`, so the
`*status != 402` / `*status != 403` clauses are functionally dead BUT
kept as intent-preserving defense-in-depth for any future refactor.

Defensive / documentation only. No SDK code paths modified. No retry
semantics changed. No version bump — under `[Unreleased]` in CHANGELOG.

Mutation test on the new 429 test: temporarily deleted the
`*status != 429` clause from `execute_with_retry`; the test failed with
"Expected range of matching incoming requests: == 3, Number of matched
incoming requests: 1" (wiremock's `.expect(3)` panicked on Drop because
the SDK only called the mock once instead of retrying twice more).
Restored the clause; test passes. Confirms the assertion isn't
tautological.

Signed-off-by: Saurabh Jain <saurabh.jain@getaxonflow.com>
@saurabhjain1592 saurabhjain1592 merged commit 8a71525 into main May 20, 2026
9 checks passed
@saurabhjain1592 saurabhjain1592 deleted the fix-2275-followup-changelog-and-429-test branch May 20, 2026 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant