Performance: pooled HTTP client + JWT verify cache + benchmarks#36
Merged
Conversation
Informed by researching how the popular harnesses stay fast, applied to the hot paths and measured, not claimed. - HttpxClient holds one httpx.Client and reuses it (keep-alive pooling) instead of a fresh connection per request; repeated calls to a service skip the TLS handshake. ~5x on loopback, more over real TLS. Adds close() + http2 flag. - TokenVerifier caches a verified Principal by token until its exp, skipping the RSA check on a repeated bearer (~160x on the hot path). Safe: a token verifies identically until expiry and no revocation list is consulted. cache_size=0 disables it. - evals/benchmark.py: throughput micro-benches (verify cached/uncached, gating, FLEX, vault) + loopback pooled-vs-unpooled HTTP. README Performance section with the numbers. Async/uvloop deliberately skipped: a 2.0 rewrite for a sync core, not a quick win; documented as a non-goal. 273 tests (+6), mypy --strict, ruff (whole repo), catalog freshness all green. v1.3.0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes "competitive performance" a measured claim, from researching how the popular harnesses stay fast and applying it to hallpass's hot paths. Two changes, both benchmarked.
HttpxClientholds onehttpx.Clientand reuses it across calls (keep-alive connection pooling) instead of opening a fresh connection per request. Repeated calls to a service skip the TCP + TLS handshake. Addsclose()and an optionalhttp2=flag.TokenVerifiercaches a verifiedPrincipalkeyed by the token until itsexp, skipping the RSA signature check on a repeated bearer. Safe: a token verifies identically until it expires and JWT verification consults no revocation list, so the cache is not observable. Bounded;cache_size=0disables it.Measured with the new
evals/benchmark.py:A repeated token is ~160x cheaper than a cold RSA verify; pooling is ~5x on loopback and more over real TLS. Async/uvloop was deliberately skipped and documented as a non-goal (a 2.0 rewrite for a synchronous core, not a quick win).
6 new tests: verify-cache skips re-verification (proved with a call-counting JWKS),
cache_size=0disables it, a different token still verifies, the client pools one connection and resets onclose(), and a benchmark sanity run. Two pre-existingHttpxClientbody-handling tests updated for the pooled path. 273 tests, mypy --strict, ruff (whole repo), catalog-freshness green. Backward compatible; v1.3.0.