feat(obs): request correlation IDs across API and workers (E4)#56
Merged
Conversation
Audit M-7: no request id tied API logs to the Celery work they enqueue. - app/observability/request_id.py: a request-id contextvar, helpers, a logging filter, and install_request_id_logging() that injects [req:<id>] into existing handlers' formatters (no duplicate handlers). - main.py: HTTP middleware binds a request id (from X-Request-Id or fresh), echoes it on the response, and installs the log format on startup. - app/observability/tasking.py: enqueue() — drop-in for task.delay() that propagates the id via Celery task headers. - app/worker.py: task_prerun re-binds the propagated id (fresh fallback); after_setup_logger formats worker logs the same way. - All task enqueues (ingest x2, embed, index, reconcile) now use enqueue(). Live-validated: response carries X-Request-Id; a supplied id is echoed; uvicorn access logs show [req:<id>] with no duplication. Tests: tests/test_request_id.py. Full default suite: 304 passed. Scope note: monitor state was NOT moved to Redis — the monitor is a single daemon process (not per-worker), so its atomic file store is correct; a Redis migration would add complexity without benefit. Closes #23
CognitiveCodeAI
added
WS-E
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
E4 — request correlation IDs (audit M-7)
Part of the remediation (
#25). Ties a single request's logs together across the API and the Celery work it enqueues.app/observability/request_id.py— request-id contextvar + helpers + a logging filter, andinstall_request_id_logging()that injects[req:<id>]into existing handlers' formatters (no duplicate handlers).main.py— HTTP middleware binds a request id (fromX-Request-Idor a fresh one), echoes it on the response, and installs the log format on startup.app/observability/tasking.py—enqueue(), a drop-in fortask.delay()that propagates the id via Celery task headers.app/worker.py—task_prerunre-binds the propagated id (fresh fallback);after_setup_loggerformats worker logs identically.enqueue().Validation
Live: response carries
X-Request-Id; a supplied id is echoed back; uvicorn access logs show[req:<id>]with no duplication.tests/test_request_id.py. Full default suite: 304 passed.Scope decision
The monitor state was not moved to Redis. The monitor is a single daemon process (not per-API-worker), so its atomic file store is correct — a Redis migration would add complexity without benefit. (Documented rather than done; the M-6 "multi-worker" concern doesn't apply to a single-daemon design.)
Closes #23