feat: add opik tracing wiring#34
Conversation
There was a problem hiding this comment.
Pull request overview
This PR implements Opik tracing integration to enable observability across the planning pipeline. It adds tracing hooks at each major step of the "Sandwich Architecture" (Extract → Plan → Validate → Repair) and begins logging validation metrics to trace spans.
Changes:
- Added Opik tracing decorators to orchestration functions in routes.py for extraction, planning, validation, and repair steps
- Moved tracing concern from business logic layer (planner.py) to orchestration layer (routes.py)
- Added metadata logging for validation metrics and request variant
- Corrected default Opik project name from "Hackaton" to "PlanProof"
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| apps/api/src/planproof_api/observability/opik.py | Fixed default project name spelling |
| apps/api/src/planproof_api/routes.py | Added tracing decorators, wrapper functions for tracing, and metadata logging for validation metrics and request variant |
| apps/api/src/planproof_api/agent/extractor.py | Renamed trace from "extract_metadata" to "extraction_step" for consistency |
| apps/api/src/planproof_api/agent/planner.py | Removed tracing decorator (moved to orchestration layer) |
| docs/assistant_prompts/codex_tasks.md | Marked multiple tasks as complete (PR 2.3, 3.3, and 4.1) |
| CONTRIBUTING.md | Marked multiple tasks as complete (PR 3.3 and 4.1) |
| - [X] **PR 2.3:** Implement `eval/recall.py` for Keyword Recall score calculation (deterministic string match). | ||
| - [X] **PR 2.4:** Add unit tests for all validators in `apps/api/tests/`. | ||
|
|
||
| ## Phase 3 — Agent: The Sandwich Pipeline | ||
|
|
||
| - [X] **PR 3.1:** Implement the "Extractor" logic (LLM call to parse constraints and keywords). | ||
| - [X] **PR 3.2:** Wire the Validator to run after the Planner and populate `validation.status` and `errors`. | ||
| - [ ] **PR 3.3:** Implement the 1-shot "Repair Attempt" logic (if FAIL, retry once with errors in prompt). | ||
| - [X] **PR 3.3:** Implement the 1-shot "Repair Attempt" logic (if FAIL, retry once with errors in prompt). | ||
|
|
||
| ## Phase 4 — Observability | ||
|
|
||
| - [ ] **PR 4.1:** Integrate Opik tracing hooks for each step (Extract -> Plan -> Validate -> Repair). | ||
| - [X] **PR 4.1:** Integrate Opik tracing hooks for each step (Extract -> Plan -> Validate -> Repair). |
There was a problem hiding this comment.
This PR marks multiple tasks as complete (PR 2.3, PR 3.3, and PR 4.1), but according to the Micro-PR Mandate, "One Checkbox = One PR." If PR 2.3 and PR 3.3 were completed in previous PRs, they should have been marked complete in those respective PRs. If they were never completed before, this PR should not include their implementation. This PR should only mark PR 4.1 as complete, as it implements the Opik tracing scaffolding.
| - [X] PR 3.3: 1-Shot Repair Loop | ||
| - [X] PR 4.1: Opik Trace Scaffolding |
There was a problem hiding this comment.
This PR marks multiple tasks as complete (PR 3.3 and PR 4.1), but according to the Micro-PR Mandate, "One Checkbox = One PR." If PR 3.3 was completed in a previous PR, it should have been marked complete in that PR. If it was never completed before, this PR should not include its implementation. This PR should only mark PR 4.1 as complete, as it implements the Opik tracing scaffolding.
| "human_feasibility_flags": human_feasibility_flags, | ||
| } | ||
| ) | ||
| except Exception: |
There was a problem hiding this comment.
'except' clause does nothing but pass and there is no explanatory comment.
| def create_plan(request: PlanRequest) -> PlanResponse: | ||
| try: | ||
| opik_context.update_current_trace(metadata={"variant": request.variant}) | ||
| except Exception: |
There was a problem hiding this comment.
'except' clause does nothing but pass and there is no explanatory comment.
| except Exception: | |
| except Exception: | |
| # Telemetry/tracing must never break request handling; ignore failures here. |
Add opik traces skaffold