feat: prompt scorer, tracer adjustments, logging, and error handling#11
Merged
Delve Auditor / Security Check
succeeded
Sep 18, 2025 in 25s
Scan completed
No vulnerabilities found.
Details
✅ No security or compliance issues detected. Reviewed everything up to caca58b.
Security Overview
- 🔎 Scanned files: 23 changed file(s)
Detected Code Changes
| Change Type | Relevant files |
|---|---|
| Configuration changes | ► pom.xml Bump Java SDK version 0.1.9 → 0.2.0 |
| Enhancement | ► src/main/java/com/judgmentlabs/judgeval/data/TraceEvaluationRun.java Add TraceEvaluationRun model for trace-based evaluations ► src/main/java/com/judgmentlabs/judgeval/tracer/Tracer.java Add asyncTraceEvaluate, trace eval serialization, logging, and ObjectMapper; improve asyncEvaluate error handling ► src/main/java/com/judgmentlabs/judgeval/tracer/OpenTelemetryKeys.java Add PENDING_TRACE_EVAL attribute key ► src/main/java/com/judgmentlabs/judgeval/internal/api/models/OtelTraceSpan.java Add OtelTraceSpan model (new) |
| Refactor | ► src/main/java/com/judgmentlabs/judgeval/data/ExampleEvaluationRun.java Refactor to use ScorerConfig list only; remove mixed/local scorer handling and builders; simplify validation and builders ► src/main/java/com/judgmentlabs/judgeval/data/Example.java Set default exampleId to a generated UUID and import UUID ► src/main/java/com/judgmentlabs/judgeval/scorers/BaseScorer.java Change toTransport signature to return ScorerConfig; import model type ► src/main/java/com/judgmentlabs/judgeval/scorers/APIScorer.java Update toTransport return type to ScorerConfig |
| API model changes | ► src/main/java/com/judgmentlabs/judgeval/internal/api/models/FetchPromptScorersRequest.java Rename/modify request to accept names (List) instead of a single name ► src/main/java/com/judgmentlabs/judgeval/internal/api/models/FetchPromptScorersResponse.java Rename/modify response to return list of PromptScorer objects (scorers) instead of a single scorer ► src/main/java/com/judgmentlabs/judgeval/internal/api/models/ScorerData.java Add id field and change evaluationModel type to String; update equals/hashCode and getters/setters |
| Internal API client updates | ► src/main/java/com/judgmentlabs/judgeval/internal/api/JudgmentSyncClient.java Rename fetchScorer → fetchScorers and update endpoint to /fetch_scorers/ and response type ► src/main/java/com/judgmentlabs/judgeval/internal/api/JudgmentAsyncClient.java Rename fetchScorer → fetchScorers and update endpoint to /fetch_scorers/ and response type |
| New scorer framework & features | ► src/main/java/com/judgmentlabs/judgeval/scorers/api_scorers/prompt_scorer/BasePromptScorer.java Introduce base class for prompt scorers with fetch/push helpers, push handling for trace flag, and common setters/getters ► src/main/java/com/judgmentlabs/judgeval/scorers/api_scorers/prompt_scorer/PromptScorer.java Add PromptScorer implementation using BasePromptScorer (non-trace prompt scorer) ► src/main/java/com/judgmentlabs/judgeval/scorers/api_scorers/prompt_scorer/TracePromptScorer.java Add TracePromptScorer implementation (trace-aware prompt scorer) |
| Removals / cleanup | ► src/main/java/com/judgmentlabs/judgeval/scorers/api_scorers/PromptScorer.java Remove legacy PromptScorer (replaced by new package implementations) ► src/main/java/com/judgmentlabs/judgeval/internal/api/models/Tool.java Delete Tool model (removed) ► src/main/java/com/judgmentlabs/judgeval/internal/api/models/TraceSpan.java Delete TraceSpan model (replaced by OtelTraceSpan) ► src/main/java/com/judgmentlabs/judgeval/internal/api/models/TraceUsage.java Delete TraceUsage model |
| Script update | ► scripts/generate_client.py Update client generation paths to use /fetch_scorers/ endpoint name |
| Other | ► src/main/java/com/judgmentlabs/judgeval/internal/api/models/FetchPromptScorersRequest.java and FetchPromptScorersResponse.java (renames) Adjust JSON properties and equality/hashCode implementations to support list-based fetch semantics |
Reply to this PR with @delve-auditor followed by a description of what change you want and we'll auto-submit a change to this PR to implement it.
Loading