-
-
Notifications
You must be signed in to change notification settings - Fork 0
Home
gaafa edited this page Apr 9, 2026
·
2 revisions
EvalCI is a zero-infra GitHub Action that runs @eval_case quality suites on every PR and blocks merge if scores drop below your threshold.
→ Full documentation: https://synapsekit.github.io/synapsekit-docs/docs/evalci/overview
| Page | Description |
|---|---|
| Overview | What EvalCI does and how it works |
| Quickstart | Get running in under 5 minutes |
| Writing Eval Cases |
@eval_case decorator reference |
| Action Reference | All inputs, outputs, and exit codes |
| Examples | RAG, agents, multi-provider, Slack notifications |
| Troubleshooting | Common errors and fixes |
# .github/workflows/eval.yml
- uses: SynapseKit/evalci@v1
with:
path: tests/evals
threshold: "0.80"
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}# tests/evals/eval_rag.py
from synapsekit import eval_case
@eval_case(min_score=0.80)
async def eval_my_pipeline():
result = await pipeline.ask("What is SynapseKit?")
return {"score": result.relevancy_score}