-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
ai/corecore functions like generateText, streamText, etc. Provider utils, and provider spec.core functions like generateText, streamText, etc. Provider utils, and provider spec.featureNew feature or requestNew feature or requestpull request welcomeFor issues with clear instructions that welcome community contirbutionsFor issues with clear instructions that welcome community contirbutions
Description
whenever we change the prompt in tigent.yml, the system prompt, or any relevant code, we want to make sure we don't break current behavior.
Ideas we had so far
- Add tests to the code with webhook fixtures and desired outcome in form of labels
- Create a dedicated evaluations repository on the @tigent org with issues and labels, then use the issues as input (dynamically requested when running evaluations) and the applied labels as output
- Use the vercel/ai repository itself by adding
evaluationswhich is an array of numbers and use issues/PRs on the ai repository as evals
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
ai/corecore functions like generateText, streamText, etc. Provider utils, and provider spec.core functions like generateText, streamText, etc. Provider utils, and provider spec.featureNew feature or requestNew feature or requestpull request welcomeFor issues with clear instructions that welcome community contirbutionsFor issues with clear instructions that welcome community contirbutions