Conversation
| This guide covers **online evaluation**, which includes: | ||
|
|
||
| The foundation of Weave's evaluation system is the [**Scorer**](./scorers) - a component that evaluates your function's inputs and outputs to measure quality, safety, or any other metric you care about. Scorers are versatile and can be used in two ways: | ||
| - **Monitors**: Passively score production traffic to surface trends and issues. No code changes required. |
There was a problem hiding this comment.
to surface trends and issues.
How do we surface trends and issues? Is there some kind of alerting mechanism somewhere in the UI? Are users supposed to write scripts to periodically query the monitors? Should someone be manually reviewing a page in the UI somewhere?
It feels like something is missing here. Can we offer the user some guidance?
| @@ -1,678 +1,438 @@ | |||
| --- | |||
| title: "Set up guardrails and monitors" | |||
| title: "Set up monitors and guardrails" | |||
There was a problem hiding this comment.
Reading through the main page in a linear manner, it's a little confusing to me why there are implementation details of both monitors and guardrails. They seem rather different, from an implementation standpoint. Setting up a guardrail doesn't seem like a natural next step after setting up a monitor.
The high-level comparison of monitors and guardrails makes a lot of sense to me. I wonder if it would make sense to keep a high-level overview here and the split out each of the detail pages.
dbrian57
changed the title
📚 Mintlify Preview Links✨ Added (2 total)📄 Pages (2)
📝 Changed (2 total)📄 Pages (1)
⚙️ Other (1)
🗑️ Deleted (1 total)View deleted files📄 Pages (1)
🤖 Generated automatically when Mintlify deployment succeeds |
2 participants
Description
Resolves DOCS-1784. This rewrites the Weave monitors and guardrails doc to: