Metrics & Alerts

MindReef automatically collects and aggregates metrics from your agent traces. Monitor performance, track costs, and get alerted when issues arise.

Available Metrics

The following metrics are automatically computed from your trace data:

Metric	Description
`trace.count`	Total number of traces
`trace.success_rate`	Percentage of traces that completed without errors
`trace.error_rate`	Percentage of traces that failed
`trace.latency.p50`	Median trace duration
`trace.latency.p95`	95th percentile trace duration
`trace.latency.p99`	99th percentile trace duration
`llm.token.input`	Total input tokens consumed
`llm.token.output`	Total output tokens generated
`llm.cost`	Estimated API costs
`hallucination.flagged_rate`	Percentage of responses flagged for hallucination

Viewing Metrics

Access metrics in the dashboard under the Metrics tab. You can:

View time-series charts for any metric
Filter by agent, environment, or custom tags
Compare metrics across different time periods
Export data for external analysis

Creating Alerts

Set up alerts to get notified when metrics cross thresholds. Navigate to Settings → Alerts → New Alert.

Alert Configuration

{
    "name": "High Error Rate",
    "metric": "trace.error_rate",
    "condition": "greater_than",
    "threshold": 5,
    "window_minutes": 15,
    "channels": ["slack", "email"]
}

Common Alert Examples

Error Rate Spike:

{
    "name": "Error Rate Spike",
    "metric": "trace.error_rate",
    "condition": "greater_than",
    "threshold": 10,
    "window_minutes": 5
}

Latency Degradation:

{
    "name": "Slow Responses",
    "metric": "trace.latency.p95",
    "condition": "greater_than",
    "threshold": 5000,  // 5 seconds
    "window_minutes": 10
}

Cost Budget:

{
    "name": "Daily Cost Limit",
    "metric": "llm.cost",
    "condition": "greater_than",
    "threshold": 100,  // $100
    "window_minutes": 1440  // 24 hours
}

Notification Channels

MindReef supports the following notification channels:

Email: Sent to team members or distribution lists
Slack: Post to a channel via webhook
Webhook: HTTP POST to your custom endpoint

Slack Integration

Connect Slack in Settings → Integrations → Slack. You'll need to create an incoming webhook in your Slack workspace.

Custom Metrics

Record custom metrics from your code:

from mindreef import record_metric

# Record a custom metric
record_metric("user.satisfaction_score", 4.5)
record_metric("agent.steps_taken", 7)