Triage
Stop drowning in traces.
Triage by pattern.
benchmax clusters production failures into a small number of reviewable issues. Your team reviews what matters, not thousands of conversations.
01·Skills
Detectors you define, running on every trace
Write a skill in plain English. It scans every conversation for matching failures — hallucinations, tool loops, context drops, over-refusals.
Skill · Draftskill_007
Name
Billing answer must cite page numbers
Detects
Flag any response that references a billing or refund policy without a specific page number from the source document.
SeverityHigh
Last 24h+12 hits
14:07· karri
Refund window is 30 days per our policy.
13:42· rohan
Prorated based on billing cycle.
12:58· mei
Cancellations process at end of term.
11:31· jayden
See billing terms for details.
02·Clustering
One issue, many traces — not the other way around
benchmax collapses repeated failures into a single reviewable issue with every occurrence attached. Review the pattern once, not 47 times.
Unsupported facts in billing answers
hv-824·Skill · Billing must cite pages·47 occurrences
Occurrences · 47sorted · most recent
2:14 PMkarri...refund window is 30 days per our policy...
1:42 PMrohan...prorated based on billing cycle...
12:58 PMmei...cancellations process at end of term...
11:31 AMjayden...see billing terms for details on that...
10:09 AMsofia...our policy allows up to 3 refunds...
+ 42 more
03·Full context
Every signal your agent saw
Review whole conversations including screenshots, videos, and uploaded documents. Multi-modal failures don't stay hidden.
Trace · karri · 2:14 PM
K
I turned off all notifications but I'm still getting the daily digest
K
settings.png
1280×720
1280×720
b
analyze_image·2 results
All your notifications are turned off in settings.
Context
Turns5
Tool calls3
Modalitiestext · img
Latency2.4s
Modelclaude-4-7
Attachments
imgsettings.png1280×720
pdfnotifications.pdfp. 4
04·Shared inbox
PM, domain expert, engineer — same view
Triage, comment, and assign together. No separate eval tool, no separate spreadsheet.
Triage · 8
S
M
P
3 onlineS
Unsupported facts in billing answersHigh
hv-824 · 47 occurrences
Sarah:nice catch — turning this into a test
M
Agent loops on password-reset flowBlocker
hv-821 · 12 occurrences
Mike:repro'd on staging. priority for release gate.
P
Tone too formal in chat welcomeLow
hv-819 · 34 occurrences