Analyze an agent trace from a customer support log. What followups are needed?
The workflow
Every action the agent took that cannot be undone is measured against what it was permitted to do at that point in the run. A breach pages on-call and the review ends there.
Did the task get done, and was the user satisfied? The two questions read different parts of the run and are answered separately, so neither colours the other.
The two answers select one of four outcomes: healthy, an expectation gap, an overt failure, or a silent failure. Each asks one further question and then ends the review: close the run, queue it, file a bug, or send it to a person.
A handful of cases: for each model, one where it alone differs from the other two; one where all three miss the reference; one where all three agree. Each model's path is drawn on the chart in its own colour and line style. Where paths share a stretch, the lines run side by side. What no model reached is faded. Underlined questions open below with every model's answer and the state it read.