AI Agent Failure Attribution: What New Research Shows | Odella Blog

Why "Which Step Broke the Agent?" Is Still Mostly a Human Job

Ask any team running AI agents in production the same question and you'll get the same wince: when a multi-step agent run fails, figuring out which step actually caused it — and why — usually means a human opening the trace and reading line by line. As agents take on longer, more complex workflows, that manual triage doesn't scale.

A new benchmark, Who&When Pro: Can LLMs Really Attribute Failures in AI Agents? by Jiale Liu, Huajun Xi, Shaokun Zhang, Yifan Zeng, Tianwei Yue, Chi Wang, Jian Kang, Qingyun Wu, and Huazheng Wang (Penn State University, AG2ai Inc., Mohamed bin Zayed University of Artificial Intelligence, Oregon State University, and Mathos AI), tests exactly this: can an LLM automatically play detective on a failed agent run and correctly identify the responsible step and the reason it went wrong? The honest answer, per their results, is: not yet — but the gap is narrowing in useful, specific ways.

What the benchmark measures

Who&When Pro is the largest and most diverse benchmark of its kind so far:

12,326 failure traces — roughly 5–65x larger than prior benchmarks in this space
26 source benchmarks spanning 9 task categories (QA, coding, data science, GUI interaction, video understanding, and more)
3 modalities — text, image, and video — the first attribution benchmark to cover all three
15 agent frameworks, both single-agent and multi-agent
18 distinct failure modes, grouped into six families: Perception, Reasoning, Planning, Action, Verification, and Coordination

The clever part is how they built it. Rather than annotating failures after the fact (noisy, expensive, and hard to verify), the team injected controlled errors into successful agent runs: take a trajectory that worked, replace one correct action with a realistic wrong one at a chosen step, replay the identical context up to that point, and let the agent continue naturally. Because reverting that single injected step would have produced success, the injected step is by construction the ground-truth cause of failure — no ambiguity, no guesswork. PhD-level human reviewers spot-checked the pipeline and confirmed 90–94% label accuracy, validating that this approach produces trustworthy data at scale.

What they found

Even the best models struggle. Top step-localization accuracy on text traces topped out around 74% (Qwen3.5-122B); diagnosing why a step failed (the failure-mode category) topped out around 22–28% macro-F1. No single model won across the board — this looks like a distinct skill, not something that falls out of general capability gains.
Text and video have opposite strengths. Text traces are easier to localize (which step failed) but harder to diagnose (why); video traces are the reverse — visual and behavioral cues help models spot what kind of error occurred, even when pinpointing the exact moment is harder.
Models take the easy answer, not the right one. LLMs frequently mislabel planning, verification, and coordination failures as generic "reasoning errors" — grabbing the most visible symptom instead of tracing back to the actual root cause.
Length is the real enemy. Attribution accuracy fell from 94% on short traces (under 3K tokens) to just 50% on long traces (12K+ tokens) — consistently, across all ten models tested. Longer, messier agent runs are dramatically harder to audit.
"Read it all, then decide" wins. Giving a model the full trajectory at once outperformed step-by-step or divide-and-conquer review strategies — and was also the cheapest option. That's a clean, actionable protocol for anyone building an attribution or monitoring tool today.
Open-weight models are genuinely competitive. Qwen3.5-122B led on text accuracy at roughly 7x lower cost than GPT-5.4; Gemma 4 led on video. Reliable failure auditing doesn't require the most expensive frontier API.
Giving models the "right answer" as a hint can backfire. It helps catch perception errors (like misread charts) but hurts diagnosis of reasoning errors — models shortcut to "did the final answer match" instead of actually tracing the logical chain that led there.

Why this matters for teams running agents

This isn't a paper about agents being unreliable — it's a paper about how hard diagnosing unreliability still is, and where the near-term wins are. A few takeaways translate directly into practice:

Feed the whole trace, not fragments. If you're building or buying agent-monitoring tooling, "all-at-once" review beats sequential inspection on both accuracy and cost — a genuinely simple design choice with an outsized payoff.
Shorter, cleaner execution logs pay off twice. Trace length isn't just a token-cost problem — it's an auditability problem. Trimming unnecessary steps, verbose intermediate output, and redundant tool calls makes failures easier to catch, for both humans and models.
Don't assume the biggest model is the best auditor. Cost-effective open-weight models held their own here, which matters for teams weighing the ROI of building internal QA tooling around agent behavior.
Full automation of root-cause analysis isn't here yet — and that's fine. Human oversight on high-stakes agent failures remains the right call for now. The realistic near-term move is LLM-assisted triage that narrows down where to look, not a fully autonomous verdict.

That's consistent with how we think about reliability at Odella: the goal isn't to pretend agents don't fail, it's to build the visibility and tooling that make failures fast to catch, explain, and fix — which is exactly the kind of infrastructure work benchmarks like Who&When Pro are helping the whole industry get more rigorous about.

Read the full paper: Who&When Pro: Can LLMs Really Attribute Failures in AI Agents? (arXiv:2607.09996).