Everyone is racing to deploy AI agents. Far fewer teams have a real answer to a deceptively simple question: how do you evaluate an AI agent before you trust it with real work? An agent benchmark says "passed." That's the entire report. It doesn't tell you whether the agent found the right answer through sound reasoning or a lucky shortcut, whether it would survive a slightly different version of the same task tomorrow, or whether it took a risky action on the way to a technically correct result. New research argues that gap isn't a rounding error — it's the difference between a benchmark score and a trustworthy answer to "can I deploy this?"
The problem: benchmarks report what, not how or why
Researchers from Princeton, the UK AI Safety Institute, and several independent AI evaluation labs — Peter Kirgis, Sayash Kapoor, Stephan Rabanser, and colleagues — lay out three specific ways pass/fail scoring misleads teams:
- Scores get inflated or deflated by shortcuts and benchmark artifacts. An agent can pass a task by exploiting a quirk in how the benchmark is built, not by actually solving the underlying problem — which means the score overstates real capability.
- Benchmark performance doesn't predict real-world reliability. Scaffold limitations and recurring failure modes that never show up in a pass/fail count still show up the moment the agent runs in production.
- Capability scores can hide dangerous or catastrophic actions. An agent that reaches the correct final answer can still take a risky, destructive, or out-of-scope action along the way — and a binary outcome metric has no way to flag it.
Their proposed fix: log analysis — systematically tracking and reviewing the full record of what an agent actually did (its inputs, tool calls, intermediate reasoning, retries, and outputs), not just whether the last step matched an expected answer.
What log analysis actually caught
The authors didn't just argue this in theory — they applied their own guiding principles to tau-Bench Airline, a widely used agent benchmark, and read the logs behind the scores. What they found:
- Pass^5 performance was under-elicited by nearly 50%. In other words, when they looked past the pass/fail number and examined how the agent behaved across five repeated attempts, the benchmark had been significantly understating what the agent could actually do — the scoring method itself was hiding real capability.
- Deployment failure modes were invisible to the outcome metric. Patterns that would matter enormously in a live business setting — the specific ways the agent got confused, retried, or handled edge cases — never appeared in the score at all. You only see them by reading the logs.
That's the core finding worth sitting with: the same benchmark, read two different ways, told two different stories. One number said "roughly this capable." The logs said something meaningfully different, in both directions — hidden capability in some cases, hidden risk in others.
Why this matters if you're actually deploying agents
This isn't an academic nitpick about how researchers should score leaderboards. It's a direct warning about how most businesses currently evaluate agents before turning them loose on real work: run a demo, check a few outputs, ship it. That process is a pass/fail benchmark with extra steps — and this research shows exactly how much it can miss.
Practical takeaways if you're putting an agent into production:
- Don't just check the final output — check the trace. Which tools did it call, in what order, and did it recover sensibly when something failed? A right answer reached through a broken path will fail differently next time.
- Test repeatedly, not once. The tau-Bench Airline finding — a ~50% gap between single-run scores and repeated-attempt performance — is a direct argument for running the same task multiple times before trusting a result, in either direction.
- Review logs for actions, not just outcomes. A summary metric can't tell you an agent quietly did something it shouldn't have on the way to a correct-looking answer. Reading the actual execution trace can.
The honest limitation
The authors are clear that log analysis doesn't replace benchmarks — it's a complement, and it's genuinely more effort than reading a scoreboard. Their recommendations are aimed at benchmark creators, model developers, independent evaluators, and deployers precisely because uptake is currently low: it's easier to publish a leaderboard number than to systematically review execution logs at scale. The paper is as much a case for building better tooling and norms around log analysis as it is a critique of pass/fail scoring itself.
Where this lands for us
This is exactly the gap we design around at Odella. An AI employee isn't judged once on a demo task — it's accountable for how it does the work, run after run, with a trace you can actually review. That's the same standard we apply in our own framework for evaluating AI agents: behavior and reliability matter as much as raw capability, because a benchmark score can't tell you what actually happened inside a task — only the logs can.
Source: Peter Kirgis, Sayash Kapoor, Stephan Rabanser, Nitya Nadgir, Cozmin Ududec, Magda Dubois, JJ Allaire, Conrad Stosz, Marius Hobbhahn, Jacob Steinhardt, Arvind Narayanan. Log analysis is necessary for credible evaluation of AI agents. arXiv:2605.08545 (2026). Read it at arxiv.org/abs/2605.08545.
Want AI that's built for reliability, not just demos? Explore Odella's AI employees or get started free.