Curated Skills Beat Big Doc Dumps: What SkillsBench Found

If you've ever handed an AI agent a giant reference manual and hoped it would figure out the right approach on its own, a new benchmark has some hard data on why that usually underperforms — and what actually works instead.

The paper

SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks (Xiangyi Li and 76 co-authors, a large multi-institution research group) tackles a question that's easy to assume the answer to but hard to actually measure: do "Agent Skills" — structured packages of procedural knowledge fed to an LLM agent at inference time — meaningfully improve performance, or is it mostly vibes?

The team built SkillsBench specifically to answer that with rigor: 87 tasks across 8 domains, each paired with a curated Skill and a deterministic verifier, so pass/fail isn't a judgment call. They ran the full 87-task benchmark twice for each of 18 model-harness configurations — once with no Skills, once with curated Skills — and compared the paired results directly.

What they found

Three results matter for anyone building or buying agent systems right now.

Skills work, and the effect is large. Across the 18 configurations, curated Skills raised the average pass rate from 33.9% to 50.5% — a 16.6 percentage-point lift, or a 25.5% normalized gain. Individual configuration-level gains ranged from +4.1 to +25.7 percentage points. This isn't a marginal nudge; for some setups, the right Skill package roughly doubled the effective capability of the same underlying model.

Focused beats exhaustive. Skill bundles with three modules or fewer outperformed larger or "throw everything in" bundles. More documentation isn't better documentation — a tightly scoped package of the exact procedural knowledge needed for a task beat comprehensive reference dumps every time. That's a direct answer to the instinct to just hand an agent everything it might conceivably need.

Smaller models with good Skills can match bigger models without them. This is the finding with the most practical weight: model scale isn't the only lever for capability. A smaller, cheaper model equipped with a well-curated Skill can perform in the same range as a larger model working from scratch. Skill quality is a substitute for raw parameter count, at least on the kinds of expertise-heavy, procedural tasks SkillsBench measures.

Why this matters if you're deploying agents at work

Most teams evaluating AI agents focus almost entirely on which model to use. SkillsBench is evidence that this is only half the picture — and possibly not the half with the most leverage.

A few practical takeaways:

Curation is a capability lever, not busywork. Building a tight, well-scoped Skill for a real workflow can move an agent's reliability more than swapping to a bigger model. If you're only tuning the model, you're leaving performance on the table.
Resist the urge to dump everything into context. The "focused beats exhaustive" finding is counterintuitive if you assume more context is always safer. It isn't. A curated three-module Skill outperformed larger bundles — precision matters more than volume.
Model choice and Skill quality are separate decisions. Don't assume you need the frontier model to get frontier-quality results on a specific task. A smaller model with the right Skill can close much of that gap, which has real cost implications at scale.
This is measurable, not vibes. SkillsBench's deterministic verifiers matter — it means Skill effectiveness can actually be tested and iterated on, the same way you'd test any other part of a production system.

The bigger picture

This is exactly the thesis behind how we think about AI employees for business at Odella: the model is necessary but not sufficient. What actually determines whether an agent is reliable in production is the scaffolding around it — the curated procedural knowledge, the memory of what's worked before, the discipline to package expertise tightly rather than hoping a bigger context window compensates for a fuzzier setup.

SkillsBench puts numbers behind something we've believed operationally: curation and reliability matter as much as the underlying model. An agent that knows how to do a job well — with focused, tested know-how, not a firehose of documentation — outperforms one that's just bigger.

Curious how curated skills and memory show up in a real AI employee? See how Odella's AI employees work.