Compound Reliability in Agent Workflows
The Core Insight
A single-step benchmark gap of 4% becomes a 26-point gap in a 10-step workflow. Per-step reliability compounds exponentially across sequential steps.
The math:
- Task success rate = (per-step success rate) ^ (number of steps)
- 93% per step × 10 steps = 48% task success
- 97% per step × 10 steps = 74% task success
- 99% per step × 10 steps = 90% task success
| Per-Step | 5 Steps | 10 Steps | 15 Steps | 20 Steps |
|---|---|---|---|---|
| 99% | 95% | 90% | 86% | 82% |
| 97% | 86% | 74% | 63% | 54% |
| 95% | 77% | 60% | 46% | 36% |
| 93% | 70% | 48% | 34% | 24% |
| 90% | 59% | 35% | 21% | 12% |
Why This Matters for Pricing
This is why a model at $22.50/M tokens can coexist with a model at $0.90/M tokens serving the "same" capability. They serve fundamentally different use cases:
- Single-step tasks (chatbots, Q&A, one-shot generation): the 4% gap is invisible. Use the cheap model.
- Multi-step agent tasks (enterprise workflows, coding agents, autonomous research): the 4% gap compounds into a chasm. The expensive model's 25x price premium is justified by 1.5-3x higher task completion rates.
Where Failures Cluster
The extra failures in cheaper/smaller models are NOT randomly distributed. They concentrate on:
- Ambiguous instructions — where the model must resolve uncertainty without asking for clarification
- Long-context reasoning — where 15+ prior steps must be held in working memory simultaneously
- Novel edge cases — inputs slightly outside the training distribution
- Complex tool interactions — multi-tool compositions where protocol limitations (e.g., MCP expressivity gaps) create silent failures
More active parameters = more computational paths to find the right answer on hard cases. This is why retrying the same step on the same model often fails the same way.
Three Approaches to Agent Reliability Engineering
1. Smart Model Routing
Route easy steps to cheap models, hard steps to expensive models. Pay premium prices only for the 20-30% of steps that need it.
How it works:
- A classifier (can be rule-based or a small LLM) evaluates each step's complexity
- Simple steps → cheap model (e.g., Haiku, Gemma 4)
- Complex reasoning steps → expensive model (e.g., Opus)
- Total cost drops 70-80% while keeping compound reliability close to expensive-everywhere
Real-world example: oh-my-claudecode (21.9K GitHub stars, April 2026) implements this with 32 specialized agent roles and smart routing between Haiku and Opus.
Tradeoff: Requires building and maintaining the routing classifier. Misrouting (sending a hard step to a cheap model) causes the same compound failure.
2. Verification Layers
After each step, run a cheap verification check: "Does this output look right?" If verification fails, escalate that specific step to a stronger model or retry.
How it works:
- Step executes on cheap model → output goes to verifier
- Verifier passes → proceed to next step
- Verifier fails → retry on more capable model, or flag for human review
Tradeoff: Each verification step is an additional inference call — a mini N-watcher (see agent-architecture-skeleton). This converts a reliability problem into a latency + cost problem. Verification itself can have false positives/negatives.
3. Task Decomposition
Rewrite hard N-step workflows as easier 3N-step workflows where each sub-step is simple enough that even cheap models handle it at 99%.
How it works:
- Analyze which steps have the highest failure rates
- Break those steps into smaller, more explicit sub-steps
- Each sub-step has less ambiguity and requires less reasoning
Tradeoff: More steps = more total inference calls = higher latency. Works well for structured workflows but poorly for tasks that inherently require holistic reasoning across many inputs.
The Fundamental Tradeoff
Every agent deployment chooses between three costs:
| Approach | Cost Type | When to Use |
|---|---|---|
| Expensive model everywhere | Model cost (highest per-query) | Mission-critical, can't afford failures |
| Routing + verification | Engineering cost (upfront build) | High volume, cost-sensitive, have engineering capacity |
| Let tasks fail | Failure cost (downstream consequences) | Low-stakes, human fallback available |
Evaluation Framework
When evaluating any agent tool, model, or startup announcement:
- Is the benchmark single-step or multi-step? If single-step, the compound gap could be enormous.
- What's the per-step reliability? Even 2-3% difference compounds into a chasm at 10+ steps.
- Who is the customer? Chatbot user (single-step) vs enterprise deploying 15-step autonomous agents.
- Which reliability approach are they using? Model routing, verification, decomposition — or just hoping?
- What's their cost-reliability curve? At what reliability level does their approach break even vs. just using the more expensive model?
Connection to Anthropic's Moat
Anthropic's real moat is not lock-in (MCP + API compatibility make switching easy) or brand. It's compound reliability in multi-step workflows. The last few percentage points of per-step accuracy are worth exponentially more than the first 90%.
This is why Opus at $22.50/M and Trinity at $0.90/M serve different markets and can coexist indefinitely.
Source: frameworks/compound-reliability-agent-workflows.md