Nine of Ten Steps: Why Agent Reliability Is an Evals Problem

It's 2 AM. Production is down. Error rates are spiking across regions, transactions are failing, and the incident has auto-escalated. A SWAT bridge spins up — fifty people on the call: SREs, application teams, database on-calls, a duty manager briefing leadership every fifteen minutes. Everyone is asking the same question: what changed?

While the bridge is still doing roll call, an AI agent has already picked up the incident. It knows what to do — there's a runbook the SRE team has polished for two years. Ten steps, no shortcuts. The agent has run it two hundred times. It knows the shape of a memory leak, the fingerprint of a failing DB failover, the smell of an expiring certificate. Most nights it hands the on-call engineer a triaged ticket with the story already told and half the fix drafted.

Tonight, it does nine of the ten steps. It skips step 5 — checking recent deployments — and writes back a confident, well-formatted root cause anyway: "Elevated latency on the primary database is causing upstream 500s. Recommend failing over to the replica."

The agent didn't lie. It skipped a step and didn't tell anyone.

Here's what it missed:

Had it checked the deploy history, it would have found that twenty minutes before the alert, someone shipped a config change that pointed the service at a stale environment variable. The database was fine. The failover would fix nothing — and bury the real cause deeper. The write-up lands in the bridge chat, sounds authoritative, and the bridge runs with it. Fifty people spend twenty minutes chasing a database problem that doesn't exist. When someone finally scrolls through the deploy history, the fix is a one-line rollback.

What the agent actually did

The raw trace tells the story plainly:

get_incident → check_health → get_pods → get_pod_logs → (nothing) →
check_dependencies → get_metrics → check_ingress → check_platform_incidents →
update_incident

Between step 4 and step 6 there's a hole. get_recent_deployments was never called. That's the gap that turned a one-line rollback into a twenty-minute chase.

The failure mode nobody greps for

Run that same agent on the next ten incidents and it nails all ten — every step, in order, grounded, correct. Then on the eleventh, for no reason you can point to, it drops step 5 again. Or step 7. Or cites a log line it never read.

This is not how traditional software fails.

When people say "AI is non-deterministic," this is what they mean. Not that the agent is random — that it's variable in ways you can't predict from the outside. The old QA reflex — did the output match what we expected? — falls apart, because the output almost always looks fine. The real question is different:

How did the agent get here, and did it actually do the work?

The answer is one equation:

Evals: turning visibility into a verdict

Observability for agents means you can replay any run — every tool call, every parameter, every response, in order. Drop tonight's incident into a trace and the gap is obvious. But observability has a ceiling: it tells you what happened, not whether it was good. A trace shows the agent called get_recent_deployments; it doesn't tell you whether the root cause is actually grounded in what that call returned.

Evals are QA for agents — scored the same way on every run, not just the ones someone happens to review. Three matter most:

That third one catches tonight's miss. Step 5 was in the runbook; step 5 was not in the trace. The eval flags it before the engineer touches the ticket. Instead of "here's a confident root cause, trust me," the ticket reads:

9/10 steps completed — deployment history was not checked. Human verification recommended before acting.

No team can afford a senior engineer re-reading every ticket an agent closes, so a model does the reading. LLM-as-a-judge puts the agent's write-up side-by-side with the evidence it actually pulled and scores it: grounded or not, steps followed or not, answer on-topic or not. It's the same skim a senior engineer gives a ticket at handoff — except it runs on every ticket, not the two or three that get sampled.

Notice what the pipeline does not do: it doesn't fix the agent. It doesn't retry the run or rewrite the root cause. It turns a silent skip into a loud, scored signal — and hands that signal to a human before the bridge acts on a bad recommendation. The fix, when it comes, is still a human fix: a glance at the deploy history, a one-line rollback, a closed ticket.

The point is boring

Evals don't make an agent smarter. They don't stop it from having off nights. They make it impossible for a skipped step to pass as a complete one. The nine boring, correct runs stay boring. The one where the agent quietly dropped step 5 gets caught the night it happens — not next week, when someone reopens the ticket and realizes the failover fixed nothing.

Thanks for reading — ping me at dengalebr@gmail.com to discuss.