AI Agents in Production: What the Demos Don't Show You

Agent demos are genuinely impressive and almost entirely uninformative. Someone types a request, a sidebar fills with tool calls, and ninety seconds later there's a pull request or a booked flight or a filled spreadsheet. It works. It always works, because you're watching the run that worked.

I've shipped agentic features and reviewed a number of other teams' attempts, and the gap between that demo and a system you'd put in front of customers is wider than the gap between a prototype API and a production one. Not because agents are fake — the capability is real — but because the demo optimizes away every property that matters in production: reliability across thousands of runs, bounded cost, recoverable failure, and some way to know whether a given run was any good.

Here's what's actually on the other side of that gap.

Problem 1: Step Reliability Compounds Against You

This is the one nobody internalizes until they've measured it. An agent run is a sequence of steps, each of which can fail. Reliability multiplies:

Per-step success	5 steps	10 steps	20 steps
95%	77%	60%	36%
98%	90%	82%	67%
99%	95%	90%	82%

A 95% per-step success rate sounds excellent. Over a twenty-step task it means roughly two runs in three fail somewhere. And "fail" rarely means a clean exception — it means step 7 picked a slightly wrong parameter and steps 8 through 20 built confidently on top of it.

The engineering consequences follow directly:

Shorter horizons beat longer ones. A five-step agent invoked four times, with a checkpoint between, is dramatically more reliable than one twenty-step agent. Decompose aggressively.
Determinism wherever you can get it. If a step can be a plain function call instead of a model decision, make it one. The most reliable agentic systems I've seen are mostly ordinary workflow code with model calls at the genuinely ambiguous junctions. This is the boring answer and it's correct.
Checkpointing matters more than retrying. Rerunning a failed twenty-step task from step 1 burns the full cost again and may fail somewhere different. Persist state between steps so you can resume.

✅

Before building an agent, ask whether the task is actually a workflow. If the sequence of steps is known in advance, write the workflow and use the model for the individual judgment calls inside it. "Agent" should be reserved for tasks where the plan itself must be discovered at runtime.

Problem 2: Tools Fail in Ways the Model Can't See

In a demo, tools are stubs or happy-path API calls. In production, tools time out, return partial data, hit rate limits, return 200 with an error body, or succeed while doing nothing. And the model only knows what the tool tells it.

The failure I keep seeing: a tool returns {"results": []} because of an auth problem, and the agent reports "I searched and found no matching records" — which is fluent, plausible, and wrong. The agent didn't lie; the tool did, by omission.

What fixes it:

Tool responses must distinguish "no results" from "couldn't look." Encode the difference explicitly in the payload the model sees:

json

{ "status": "ok",    "results": [] }
{ "status": "error", "reason": "auth_failed", "retryable": true, "results": null }

The model handles this well when it's told; it cannot infer it.

Tool descriptions are prompts, and they're the highest-leverage text in the system. Half the "the model picked the wrong tool" bugs I've debugged were really "two tool descriptions were ambiguous about which handles which case." Spend real time on descriptions, parameter names, and worked examples — more time than you spend on the system prompt.

Idempotency isn't optional. Agents retry. Agents loop. If send_email or create_ticket can be invoked twice with the same arguments, it will happen, and the user will see it. Every mutating tool needs an idempotency key or a dedupe window.

Narrow beats general. A search_orders(customer_id, date_range, status) tool is used correctly far more often than a run_sql(query) tool. The general tool is more powerful in the demo and vastly more dangerous in production — it moves correctness from your code into the model's SQL.

Problem 3: Cost Is Unbounded by Default

This is the one that produces the alarming invoice. In a chat completion, cost is roughly linear in input and output. In an agent loop, the conversation grows with every step — each tool call and result is appended to the context the next step sees.

The shape is quadratic. Twenty steps with growing context can cost more than an order of magnitude what a naive estimate suggests, and a looping agent — one that gets stuck retrying a failing tool — can burn an unbounded amount before anyone notices.

Non-negotiable controls, all of which should exist before the first production run:

Hard step cap per run. Not a suggestion in the prompt — a counter in the loop that terminates.
Token budget per run, checked before each model call, with the run failing loudly on breach.
Loop detection. Track a hash of (tool, arguments). Three identical calls means the agent is stuck; break out and escalate rather than letting it grind.
Context compaction. Summarize or prune older tool results instead of carrying every raw payload forever. Most tool outputs stop being relevant after a few steps.
Per-tenant spend caps if the feature is customer-facing, so one pathological user can't consume the budget for everyone.

The relevant discipline is covered properly in cost optimization and model routing — routing the cheap steps to small models matters even more in agent loops than in single-shot calls, because you're paying for many calls per user request.

Problem 4: You Can't Tell a Good Run from a Bad One

Traditional observability tells you an agent run completed in 47 seconds, made 12 tool calls, and returned 200. It tells you nothing about whether the answer was right or the actions were appropriate.

What production agent observability needs:

Full trace per run — every prompt, tool call, argument, result, and decision, stored and queryable. When a user reports a bad outcome, replaying the exact trace is the only way to diagnose it. This is not optional and it's expensive in storage; budget for it.
Step-level outcome tagging, not just run-level. "Which step went wrong" is the question you'll ask every time.
Trajectory evaluation, not just final-answer evaluation. Two runs can produce the same output while one took a sensible path and the other stumbled into it. Score the path.
Sampled human review, ongoing. A weekly hour spent reading twenty random traces surfaces failure patterns that no automated metric catches. Every team I know that runs agents well does this.

⚠️

The metric that matters least is the one teams report most: "the agent completed the task." Completion says nothing about correctness. Track task success as judged against a rubric, and track it separately from completion — the gap between the two numbers is your real quality problem.

Problem 5: Autonomy Without Boundaries Is a Security Model

An agent with tools is a program whose control flow is decided by text that may partly come from untrusted sources — a retrieved document, a web page, a support ticket. That's prompt injection with side effects, and it's the concern that should gate how much autonomy you grant.

The practical boundaries:

Separate read tools from write tools, and hold them to different standards. Reads can be broad. Writes should be narrow, idempotent, logged, and — for anything consequential — gated.
Human approval on irreversible actions. Sending customer emails, moving money, deleting data, merging code. The approval step is not a failure of the agent's design; it's the design.
The agent's permissions are the user's permissions, never the service's. An agent running with a superuser token is one injected instruction away from being a data exfiltration tool.
Treat all tool output as untrusted input. Content retrieved from a document or web page must not be able to issue instructions that the agent follows as if they came from the user.

The long-running workflows and human approvals guide goes deeper on the approval patterns, and security, reliability and guardrails covers the injection surface.

What Actually Works

Having listed everything that breaks, the constructive version. The agentic systems I've seen succeed in production share a profile:

Narrow domain. "Triage incoming support tickets" ships. "Be a general assistant for our company" does not.
Mostly deterministic scaffolding. A workflow with model calls at the ambiguous points, not an open-ended loop hoping to find a plan.
Small, well-described, idempotent tools — closer to ten focused tools than three powerful ones.
Aggressive bounds on steps, tokens, wall-clock time, and repeated calls.
Human-in-the-loop at the consequential moments, positioned so the human reviews a proposal rather than supervising every step.
An eval set built from real traffic, run in CI before prompt or model changes ship.
A graceful degraded mode — when the agent can't complete a task, it hands off to a human with the context it gathered, rather than guessing.

None of that is exotic. It's the same reliability engineering you'd apply to any distributed system with an unreliable dependency — which is precisely what an agent is. The demos skip it because a demo doesn't need it. Your users do.

Frequently asked questions

Why do AI agents that work in demos fail in production?

Demos show a single successful run of a short task with cooperative tools. Production runs the same agent thousands of times against tools that time out, return partial data, or fail silently — and per-step reliability compounds, so a 95% per-step success rate yields only about 36% success over twenty steps. The demo also hides cost growth, loop behavior, and the absence of any way to tell a good run from a bad one.

How do I control AI agent costs?

Agent context grows with each step, so cost scales roughly quadratically with run length. Enforce a hard step cap in the loop, a per-run token budget checked before each model call, loop detection on repeated (tool, arguments) pairs, context compaction that prunes or summarizes old tool results, and per-tenant spend caps for customer-facing features.

Should I build an agent or a workflow?

Build a workflow if the sequence of steps is known in advance, and use model calls for the individual judgment calls within it. Reserve agents for tasks where the plan itself must be discovered at runtime. Most production systems that are described as agents are mostly deterministic workflow code with model calls at the genuinely ambiguous junctions — and that is why they work.

What is the biggest security risk with tool-using agents?

Prompt injection with side effects. An agent's control flow is influenced by text that may come from untrusted sources like retrieved documents or support tickets, so injected instructions can trigger real tool calls. Mitigate by separating read and write tools, requiring human approval for irreversible actions, running the agent with the user's permissions rather than the service's, and treating all tool output as untrusted input.

How do you evaluate an AI agent?

Evaluate the trajectory, not just the final answer — two runs can produce the same output while one took a sensible path. Store a full trace of every prompt, tool call, and result per run; tag outcomes at the step level; score task success against a rubric separately from task completion; and sample traces for human review weekly, which surfaces failure patterns automated metrics miss.

Go deeper: the Gen-AI roadmap sequences this material, with agent systems, multi-agent architectures, tool calling and orchestration, and observability and tracing covering the pieces in depth.