A demo agent and a production agent are not the same animal in different clothes. The demo works because you were there — patient, forgiving, feeding it clean inputs one at a time. Production is none of those things. It shows up as thousands or millions of requests, with malformed inputs, flaky tools, and the occasional model response that confidently invents an API that never existed.
Once you cross that line, the hard problems stop being about prompting and start being about control. Here are ten failure modes I keep seeing in large-scale agentic systems, and the patterns teams reach for to contain each one. Almost none of them are exotic — most are old distributed-systems lessons, rediscovered with a language model sitting in the middle.
1. The agent that never stops
It plans, searches, evaluates — then decides it isn’t quite sure, and plans again. Each lap looks reasonable on its own; together they burn tokens and never converge on an answer.
The fix. Bound every dimension of a run up front — iterations, tool calls, wall-clock time, and spend — and treat the first threshold it crosses as a hard stop that returns the best answer so far, escalating to a human when the stakes warrant it.
max_iterations: 10
max_tool_calls: 20
max_cost: $0.10
max_time: 30s
Seen in OpenAI’s agent tooling, Anthropic’s Claude workflows, and orchestration frameworks like LangGraph.
2. Tools that don’t exist
The model announces “Calling the CRM API…” and then reaches for a tool that isn’t there, or passes arguments that don’t typecheck. Confident, fluent, wrong.
The fix. Put a contract between the model and every tool. The model proposes a structured call; a schema validates it before anything executes, so a malformed call fails at the boundary instead of halfway through a side effect.
{ "tool": "customer_lookup", "customerId": "123" }
In practice that means JSON Schema, Pydantic, or OpenAPI guarding every call.
3. Context that balloons
Every step bolts more onto the prompt — the question, each search result, every tool output, every reflection. A few laps later the context is enormous, and cost and latency climb right along with it.
The fix. Add a compression layer between steps. Keep the goal, the facts that matter, and the decisions already made; discard raw tool logs and intermediate reasoning. A summariser turns a sprawling history into a compact working state — a pattern that shows up everywhere in multi-agent systems.
4. The twenty-five-second answer
The workflow touches search, then the database, then the CRM, then inventory, then pricing — each a few seconds, all in series. The user is long gone before the agent finishes.
The fix. Anything without a data dependency runs at the same time: fan the independent calls out, then gather them back in. Orchestrators like Temporal, LangGraph, and Microsoft Semantic Kernel make fan-out/fan-in a first-class pattern.
5. The agent that forgets why it’s here
Asked for the cheapest Dell server for AI workloads, it drifts — into GPU architectures, then CUDA history, then NVIDIA’s roadmap. Each step is a plausible neighbour of the last, and none of them is the task.
The fix. Make the goal persistent state, not a fading memory. The orchestrator re-injects it into every step, so each decision is judged against the original intent rather than the previous tangent.
{ "goal": "Find the cheapest Dell server for AI workloads" }
6. A thousand identical searches
A thousand users ask “what’s today’s NVIDIA news?” and the agent runs a thousand near-identical web searches. The same work, a thousand times, on a thousand separate bills.
The fix. Cache the expensive, repeatable parts — search results, embeddings, even whole agent outputs — behind something like Redis. The first request pays; the rest are nearly free.
7. Agents with their hands on the controls
An agent that can delete records, issue refunds, or reshape infrastructure is one bad decision away from an incident.
The fix. Split actions by reversibility. Reads execute on their own; writes pass through a human approval gate, with thresholds for the costly ones — a refund over $1,000 waits for a person. This is standard in banking, healthcare, and enterprise SaaS.
8. Too many cooks
A swarm of agents — planner, researcher, reviewer, coder, tester — with no clear ownership turns into an endless group chat where nobody actually finishes anything.
The fix. Give each agent a single responsibility, a defined input, and a defined output, and arrange them as a pipeline rather than a free-for-all. Less conversation, more conveyor belt.
9. The bill that 1000×'d
A hundred requests a day looks cheap. A million a day, all routed to your largest model, does not.
The fix. Route by difficulty. Intent detection goes to a small model, planning to a mid-tier one, the final synthesis to the large one. Matching model size to task is one of the biggest levers you have on cost.
10. Flying blind
A user says “the agent gave a wrong answer,” and nobody can say which prompt ran, which tool failed, or which model replied. Without a trail, debugging is archaeology.
The fix. Trace everything behind a single request id — agent state, tool calls, model calls, latency, cost, and errors. LangSmith, Arize Phoenix, and OpenTelemetry are common homes for that data.
What the whole thing looks like
Add these up and the “agent” turns out to be a small part of a much larger system. A production stack usually looks something like this — a spine of orchestration with reliability machinery around the edges.
In production, the hardest part was never building the agent — it’s governing it. The failures that take systems down are rarely the model being dim; they’re unbounded loops, runaway cost, missing guardrails, and the inability to see what happened. A production-grade agent loop is, in the end, a distributed system with a language model in the middle — and it rewards exactly the discipline you’d bring to any other distributed system.