Multi-Agent Architectures
Why cramming refunds, shipping, and account changes into one agent's toolset breaks down, and how orchestrator-worker patterns fix it.
One Agent, Too Many Jobs
The refund agent works well. The product team wants to extend the same pattern to a full customer-operations assistant: shipping issues, account changes, and refunds, all in one bot. The naive path — add shipping tools and account tools to the same agent's toolset, extend its system prompt to cover all three domains — starts breaking down almost immediately. The system prompt grows long and tries to cover three different sets of rules at once. The tool list grows to a dozen-plus definitions, most irrelevant to any given question, quietly eating into the context budget from Phase 3 on every single call. And the agent occasionally reaches for the wrong tool entirely — calling a shipping-tracking function on a refund question because both mention "order," a mistake that's much less likely when an agent's entire toolset is narrowly scoped to one domain.
Orchestrator-Worker: Specialize, Then Coordinate
The fix isn't a smarter single agent — it's several narrower ones. An orchestrator agent receives the customer's message, decides which domain it belongs to (the same classification pattern from Phase 2, now routing to an agent instead of a category label), and delegates to a worker agent built specifically for that domain — its own focused system prompt, its own narrow toolset, nothing about shipping cluttering the refund worker's context and vice versa.
Each worker is, structurally, everything covered in the previous two guides — a ReAct loop or a LangGraph node with its own state, scoped to one job. The orchestrator's only responsibilities are classifying the request and synthesizing the workers' results into one coherent reply; it never needs the refund agent's eligibility logic or the shipping agent's carrier-tracking details directly.
Information Isolation Is a Deliberate Choice, Not an Oversight
A natural instinct is to let every agent see everything, "just in case." Resist it: information isolation between workers is a direct application of two principles from earlier in this roadmap, not a limitation to work around. Phase 3's context budgeting argued that including irrelevant content degrades performance and wastes tokens — a refund worker doesn't need shipping-carrier API schemas in its context any more than the coding agent needed thirty turns of stale tool output. And Phase 2's least-privilege principle argued that a component should only have access to what it strictly needs — a shipping worker with no access to issue_refund at all is a shipping worker that structurally cannot be tricked into issuing a refund, regardless of what a cleverly worded message tries to get it to do, which shrinks the blast radius of a prompt injection dramatically compared to one agent holding every tool.
Anthropic's account of building their multi-agent research system is a useful real-world reference for the actual tradeoffs here: parallelizing work across specialized agents can meaningfully improve both quality and latency for tasks that decompose well, but it comes at a real token-cost multiplier — multiple agents each running their own reasoning loop costs more than one agent doing the same total work, and coordination overhead (the orchestrator's classification and synthesis steps) is a real cost, not a free abstraction. LangGraph's multi-agent concepts documentation covers the concrete orchestration patterns — supervisor/worker being the most common — for implementing this structure.
Multi-agent systems are a genuine cost-quality tradeoff, not a strict upgrade. A single well-scoped agent handling one domain is almost always cheaper and simpler than an orchestrator plus workers — reach for multi-agent specifically when a single agent's toolset or system prompt is genuinely straining across unrelated domains, not by default.
Check yourself
Why is giving the shipping worker agent no access to the issue_refund tool a deliberate security benefit, not just a minor convenience?
Closing This Phase
The refund agent that opened this phase as a single ReAct loop is now one specialized worker inside a broader, orchestrated customer-operations system — reasoning explicitly about each step (ReAct), running inside a framework that handles state and branching (LangGraph or PydanticAI), and isolated from unrelated domains by design (orchestrator-worker). None of that makes these agents reliable on its own, though — every guide in this phase quietly assumed tool calls succeed, APIs respond, and long workflows don't get interrupted halfway through. Phase 8 is the unglamorous engineering that makes those assumptions actually hold in production.
Frequently asked questions
How does the orchestrator decide which worker to route to?
Usually the same structured-output classification pattern from Phase 2 — the orchestrator makes one cheap, fast LLM call (often with a smaller, less expensive model than the workers use) that returns a category, which then determines which worker handles the request. For ambiguous messages, some orchestrators route to multiple workers and merge results, rather than forcing a single choice.
Is multi-agent always better than a single agent with a very good system prompt?
No — a single, well-scoped agent for a genuinely narrow domain often outperforms a multi-agent system on cost and simplicity, with no quality loss. Multi-agent architectures earn their overhead specifically when the problem space is broad enough that one system prompt and toolset can't stay focused, which is exactly the shipping-refund-account situation in this guide, not a universal default.
Do workers ever need to talk to each other directly, rather than only through the orchestrator?
It's possible, but most production systems deliberately route all cross-worker communication through the orchestrator rather than letting workers call each other directly — it keeps the information-isolation boundary from this guide intact and gives you one place (the orchestrator) to reason about the whole system's behavior instead of an unpredictable mesh of agent-to-agent calls.