A multi-agent system is one in which several AI agents, each with its own scope and tools, work on parts of a larger problem and coordinate the result. The architecture is fashionable, occasionally correct, and responsible for a substantial share of the enterprise AI projects that are quietly cancelled.
The honest starting position is that most problems described as needing multiple agents need one agent with more tools, or a workflow with one model call. Multiplying agents multiplies coordination, and coordination is where reliability goes to die. Gartner expects over 40% of agentic AI projects to be cancelled by the end of 2027 on escalating cost, unclear value and inadequate risk controls, and a follow-on forecast expects 40% of enterprises to demote or decommission autonomous agents by 2027. Multi-agent designs concentrate all three of those risks.
That said, there are problems where multiple agents are the right answer. This article is about telling the difference, and about building the ones that are worth building.
When multiple agents are actually justified
Three conditions. If fewer than two hold, build a single agent.
Genuinely different tool domains with different permissions. An agent that queries the core banking system and an agent that drafts customer correspondence need different data access, different approval rules and different audit treatment. Splitting them is a security boundary, not an architectural preference — and security boundaries are the strongest reason to have more than one agent.
Different reliability requirements. A research step that can be wrong and retried has no business sharing a process with a step that writes to a ledger. Separating them lets you apply strict controls where they are needed and loose ones where they are not.
Independent work that can proceed in parallel. Ten documents to analyse independently, with results merged at the end. This is the cleanest multi-agent case and the one that most reliably pays for itself, because the coordination is trivial — fan out, fan in, no negotiation between agents.
What does not justify multiple agents: role-play decomposition. Giving one model the persona of a researcher, another the persona of a critic and a third the persona of an editor feels like a team and behaves like one model called three times with more latency and more places to go wrong. If the agents do not have different tools or different permissions, they are not different agents.
The four architectures that work
Supervisor and workers. One coordinating agent decomposes the task, dispatches to specialist agents, and assembles the result. The workers do not talk to each other. This is the default and it should be the default: control flow is centralised, so there is exactly one place where the plan lives, one place where budgets are enforced and one place to look when something goes wrong.
Pipeline. A fixed sequence of specialists, each taking the previous one's output. Extraction, then validation, then enrichment, then write. Deterministic order, easy to test stage by stage, easy to audit. Where the sequence genuinely is fixed, this is not really a multi-agent system at all — it is a workflow with model calls in it, which is a compliment.
Fan-out and merge. Identical agents working on independent items in parallel, with results combined by a deterministic merge step. The merge step should be code, not a model, wherever the combination rule can be expressed.
Blackboard. Agents read and write to shared state and act when they see something relevant. Powerful and genuinely hard to reason about, because behaviour is emergent and reproduction of a bug requires reproducing the whole state history. Reach for it rarely, and never for a first deployment.
The architecture that does not work in an enterprise setting is free-form peer negotiation — agents deciding among themselves who does what. It demos impressively, produces non-reproducible behaviour, and cannot be explained to an auditor.
The failure modes
Six, all observed rather than theorised, and each one has a specific mitigation.
Compounding error. Each agent is 95% reliable. Five in sequence gives roughly 77%. Ten gives 60%. The system does not fail loudly at step one; it drifts, and the output looks plausible. Mitigation: shorter chains, deterministic verification between stages, and a refusal to design tasks that require a long unbroken sequence to succeed.
Cost explosion. Agents calling agents multiply token consumption in ways that are hard to predict from a design document, because the number of calls depends on task difficulty. A run that costs cents in testing costs dollars on a hard case, and there is no ceiling unless you impose one. Mitigation: a hard token, step and wall-clock budget per run, enforced by the orchestration layer rather than by the model, plus routing simple sub-tasks to small models — self-hosted inference at batch runs around $0.18 per million output tokens on an H100 against dollars per million for frontier hosted models, and in a multi-agent system that ratio is applied many times over.
Deadlock and loops. Agent A waits for B, B waits for A. Or two agents hand a task back and forth, each deciding it belongs to the other. Mitigation: a supervisor topology with no peer-to-peer waiting, and a global step counter that terminates the run regardless of what the agents believe.
Permission escalation through composition. Agent A may read customer data. Agent B may send email. Neither is dangerous alone; the composition sends customer data by email. This is the most serious multi-agent-specific risk, and it is invisible in a per-agent security review. Mitigation: assess permissions at the level of the whole system, and enforce entitlements at the point of the tool call under the requesting user's identity — never under a service account.
Untraceable behaviour. Something went wrong and nobody can say which agent decided what. Mitigation: a single correlation identifier across the entire run, with every agent's inputs, tool calls, arguments and outputs recorded against it. Retrofitting this is not possible; the record either exists or the incident is unexplainable.
Prompt injection propagating between agents. A document read by the research agent contains instructions; the research agent's output is trusted input to the agent with write access. Mitigation: treat every inter-agent message as untrusted data, validate structure at each boundary, and never let retrieved content reach a writing agent without passing a non-model check.
What has to exist underneath
A multi-agent system is not a prompt pattern. It requires an orchestration layer with five specific capabilities.
Identity propagation through every hop, so that agent number four still acts under the identity of the person who made the request. Anything less flattens entitlements across the organisation.
A tool registry with per-agent scoping. Which tools this agent may call, in this context, with which argument constraints. Structural, not a sentence in a prompt.
Durable state. A failure at step seven that restarts at step one is the most common cause of a multi-agent system burning budget without producing anything.
Run-level budgets for tokens, steps, cost and wall-clock, enforced outside the models.
Correlated audit across the whole run, at the granularity an auditor will ask for. In a regulated institution this is what an ISO/IEC 42001 audit expects to see.
Without these five, what you have is several scripts calling models, and it will behave like it.
Building the first one
Start with fan-out and merge, on independent items, with a deterministic merge. It is the architecture with the least coordination and the clearest value, and it teaches the team the operational realities — cost variance, retry behaviour, partial failure — without the risk profile of a supervisor topology making decisions.
Give every agent read-only tools except one, and make that one write reversible: create a draft, open a ticket, propose a change. Irreversible actions belong behind a human until the system has a track record.
Build the evaluation harness before the first production run — fifty to a hundred real cases from your own domain, scored end to end rather than per agent. Per-agent accuracy is a misleading metric in a system where errors compound; the only number that matters is whether the whole run produced a correct result.
Then measure the thing that decides whether the project survives: how often a human quietly redoes the work afterwards. That number, not task completion rate, is what separates the multi-agent systems that stay in production from the 40% that do not.
Key points
- Most problems described as multi-agent need one agent with more tools, or a workflow with one model call. Role-play decomposition is not an architecture.
- Justified by different tool domains with different permissions, different reliability requirements, or genuinely parallel independent work.
- Use supervisor-and-workers, pipeline, or fan-out-and-merge. Free-form peer negotiation demos well and cannot be audited.
- Errors compound: five 95% agents in sequence give 77%. Design for short chains with deterministic verification between them.
- Permission escalation through composition is the multi-agent-specific risk and is invisible in per-agent review. Assess the whole system.
- Correlated audit across the run must exist from the start. It cannot be reconstructed after an incident.
HAVAA runs multi-agent systems inside the customer's perimeter with identity propagation, per-agent tool scoping and correlated audit built in. Related reading: what is an AI agent and what is MCP.