Multi-Agent Systems

A single agent with many tools hits a ceiling. Prompt complexity grows, tool selection degrades, and security boundaries blur. Multi-agent systems solve this by splitting work across specialised agents that coordinate to complete a task.

The coordination overhead is real. Every pattern below adds latency, failure modes, and debugging complexity. Use the lowest level of complexity that reliably meets your requirements.

When Single Agent is Enough

Before adopting multi-agent, evaluate whether you actually need it.

LevelDescriptionWhen to use
Direct model callSingle LLM call, no toolsClassification, summarisation, translation
Single agent + toolsOne agent with 5-8 tools, loops until doneMost enterprise use cases (80% of scenarios)
Multi-agentMultiple specialised agents coordinatingCross-domain problems, security boundaries, tool overload

Most teams jump to multi-agent too early. A single well-designed agent with good tools handles the majority of production scenarios. Multi-agent adds coordination overhead, latency, and failure modes that you must justify.

Orchestration Patterns

Five Patterns
Covered in turn below
Sequential
Pipeline
Concurrent
Fan-out, fan-in
Handoff
Delegation
Group Chat
Discussion
Magentic
Dynamic planning

Sequential (Pipeline)

Agents chained in a fixed order. Each processes the output of the previous one.

Input
Agent 1
Draft
Agent 2
Review
Agent 3
Polish
Output

Use when: Clear linear dependencies, progressive refinement (draft-review-polish), data transformation pipelines.

Avoid when: Stages can run in parallel, early stages might fail and poison downstream, workflow needs backtracking.

Example: Contract generation - template selection agent, clause customisation agent, compliance review agent, risk assessment agent.

Concurrent (Fan-out/Fan-in)

Multiple agents process the same input simultaneously. Results are aggregated.

Dispatch
Input
Dispatcher
In Parallel
Each agent works independently
Agent 1
Technical analysis
Agent 2
Financial analysis
Agent 3
Risk analysis
Combine
Aggregator
Combined Result

Use when: Independent analyses from multiple perspectives, time-sensitive scenarios where parallel processing reduces latency, ensemble reasoning.

Avoid when: Agents need to build on each other's work, resource constraints make parallel execution impractical, no clear conflict resolution strategy.

Example: Investment analysis - fundamental analyst, technical analyst, sentiment analyst, ESG analyst all evaluate the same stock simultaneously.

Handoff (Routing/Delegation)

One active agent at a time. Each agent can transfer control to a more appropriate specialist.

Triage
User
Triage Agent
Specialists
Routed by issue type
Billing Agent
Billing issues
Technical Agent
Technical issues. Escalates when it cannot resolve
Human Support
Takes escalations from the technical agent
Outcome
Done
Resolved

Use when: Right specialist emerges during processing, different security boundaries per agent, customer support routing.

Avoid when: Appropriate agent is known upfront (use direct routing instead), multiple agents should work concurrently, risk of infinite handoff loops.

Example: Customer support - triage agent routes to billing, technical, or account specialist based on conversation context.

Group Chat (Roundtable)

Multiple agents participate in a shared conversation thread. A chat manager controls turn order.

Each turn reads the shared thread
Chat Manager
Agents A, B, C
Shared Thread

Use when: Consensus-building, brainstorming, iterative maker-checker validation, quality assurance with structured review.

Avoid when: Simple task delegation is sufficient, real-time processing makes discussion overhead unacceptable, more than 3 agents (control becomes difficult).

Example: Park development proposal - community engagement agent, environmental planning agent, and budget agent debate trade-offs before recommendation.

Magentic (Dynamic Planning)

A manager agent builds and adapts a task plan dynamically. It consults specialists, updates the plan based on findings, and tracks progress in a ledger.

Problem
The plan is rebuilt each round
Manager Agent
Agents
Goal achieved?
Result
Task Ledger
Kept by the manager
Ledger
Goals, progress, and status

Use when: Open-ended problems without a predetermined solution path, need to produce a documented plan, agents have tools that modify external systems.

Avoid when: Solution path is deterministic, task is low complexity, time-sensitive (this pattern is slow to converge).

Example: Incident response - manager agent creates remediation plan, consults diagnostics agent, infrastructure agent, and rollback agent, adapts plan as new information emerges.

Pattern Selection

PatternCoordinationRoutingBest forWatch out for
SequentialLinear pipelineDeterministic orderStep-by-step refinementEarly failures propagate
ConcurrentParallelDeterministic or dynamicMultiple perspectives, latency-sensitiveContradictory results need resolution
HandoffDynamic delegationAgents decide transfersRight specialist emerges during processingInfinite handoff loops
Group ChatConversationalManager controls turnsConsensus, maker-checker validationConversation loops, hard to control
MagenticPlan-build-executeManager assigns dynamicallyOpen-ended problemsSlow to converge, stalls on ambiguity

Implementation

LangGraph (most flexible)

from langgraph.graph import StateGraph, MessagesState, START, END
from langgraph.prebuilt import create_react_agent

# Define specialist agents
researcher = create_react_agent(llm, tools=[search_tool, web_tool])
writer = create_react_agent(llm, tools=[write_tool])
reviewer = create_react_agent(llm, tools=[review_tool])

# Sequential pipeline
graph = StateGraph(MessagesState)
graph.add_node("research", researcher)
graph.add_node("write", writer)
graph.add_node("review", reviewer)

graph.add_edge(START, "research")
graph.add_edge("research", "write")
graph.add_edge("write", "review")
graph.add_edge("review", END)

pipeline = graph.compile()
result = pipeline.invoke({"messages": [{"role": "user", "content": "Write a technical brief on RAG evaluation"}]})

Azure Foundry Connected Agents (managed)

# Handoff pattern with Azure Foundry
billing_agent = client.agents.create(
    model="gpt-4o", name="billing",
    instructions="Handle billing inquiries only.",
    tools=[{"type": "function", "function": billing_tools}]
)

tech_agent = client.agents.create(
    model="gpt-4o", name="technical",
    instructions="Handle technical support only.",
    tools=[{"type": "file_search"}]
)

triage = client.agents.create(
    model="gpt-4o", name="triage",
    instructions="Route to billing or technical based on user intent.",
    tools=[
        {"type": "connected_agent", "connected_agent": {"id": billing_agent.id}},
        {"type": "connected_agent", "connected_agent": {"id": tech_agent.id}}
    ]
)

Production Considerations

Context management

Each agent transition grows the context window. Strategies:

  • Summarise between agents (pass summary, not full history)
  • Scope context per agent (only pass what that agent needs)
  • Use external state store for shared data (not the message thread)

Failure handling

  • Set iteration caps on every loop (prevent infinite cycles)
  • Implement circuit breakers on agent-to-agent calls
  • Validate agent output before passing to next agent
  • Design graceful degradation (escalate to human when stuck)

Cost

Multi-agent multiplies model invocations. Each agent consumes tokens for instructions, context, reasoning, and tool calls.

  • Assign cheaper models to simpler agents (GPT-4o-mini for routing, GPT-4o for reasoning)
  • Monitor token consumption per agent per run
  • Apply context compaction between agents

Framework Comparison

FrameworkPatterns supportedStrengths
LangGraphAll (graph-based, fully custom)Most flexible, checkpointing, human-in-the-loop
Azure Agent FrameworkSequential, concurrent, group chat, handoff, magenticManaged, declarative workflows
CrewAISequential, concurrent (role-based)Simple API, role/goal/backstory model
OpenAI Agents SDKHandoff, tool-useLightweight, OpenAI-native
AutoGenGroup chat, concurrentResearch-oriented, flexible conversation patterns

Key Takeaways

  • Start with a single agent. Move to multi-agent only when a single agent demonstrably fails due to prompt complexity, tool overload, or security boundaries.
  • Pick the simplest pattern that works. Sequential and handoff cover 80% of multi-agent needs.
  • Limit group chat to 3 agents maximum. More agents make conversation control exponentially harder.
  • Context management between agents is the primary engineering challenge. Summarise, scope, and use external state.
  • Monitor cost per agent per run. Multi-agent systems can burn tokens fast without visibility.

Next: Autonomous AI - once agents coordinate, the question is how much they should do without a person in the loop.