Multi-Agent Systems

A single agent with many tools hits a ceiling. Prompt complexity grows, tool selection degrades, and security boundaries blur. Multi-agent systems solve this by splitting work across specialised agents that coordinate to complete a task.

The coordination overhead is real. Every pattern below adds latency, failure modes, and debugging complexity. Use the lowest level of complexity that reliably meets your requirements.

When Single Agent is Enough

Before adopting multi-agent, evaluate whether you actually need it.

Orchestration Patterns

Sequential (Pipeline)

Agents chained in a fixed order. Each processes the output of the previous one.

Use when: Clear linear dependencies, progressive refinement (draft-review-polish), data transformation pipelines.

Avoid when: Stages can run in parallel, early stages might fail and poison downstream, workflow needs backtracking.

Example: Contract generation - template selection agent, clause customisation agent, compliance review agent, risk assessment agent.

Concurrent (Fan-out/Fan-in)

Multiple agents process the same input simultaneously. Results are aggregated.

Use when: Independent analyses from multiple perspectives, time-sensitive scenarios where parallel processing reduces latency, ensemble reasoning.

Avoid when: Agents need to build on each other's work, resource constraints make parallel execution impractical, no clear conflict resolution strategy.

Example: Investment analysis - fundamental analyst, technical analyst, sentiment analyst, ESG analyst all evaluate the same stock simultaneously.

Handoff (Routing/Delegation)

One active agent at a time. Each agent can transfer control to a more appropriate specialist.

Use when: Right specialist emerges during processing, different security boundaries per agent, customer support routing.

Avoid when: Appropriate agent is known upfront (use direct routing instead), multiple agents should work concurrently, risk of infinite handoff loops.

Example: Customer support - triage agent routes to billing, technical, or account specialist based on conversation context.

Group Chat (Roundtable)

Multiple agents participate in a shared conversation thread. A chat manager controls turn order.

Use when: Consensus-building, brainstorming, iterative maker-checker validation, quality assurance with structured review.

Avoid when: Simple task delegation is sufficient, real-time processing makes discussion overhead unacceptable, more than 3 agents (control becomes difficult).

Example: Park development proposal - community engagement agent, environmental planning agent, and budget agent debate trade-offs before recommendation.

Magentic (Dynamic Planning)

A manager agent builds and adapts a task plan dynamically. It consults specialists, updates the plan based on findings, and tracks progress in a ledger.

Use when: Open-ended problems without a predetermined solution path, need to produce a documented plan, agents have tools that modify external systems.

Avoid when: Solution path is deterministic, task is low complexity, time-sensitive (this pattern is slow to converge).

Example: Incident response - manager agent creates remediation plan, consults diagnostics agent, infrastructure agent, and rollback agent, adapts plan as new information emerges.

Pattern Selection

Implementation

LangGraph (most flexible)

from langgraph.graph import StateGraph, MessagesState, START, END
from langgraph.prebuilt import create_react_agent

# Define specialist agents
researcher = create_react_agent(llm, tools=[search_tool, web_tool])
writer = create_react_agent(llm, tools=[write_tool])
reviewer = create_react_agent(llm, tools=[review_tool])

# Sequential pipeline
graph = StateGraph(MessagesState)
graph.add_node("research", researcher)
graph.add_node("write", writer)
graph.add_node("review", reviewer)

graph.add_edge(START, "research")
graph.add_edge("research", "write")
graph.add_edge("write", "review")
graph.add_edge("review", END)

pipeline = graph.compile()
result = pipeline.invoke({"messages": [{"role": "user", "content": "Write a technical brief on RAG evaluation"}]})

Azure Foundry Connected Agents (managed)

# Handoff pattern with Azure Foundry
billing_agent = client.agents.create(
    model="gpt-4o", name="billing",
    instructions="Handle billing inquiries only.",
    tools=[{"type": "function", "function": billing_tools}]
)

tech_agent = client.agents.create(
    model="gpt-4o", name="technical",
    instructions="Handle technical support only.",
    tools=[{"type": "file_search"}]
)

triage = client.agents.create(
    model="gpt-4o", name="triage",
    instructions="Route to billing or technical based on user intent.",
    tools=[
        {"type": "connected_agent", "connected_agent": {"id": billing_agent.id}},
        {"type": "connected_agent", "connected_agent": {"id": tech_agent.id}}
    ]
)

Production Considerations

Context management

Each agent transition grows the context window. Strategies:

• Summarise between agents (pass summary, not full history)
• Scope context per agent (only pass what that agent needs)
• Use external state store for shared data (not the message thread)

Failure handling

• Set iteration caps on every loop (prevent infinite cycles)
• Implement circuit breakers on agent-to-agent calls
• Validate agent output before passing to next agent
• Design graceful degradation (escalate to human when stuck)

Cost

Multi-agent multiplies model invocations. Each agent consumes tokens for instructions, context, reasoning, and tool calls.

• Assign cheaper models to simpler agents (GPT-4o-mini for routing, GPT-4o for reasoning)
• Monitor token consumption per agent per run
• Apply context compaction between agents

Framework Comparison

Key Takeaways

• Start with a single agent. Move to multi-agent only when a single agent demonstrably fails due to prompt complexity, tool overload, or security boundaries.
• Pick the simplest pattern that works. Sequential and handoff cover 80% of multi-agent needs.
• Limit group chat to 3 agents maximum. More agents make conversation control exponentially harder.
• Context management between agents is the primary engineering challenge. Summarise, scope, and use external state.
• Monitor cost per agent per run. Multi-agent systems can burn tokens fast without visibility.