A single agent gets you to "demo" fast. It almost never gets you to "Tuesday morning at 3am, the on-call pager." Once a workflow exceeds five or six tools and two domains of expertise, single-agent loops start drifting, because the model holds the whole problem in working memory and gets things subtly wrong. The fix is decomposition: small, specialized agents handed off through a typed message bus, with one supervisor watching the budget. LangGraph is the framework we landed on for that.
The short answer: multi-agent orchestration splits a complex AI workflow into small specialized agents — planner, researcher, writer — connected as a LangGraph graph with shared state, typed message contracts, and a deliberately simple supervisor that routes work and enforces budgets. It becomes necessary past roughly five or six tools and two domains of expertise, and it is the architecture AGI Software Solutions ships for production agent systems.
Why orchestration, not bigger context
"Just give it more context" is the wrong answer past a certain complexity. Long contexts dilute attention, raise cost, and make eval debugging miserable. What actually scales is splitting concerns: a planner that holds the overall goal, a research agent that holds search context, a writing agent that holds the draft, and so on.
LangGraph models this as a directed graph of nodes (agents and tools) with shared state. Each node reads what it needs, writes what it produces, and the supervisor decides where control goes next.
The supervisor pattern
The supervisor is dumb on purpose. The smarter you make it, the more it starts duplicating worker logic, and the messier failure modes get.
One node holds the plan. It does not call tools. It does not write content. It reads the shared state, asks "what is the next thing that needs to happen," and routes to a worker.
- One responsibility: deciding which worker runs next, or whether we are done.
- Bounded budget: total tool calls, total tokens, wall-clock time, all enforced by the supervisor.
- Visible to humans: every routing decision is logged with the reason and the state diff that produced it.
Message contracts between agents
Agents do not pass natural language at each other, because that is how you get the LLM-telephone-game failure mode. They pass typed structures instead. We use Pydantic models with strict schemas, and every node validates its input on entry.
This sounds boilerplate-heavy. It is. It also caught roughly half of the bugs that surfaced in week one of production.
Retry policies
Retries belong at the lowest layer that has enough context to know what failed. A flaky HTTP call gets retried inside the tool. A bad LLM response gets retried inside the agent with temperature shifted up. A wrong routing decision does not get retried at all; it gets escalated instead, because retrying it just burns budget.
We default to: three retries with exponential backoff on infrastructure errors, one retry with a corrective prompt on model errors, zero retries on logic errors.
Where humans belong in the loop
Two places: approval gates before irreversible actions, and triage queues for stuck runs.
- Approvals happen at known points: sending email, charging cards, mutating customer data. The agent pauses, posts a structured proposal to a queue, and waits.
- Triage happens when the supervisor gives up. Budget exhausted, three failed retries on the same step, or a confidence score below threshold → the run is paused with full context for a human to inspect.
What we wish we knew at the start
- Write the message schemas first. The graph follows naturally from the types.
- Instrument the supervisor heavily. If you can replay routing decisions, you can fix almost any bug.
- Resist building more roles than you need. Two specialists who hand off cleanly beat five that overlap.
- Test the unhappy paths. Eval suites that only check the happy plan miss every interesting failure.
Frequently asked questions
What is LangGraph?
A framework for building multi-agent systems as a directed graph of nodes (agents and tools) with shared state. Each node reads what it needs, writes what it produces, and a supervisor decides where control goes next.
When do you need multi-agent instead of a single agent?
Past roughly five or six tools and two domains of expertise. Single-agent loops drift there because the model holds the whole problem in working memory. Decompose into specialists with typed handoffs and one supervisor watching the budget.
What is the supervisor pattern?
One deliberately dumb node that holds the plan, calls no tools, writes no content. It decides which worker runs next or whether the run is done, enforces budgets, and logs every routing decision with its reason so runs can be replayed.
How do agents communicate with each other?
Typed message contracts, not natural language — prose between agents is the LLM-telephone-game failure mode. Strict schemas validated on entry to every node caught roughly half our week-one production bugs.
Where do humans belong in the loop?
Approval gates before irreversible actions, and triage queues for stuck runs: budget exhausted, repeated failures, or low confidence pauses the run with full context for a person to inspect.