Agent Design Patterns

Nine ways to wire LLM calls into a system; all take a text task in and return one text answer plus tool side effects, differing only in the control flow between calls.

Core vocabulary

• Tool: a function you describe to the model; it emits a structured call request, your code runs it, you feed the result back as text. The only way a model touches the outside world.
• Token: unit read and billed, ~4 chars of English (500 words ≈ 650 tokens).
• Context: everything sent with one call: system prompt + tool definitions + conversation so far + new request.
• Context window: hard token cap on one call; exceeding it is an error, not gradual decay.
• Stateless: the interface remembers nothing, so a multi-step agent resends the whole conversation every step. This is why context size (= cost) is the axis every pattern trades against.

Cost model

• 1× = one plain call on prefix P. N× = reads/writes N times those tokens.
• P = fixed prefix (system prompt + tool defs), paid in full every call, never grows. a = per-turn delta (model reply + tool result).
• Growth of an n-turn loop: n·P + a·n²/2. The history term a·n²/2 is quadratic: double the turns, roughly quadruple it.
• N× is a token multiplier at a fixed tier, not dollars. Tiering (moving work to a cheaper model) is a separate lever: Routing is ~1.09× in tokens but 0.53× in dollars.

Choosing a pattern (by your task)

• Distinct input categories → Router (put in front, then pick a pattern per branch).
• Decomposable, fixed steps → Prompt Chaining.
• Decomposable, independent steps → Parallelization.
• Decomposable, count varies → Orchestrator–Worker.
• Not decomposable, quality cheaply scorable → Evaluator–Optimizer.
• Not scorable, must observe before every decision → ReAct.
• Not scorable, checkpoints suffice → Plan-and-Execute.

Rule: start at the simplest tier, escalate only on measured failure (a named accuracy floor on a named input slice).

The nine patterns

• Three freedom tiers: pure workflow (Chaining, Parallelization, Router — you write the flow), bounded agency (Orchestrator–Worker, Evaluator–Optimizer — model gets one decision), true agent (ReAct, Plan-and-Execute — model chooses next step).
• Reflexion and Autonomous are absent from the decision tree: Reflexion is a modifier (a base loop + a lesson store that survives the attempt); Autonomous is a true agent with the human deleted (a governance choice).

Key rules and gotchas

• Chaining: gate each step against the original input, not the previous step (subset test fields(draft) <= spec.declared_fields, not valid_json); a failed gate rejects, never continues; cap retries.
• Routing: always include a fallback route; log the routing distribution; let handlers escalate back. Validate the label before dispatch (label not in LABELS → unknown).
• Parallelization: sectioning splits work (combiner = concat if chunks don’t overlap, else an LLM merge call not in the N×); voting splits risk. Voting cuts variance, not bias — identical samples give correlated wrong answers; use diverse lenses. Let K-of-N decide only whether to block, never whether to report.
• Orchestrator–Worker: value is capability (context isolation), not speed — each worker gets a blank window; synthesizer is a separate call. Disjoint scopes in briefs; surface conflicts, don’t average. w·W (workers) is the expense, orchestrator O is ~6–10%.
• Evaluator–Optimizer: judge runs as a separate call with fresh context (avoids sycophancy + recency); feedback must be specific unmet criteria, not “rejected”. Cap rounds; rounds 1–2 capture most gain. Cache the judge’s system prompt (~0.1× input, needs ≥512 tokens).
• ReAct (Reason+Act): thought → action (tool_use) → observation (tool_result, user role). Loop is while resp.stop_reason == "tool_use". Per-turn cost is linear; running total is quadratic. A step cap is non-negotiable. Break loops with a trip message, not a halt.
• Plan-and-Execute: recon → strong planner once → cheap executor per step → replan only on failure (cap at 3, then fail loudly). Cheaper than ReAct because gap widens with horizon (linear vs quadratic). Only pattern where a human can review before execution.
• Reflexion: write to store only on failure; lessons go in the system prompt in fixed order (for caching). Only pays off cross-task if lessons transfer (same codebase/schema/API). Lessons must be specific and falsifiable; a resent store is a standing per-run token tax.
• Autonomous: needs all three to be safe — reversible actions, machine-checkable success signal, hard budget. Score drift against the frozen original goal string, never the current framing. Machine verification asks “did it work”; self-critique asks “was it the right task”.

The dangerous term

n·P + a·n²/2      ← only ReAct grows quadratically

Only ReAct has quadratic context growth, and it is the pattern most people default to — the source of most agent cost surprises. Cheapest patterns are also the most debuggable: every star lost is a decision handed to the model that you pay for and cannot replay.