The agent design round is 45 minutes, one vague sentence, and a whiteboard: turn it into five defensible artifacts using a seven-step method, and know what breaks.
The five artifacts you produce
- Tier decision (call / workflow / agent) with reason and what would change your mind.
- Loop diagram: what calls the model, what it can do, how it stops.
- Tool table: name, args, when to call, reversibility.
- Failure list: what breaks, the signal it broke, the guard.
- One cost number per task, plus the first lever to cut it.
The round scores whether you have a repeatable procedure and whether you know what breaks, not whether you can build it.
The seven steps (40 min + 5 buffer)
Boundaries to memorize (a clock shows time, not elapsed): 5 / 7 / 12 / 20 / 26 / 34 / 40. Each step consumes the previous one’s output, so order is fixed. Steps 3 and 6 carry the most signal; never cut them.
| # | Step | Min | Produces |
|---|---|---|---|
| 1 | Clarify | 5 | scope, scale, risk tolerance |
| 2 | Tier | 2 | call / workflow / agent |
| 3 | Loop | 5 | control-flow skeleton + 4 labels |
| 4 | Tools | 8 | action space (table, not JSON) |
| 5 | Context | 6 | window layout, cache plan |
| 6 | Failures | 8 | guards + detection |
| 7 | Evals & cost | 6 | the numbers |
Tier decision + cost multipliers
Ask in order: (1) Can you draw the flowchart today? -> workflow (+ agent escape hatch). (2) Does the next step depend on what the env returns? -> agent. (3) One shot enough? -> single call, else workflow. Most “design an agent” prompts are a disguised workflow.
- Workflow with
Nstages: aboutN×a single call (flat context each stage). - ReAct agent (reason + act):
5-20×, because context growth is quadratic (whole history resent every call). - The multiplier needs a denominator: an 8-call ReAct agent is ~
11×a standalone call but only ~1.4×an 8-stage workflow.
The loop and its four labels
Branch on stop_reason. tool_use -> execute, append results, run guards (pass = iterate, trip = halt + report). end_turn -> verify (it means the model stopped generating, not that it succeeded); fail re-enters the loop.
- Context: what’s in the window, in order.
- Tools: the action space.
- Stop: verifiable, not just
end_turn. - Guards: step cap, dollar ledger, loop detector (hash of
(tool, args)).
Context window: stable-first for caching
Render order is fixed by the API: tools -> system -> messages. You place only the cache breakpoint. Layout: tool schemas, system prompt, [breakpoint], retrieved docs, history, current turn (volatile, never cached).
- Caching is a prefix match (attention is causal: K/V depend only on preceding tokens). One edited token before the breakpoint invalidates the whole prefix.
- Growth: offload big tool outputs to disk + keep a pointer (turns
O(n²)intoO(n)); compact older history at threshold (costs one cold turn). - Persist across sessions: semantic (what’s true), episodic (what happened), procedural (what was learned).
- Watch
cache_read_input_tokens; zero means a silent invalidator.
Failure modes (choose 2-3, go deep, name detection)
Detection is the part that separates operators from paper designers. Frame each: what breaks, how you’d know, the guard.
| Failure | Detect | Guard |
|---|---|---|
| Infinite loop | hash of (tool, args) repeats | trip at 3; feed message back before halting |
| Runaway cost | budget ledger (sum all 4 usage fields) | warn 80%, hard stop 100%, return partial |
| Irreversible mistake | — (signal arrives too late) | read-only creds; two-phase commit; don’t expose it |
| Prompt injection | untrusted-content classifier | egress allowlist; break the lethal trifecta |
| Wrong tool | tool-call precision on labelled set | negative condition in the description |
| Hallucination | grounding check (resolve citation IDs) | require citations; instruct abstention; give_up branch |
| Silent truncation | stop_reason == "max_tokens" | stream; raise cap; never fake success |
| Degraded after deploy | cache_read_input_tokens drops to 0 | hash rendered prefix; alert on change |
| Late-session drift | plan vs. immutable original goal | re-score against stored original, re-inject verbatim |
- Lethal trifecta: private data + untrusted content + egress. Any two is survivable; egress is the removable leg (rendered markdown image URLs are egress).
- Safety goes in the harness, not the prompt: the model can’t route around a check it never sees. Approval tokens are minted outside its context.
Evals & cost
Evals: ~20 hand-written cases growing from prod failures; assert outcome strictly, trajectory loosely; prefer code assertions, reserve a calibrated LLM judge (~50 labels, report agreement rate); CI gate on aggregate success + zero safety violations (hard gate); run each case 3× and take majority.
Cost, per million tokens: input $5, output $25, cached read 0.10× input, cache write 1.25× input. State the shape in 4 numbers: calls, prefix size, per-turn add, output length.
- Example shape (8 calls, ~9k prefix, ~1.7k/turn, ~800 out, ~15k avg input): $0.76 uncached -> $0.34 cached -> $0.15 routed.
- Order: caching first (only free lever), then fewer turns / shorter outputs (output is ~47% once cached, uncacheable), then routing.
- Route at task boundaries, not per turn: caches are model-scoped, so downgrading mid-conversation forces a full re-read.
Gotchas
- Never ask “what model should I use?” — pick one and justify it.
- Don’t write full JSON schemas unless asked; the four-column table carries the signal.
- Clarify: ask ~3 questions that each change the design, then state assumptions and move. Not zero, not ten.
- “I’d add retries” is not a guard; it pays for the bug twice.
- Don’t-know: derive from the adjacent mechanism (labelled), or name the experiment + metric, or say so plainly and bound the blast radius. Bluffing is the only outright fail.
- Minute-25 checkpoint: cut order is JSON schemas -> memory detail -> eval mechanics -> cost precision. Never cut: loop diagram, failure modes + detection, one cost number.