Agent Foundations

Most systems called agents are not agents. In this lesson, we’ll settle the question with one test, then build the loop that every agent is made of.

By the end you should be able to:

• define six terms used throughout the series: model, tool, loop, harness, agent, orchestrator;
• place any system on one of three tiers, using two questions;
• write the agent loop in pseudocode and against the real API;
• read the exact message array sent on each turn;
• name the five mistakes and the errors they produce.

The shape of the whole thing

Before any mechanism, fix what goes in and what comes out.

You give the system a goal in plain English, for example “fix this failing test.” It returns a final text answer, plus whatever its tools changed along the way.

In between, the same kind of request is sent to the model over and over. Each request carries three things:

• tools: the definitions of the functions the model is allowed to ask for,
• system: the standing instructions,
• messages: the conversation so far.

Each response comes back as two things: a list of content blocks, and a single string called stop_reason that says why the model stopped.

What a content block is

A content block is one typed piece of a response: a small object with a type field naming what it is, plus the fields that type carries.

Two block types matter here:

• a text block (type: "text") holds prose the model wrote;
• a tool-use block (type: "tool_use") holds an id, the name of a tool, and the arguments to call it with.

One response can contain several blocks, of both kinds. Here is a two-block response, the model saying what it is about to do, then asking for it:

resp.content == [
    TextBlock(text="I'll look up the weather in Paris."),
    ToolUseBlock(id="toolu_01A", name="get_weather", input={"city": "Paris"}),
]

That is why the response is a list, not a string.

That is the entire input-output contract: a goal in, text plus side effects out; tools + system + messages sent, content blocks + stop_reason returned. Everything else in this chapter is the code around it.

The six words, in plain language

These terms are used loosely elsewhere. Here they mean exactly this:

• A large language model (LLM), “the model” from here on, is a function from text to text. It has no memory between calls: whatever it needs to know must be inside the request you send. A token is the unit it reads and bills in, a chunk of text averaging about four characters of English.

• A tool is an ordinary function you wrote, get_weather, run_tests, send_email. You describe it to the model as a name, a sentence of prose, and a schema of its arguments (a machine-readable list of the argument names and their types). The model cannot execute anything. It emits a structured request saying “call get_weather with {"city": "Paris"}”, and your code decides whether and how to run it.

• The loop is the repetition that follows from that split. Call the model; see if it asked for a tool; run the tool; append the result to the conversation; call the model again with the longer conversation. It ends when the model stops asking for tools, or when your code stops it.

• The harness is all the code you write around the model, assembling requests, running tools, counting spend, deciding when to quit, writing logs. The model is one stateless function call; the harness is the actual program.

• An agent is that loop plus one property: the model, not your code, chooses what happens next and when to stop.

• An orchestrator is a model call whose job is to split a task into pieces, hand each piece to a separate agent (a worker), and combine what comes back; the multi-agent chapter builds one.

Two facts everything else rests on

1. The API is stateless. The API is the network endpoint you send each request to. Stateless means the server keeps nothing between calls, so turn 20 only knows about turn 1 because you resent turn 1.

2. You are billed per token, on every request. Resending a growing conversation on every turn is where an agent’s money goes.

The LLM internals chapter derives both from the mechanism; you do not need it to follow this chapter.

The three tiers

Most projects that call themselves agents should not be agents. To place any system precisely, you need a ladder of tiers, a test for which rung it sits on, and a sense of what the top rung costs.

The ladder

Three rungs, ordered by how much freedom you hand the model. Reading left to right, more of the control flow moves out of your code and into the model.

flowchart LR
    A["Single LLM call<br/>prompt → output<br/>least freedom"] --> B["Workflow<br/>you write the control flow<br/>one step up"]
    B --> C["Agent<br/>model writes the control flow<br/>most freedom"]

    style A fill:#2d6a4f,color:#fff
    style B fill:#40916c,color:#fff
    style C fill:#95d5b2,color:#000

What changes between the rungs is one substitution, who computes the next step:

• A single call has no step N+1 at all. One prompt, one output, done.
• In a workflow, you write the control flow. The sequence of steps exists in your code before any request is sent.
• In an agent, the model writes the control flow at runtime, one decision at a time. It never exists anywhere you can read before the run.

Comparing the three tiers

Two terms the table uses:

• Fixed DAG: a directed acyclic graph: a set of stages wired together in advance, with no cycles. Every possible path through it can be drawn in advance.
• Trace: the recorded log of one run: every request sent, every response, every tool call and its result. Debugging an agent means reading traces, because there is no source code describing the path it took.

The cost column is in multiples of what a single model call costs; the 5–20× is derived in The cost of an agent below.

The dividing line is not tool use

A workflow can call tools. Tool use is not the test.

What distinguishes an agent is that the model chooses which step comes next, and when to stop. Two lines of pseudocode make the difference exact. f is whatever computes the next step, and a forward pass is one run of the model over its whole input, producing its next output:

workflow:  next_step = f(current_step)              # f is your code
agent:     next_step = model(history, tools)        # f is a forward pass

So in an agent, the function that picks the next step is not one you can read. It is a model inference, and it runs afresh every turn.

That substitution is what buys you adaptivity and what costs you predictability. You cannot enumerate the paths, so you cannot test them exhaustively, so every guarantee has to come from invariants: properties you enforce on every path, such as “this agent can never send an email without approval.” That is the subject of the reliability and guardrails chapter.

Running the rule on three real systems

Apply the substitution to three systems: one that reads like an agent and isn’t, one that genuinely sits on the line, and one that is unambiguously an agent. The middle case is what makes the definition usable.

(a) A model-based router: looks like an agent, isn’t.

A call to a small, cheap model reads an incoming support ticket and picks one of four branches: refund, technical, abuse, fallback.

A model is choosing what runs next, so next_step = model(...) seems to fit. The verdict is still workflow, for two reasons. First, the branch set is enumerated in your code at design time, so the reachable state space, the complete set of situations the system can ever be in and therefore the complete set of runs you would have to exercise to test it exhaustively, is four paths, all of which you can test. Second, the model makes exactly one decision per request. After the branch is chosen, the rest of the run is fixed code. You planned all four routes; the model only picked an index into your list.

(b) An evaluator–optimizer loop: the genuinely ambiguous one.

A generator drafts an answer, a separate model acting as a judge scores the draft, and on a failing verdict the loop runs again.

Now a model decides whether there is a step N+1. It is choosing when to stop, which is half of the agent definition.

The verdict is workflow with a bounded loop. The step sequence is still fixed, generate, then evaluate, then generate. The iteration count is capped by your code. And the model chooses from a two-element set: {iterate, stop}. You cannot enumerate every possible draft, but you can enumerate every possible path, which is what testability needs.

It flips, though. Let the judge return which of several repair tools to invoke instead of a pass/fail bit, or drop the round cap so the loop ends only when the judge says so, and both halves of the definition now hold: an unbounded action space and model-owned termination. Same diagram, same two model calls per round, different tier. So you classify by what the model is allowed to choose from, not by how the boxes are drawn.

(c) “Fix this failing test,” with a shell tool: an agent.

Try to write the paths down. The model may read the test, then the implementation, then a third file it inferred from the traceback. Or it may run the test first to see the failure. Or run git log on the file to find the regression.

It stops when the test passes, which nothing in your code can predict a turn ahead. There is no finite branch set to enumerate, because the argument space of a shell command is unbounded, and the number of steps is a property of the bug, not of your design.

The verdict is agent. The tool count here is one, which is why “does it call tools” was never the question.

The rule, sharpened

The model choosing among branches you wrote is a workflow. The model choosing from an unbounded action space, and choosing when to stop, is an agent.

Two diagnostics fall out of it, and you can check both before writing any code:

1. Can I draw the complete set of paths?
2. Does my code, not the model, decide when the run ends?

Two yeses mean you have a workflow. Run them on the tier table’s examples: “classify a ticket” has one path, “extract → validate → route” has a fixed three, and “fix this failing test” has a number nobody can write down.

The cost of an agent

An agent has no server-side memory, so every turn resends the full conversation so far. That single fact is the cost model.

Write P for the fixed prefix that rides on every request, the system prompt plus the tool schemas, and a for the tokens each turn appends. Turn n sends P + n·a, so the total over n turns is:

n·P  +  a·(1 + 2 + … + n)   =   n·P  +  a·n(n+1)/2   ≈   n·P + a·n²/2

The prefix is paid n times; the per-turn additions stack up quadratically. So a workflow with N independent stages grows linearly (N·(P + a)), while an agent grows quadratically: double the turns and the n² term quadruples.

Back-of-envelope with a 2,000-token prefix and 500 tokens added per turn: a 20-turn agent runs on the order of 145,000 tokens. That is 58× one single call (2,500 tokens) and 2.9× twenty independent single calls (50,000 tokens). Always say which denominator you mean; the two answers differ by more than an order of magnitude. The tier table’s 5–20× is the same formula at 4 to 10 turns.

Two consequences of keeping P in the formula:

• The prefix dominates. Drop it (P = 0) and the same 20-turn agent is only 10.5× twenty single calls instead of 2.9×. So the first cost lever is making P cacheable, not making the agent shorter. Caching means the provider stores the processed form of an unchanged prefix and bills roughly a tenth of the normal rate to reuse it; the LLM internals chapter derives the rules.
• The multiple is open-ended. At 40 turns the agent is already ~196× one single call. The growth never levels off.

This is the honest reason to prefer a workflow, and a much stronger one than “workflows are simpler.”

The agent loop

Every agent is a variation on one loop, and almost everything that goes wrong with it goes wrong at its exits, the endings it must branch on, and the guards that end it when the model won’t.

Focus on the diamond D. It has five outgoing arrows, and four of those five are branches people forget to write. The letters on the nodes match the pseudocode line comments in Tier 1 — pseudocode.

flowchart TD
    S(["S · User goal"]) --> M["M · Call model with<br/>history + tools"]
    M --> D{"D · stop_reason"}
    D -->|tool_use| E["E · Execute tools"]
    E --> A["A · Append results<br/>to history"]
    A --> G{"G · Guards pass?"}
    G -->|yes| M
    G -->|no| X(["X · Halt: budget /<br/>step cap / loop"])
    D -->|end_turn| F(["F · Final answer"])
    D -->|max_tokens| T(["T · Truncated — NOT success"])
    D -->|refusal| R(["R · Declined — content may be empty"])
    D -->|pause_turn| P["P · Paused — append<br/>and resend"]
    P --> M

    style S fill:#1d3557,color:#fff
    style F fill:#2d6a4f,color:#fff
    style X fill:#9d0208,color:#fff
    style T fill:#bc6c25,color:#fff
    style R fill:#bc6c25,color:#fff
    style P fill:#bc6c25,color:#fff

Four things to take from it.

S · User goal is the only input; everything else in the diagram is state your harness carries between calls.

D · stop_reason has five branches because the API returns five values you must handle (see Stop reasons you must handle). A loop coded from a four-branch diagram has the pause_turn bug built in.

Only two branches come back. tool_use goes round through execute → append → guards, and pause_turn goes straight back to the model call. The other three leave the loop for good.

The three amber branches all fail quietly. Truncated (max_tokens) and Paused (pause_turn) are both well-formed success responses, HTTP 200, that a naive else: return text hands back to your caller as the finished answer. Declined (refusal) fails the other way: it crashes on resp.content[0], because there may be nothing at index 0.

Emptiness is not exclusive to refusal: an end_turn can arrive with no text block and break the success path the same way. So the habit to build is to read stop_reason before you index content, on every branch.

Why pause_turn exists

pause_turn is the only non-final branch besides tool_use, and its arrow goes straight back to the model call, because a pause is a turn that has not finished, not a turn that ended.

It comes from a server tool: a tool the provider runs on its own infrastructure, not in your harness. You never see a tool_use block for it and you never write code to execute it. Web search and code execution are the usual examples. You declare the tool and the provider does the work server-side.

The provider is running its own little loop over there, search, read, search again, and that loop has an iteration cap. When it hits the cap before the model is done, you get back everything produced so far with stop_reason: "pause_turn".

To resume: append that content to the conversation as the assistant turn, and send the request again with no new user message. The provider recognises the unfinished turn and picks up where it left off. Adding a “please continue” message instead changes the conversation the model is resuming, which is the mistake.

The guard node, and why it sits where it does

A guard is a check your harness runs between turns that can end the run regardless of what the model wants.

Guards pass? (G) runs after results are appended to history and before the next model call. That is the only point in the cycle where the run’s full cost so far is known and no further money has been committed. Put it before tool execution and you halt on a budget you have not yet spent; put it after the model call and you have already paid for the turn that broke the cap.

The Halt node names three causes, each a different piece of harness state:

• Step cap: an integer counter of how many times round the loop you have been. The cheapest guard, and the one whose absence is mistake #5 below.
• Budget: a running ledger of tokens or dollars, summed across every call. A step counter cannot substitute for it, because turn cost is not constant: the quadratic above means turn 20 costs far more than turn 2.
• Loop: a repeated fingerprint of (tool, args), meaning the model has asked for the identical call it already made.

All three are the harness’s job, not the model’s: a model asked to respect a budget it cannot observe is being asked to guess.

What actually happens on one iteration

Six things happen between one model call and the next. Steps 2 and 3 are the two phases of inference, which is where the loop’s cost and latency come from; the LLM internals chapter derives them.

1. Serialize. Your harness turns tools, system, and messages into one token sequence, in that order. That order is why stable content must come first if it is to be cacheable.

2. Prefill. The whole input is processed in parallel, in one pass. Processing a token produces a small bundle of numbers the model needs again for every later token, kept in a KV cache (key–value cache). If the opening span of your request matches one the provider already processed, it reuses that stored cache and bills roughly 10% of the normal rate for the reused part.

3. Decode. Output tokens are generated strictly one at a time, each re-reading everything before it. This is why output tokens cost several times what input tokens cost, and why a long answer takes longer than a long question. It is also why streaming exists, delivering each token as it is produced instead of waiting for the whole response.

4. Tool request, if any. If the model wants a tool, it emits a structured tool-call block, not prose. The provider’s decoder forces that block to match your schema by refusing any token that would break it, the code around the model enforces the schema, not the model itself. stop_reason comes back as "tool_use".

5. Execute and append. Your harness runs the requested calls and appends the results to history as one single user message. Sending them as several is mistake #3 below.

6. Guard. Guards run on the now-longer history. If they pass, control returns to step 1 with a longer sequence. If not, the run halts.

Step 6 is the whole cost story. Nothing is remembered server-side; the only reason turn 20 knows about turn 1 is that you resent it. And the Final answer node is reached only from end_turn: every other exit is a partial result in a success-shaped response object.

The four moving parts

Once the loop is in place, there are exactly four things left to design, and each has its own chapter:

• Context: everything the model sees this turn. The memory and context chapter.
• Tools: the action space, the complete set of things the agent can do. The tools and MCP chapter.
• Stop condition: how the loop ends when things go right. Stop conditions below.
• Guards: what stops it when the stop condition doesn’t fire. The reliability and guardrails chapter.

Building it: three tiers

Here is the same loop written three times, as six lines of pseudocode, as an explicit graph in a framework, and as production code against the real API. Reading them in order shows which lines are essential, which are library convenience, and which break in production.

Tier 1 — Pseudocode

Six lines, one per node of the loop diagram; the trailing comments name the node each line corresponds to.

history = [user_goal]
while True:
    reply = model(history, tools)                     # M: call model
    if reply.wants_no_tools: return reply.text        # D -> end_turn -> F
    results = [run(call) for call in reply.tool_calls] # E: execute tools
    history += [reply, results]                       # A: append, ONE message
    if over_budget() or over_steps(): halt()          # G -> X

reply.wants_no_tools collapses all four non-tool_use endings into a single branch. That is deliberate: this tier is the shape of the loop (“tools, or done”) not a loop you would ship. Tier 3 below splits that one branch into the four the real API requires.

Two details are choices, not accidents:

• history += [reply, results] appends reply whole, the assistant turn including its tool-call blocks, and then all of results as a single message. Splitting that list into one message per result is mistake #3 below, the one that costs 3× with no error to tell you.
• The guard sits at the end of the loop body, after the append. Moving it above model(...) would halt the run over spend that has not happened yet.

Tier 2 — LangGraph

LangGraph is an open-source library that expresses the loop as an explicit state machine, a set of named nodes plus rules for which runs next, instead of a while statement.

Seven of its names appear below. Read these first, or the block is noise:

• StateGraph: the builder. You construct it with the state schema the graph carries, then hang nodes and edges off it.
• MessagesState: a prebuilt state schema whose only field is a messages list with append semantics. A node returns the messages it adds, not the whole history.
• ToolNode: a prebuilt node that reads the tool calls off the last message, runs them, and returns the result messages.
• START and END: the graph’s sentinel nodes, marking entry and exit.
• .compile(): freezes the assembled builder into a runnable graph, and attaches the checkpointer.
• add_conditional_edges: attaches a Python function that returns the name of the next node. This is why should_continue below is the same branch as the stop_reason diamond, written as a test over the last message.

Two names are yours, not the library’s:

• llm: a chat-model client, e.g. ChatAnthropic(model="claude-opus-5") from langchain_anthropic.
• tools: the list of Python functions the model may call. llm.bind_tools(tools) attaches that list so every call advertises those tools.

The value of the framework is not the abstraction. It is the checkpointer: the component that writes the graph’s state out after every node, so a run can be resumed later. That is what buys durable state, resumption after a crash, and human approval pauses that survive a process restart.

from langgraph.graph import StateGraph, START, END, MessagesState
from langgraph.prebuilt import ToolNode
from langgraph.checkpoint.memory import MemorySaver

def call_model(state: MessagesState):
    return {"messages": [llm.bind_tools(tools).invoke(state["messages"])]}

def should_continue(state: MessagesState):
    return "tools" if state["messages"][-1].tool_calls else END

g = StateGraph(MessagesState)
g.add_node("agent", call_model)
g.add_node("tools", ToolNode(tools))
g.add_edge(START, "agent")
g.add_conditional_edges("agent", should_continue, ["tools", END])
g.add_edge("tools", "agent")          # the cycle

app = g.compile(checkpointer=MemorySaver())
app.invoke({"messages": [("user", goal)]},
           config={"configurable": {"thread_id": "run-1"}})

The thread_id is the resume key: it names the conversation whose saved state should be loaded. Without a checkpointer, a crash mid-run loses everything.

But MemorySaver does not deliver durability. It is in-process and in-memory. It survives a graph interrupt within one running process, which is what makes human approval pauses work in a notebook, and it dies with the process. Crash survival needs a durable backend: SqliteSaver for a single host, PostgresSaver for anything on more than one. MemorySaver is the demo, not the design.

Tier 3 — Anthropic SDK

The loop written directly against the API through the official Python client (its SDK), with no framework in between. It is short enough to write from memory. The part that matters is the run of stop_reason checks before the tool-handling code, five branches in a deliberate order.

import anthropic

client = anthropic.Anthropic()

SYSTEM = "You are a weather assistant. Answer only from tool results."

def handle_refusal(details) -> str:
    # stop_details is populated only when stop_reason == "refusal".
    cat = getattr(details, "category", None)
    return f"Declined ({cat}); nothing to retry with the same prompt."

TOOLS = [{
    "name": "get_weather",
    "description": "Get current weather for a city. Use when the user asks "
                   "about weather, temperature, or conditions right now.",
    "input_schema": {
        "type": "object",
        "properties": {"city": {"type": "string", "description": "City name"}},
        "required": ["city"],
    },
}]

def run_tool(name: str, args: dict) -> str:
    if name == "get_weather":
        return f"18C and raining in {args['city']}"
    raise ValueError(name)

def agent(goal: str, max_steps: int = 12) -> str:
    messages = [{"role": "user", "content": goal}]

    for _ in range(max_steps):                       # guard: step cap
        resp = client.messages.create(
            model="claude-opus-5",
            max_tokens=8192,
            system=[{"type": "text", "text": SYSTEM,
                     "cache_control": {"type": "ephemeral"}}],   # stable prefix
            tools=TOOLS,
            messages=messages,
        )

        if resp.stop_reason == "refusal":            # HTTP 200, content may be []
            return handle_refusal(resp.stop_details)
        if resp.stop_reason == "max_tokens":         # silently truncated
            raise RuntimeError("output truncated; raise max_tokens or stream")
        if resp.stop_reason == "pause_turn":         # server-tool turn paused, not finished
            messages.append({"role": "assistant", "content": resp.content})
            continue                                 # re-send with no new user message
        if resp.stop_reason != "tool_use":           # end_turn
            # the "" default is load-bearing: end_turn can carry no text block
            return next((b.text for b in resp.content if b.type == "text"), "")

        messages.append({"role": "assistant", "content": resp.content})

        results = []
        for block in resp.content:
            if block.type != "tool_use":
                continue
            try:
                out, err = run_tool(block.name, block.input), False
            except Exception as e:
                out, err = f"Error: {e}", True        # errors go back as results
            results.append({
                "type": "tool_result",
                "tool_use_id": block.id,
                "content": out,
                "is_error": err,
            })

        messages.append({"role": "user", "content": results})

    raise RuntimeError("step cap exceeded")

The cache marker, and why it does nothing here

"cache_control": {"type": "ephemeral"} on the system block is the caching marker from the cost section. Ephemeral means short-lived: the provider keeps the processed form of everything up to the marker for a few minutes and bills roughly a tenth of the normal rate to reuse it, then drops it. You mark a boundary; reuse either happens or it doesn’t.

At this example’s size the marker is a no-op, and it is worth seeing why. A cached prefix has to clear the model’s minimum cacheable length: the smallest prefix the provider will store. That floor is 512 tokens on claude-opus-5, and the floors are not ordered by generation: 1,024 on claude-sonnet-5, 4,096 on claude-haiku-4-5. This prefix (the get_weather schema plus the short system string) is on the order of 85 tokens, about a sixth of the floor, so nothing caches.

Below the floor nothing caches, there is no error, and the only symptom is cache_creation_input_tokens: 0 in the response. Keep the marker anyway: a real agent’s dozen-odd tool schemas run to a couple of thousand tokens and clear 512 comfortably. But verify rather than assume: print usage.cache_read_input_tokens on the second call, and if it is zero you are paying full price for the prefix on every turn.

Six lines carry the whole lesson

And none of them is the API call itself.

1. The stop_reason checks are ordered, and the order is the point. refusal, max_tokens, and pause_turn are all tested before the != "tool_use" fallthrough. Invert them and each becomes “not tool_use”, falls into the end_turn branch, and a truncated half-answer gets returned as the finished product.

2. The pause_turn branch appends and continues instead of returning. It appends resp.content as the assistant turn, then loops, re-sending with no new user message so the provider finishes the turn it was in. Without this branch, != "tool_use" catches pause_turn too and the loop returns a partial answer as final. This agent declares only get_weather, which runs in your harness, so it never actually pauses today, the branch is here because it costs three lines now and is invisible to debug later, on the day someone adds a server tool.

3. A pause spends a step, because the branch is a continue inside for _ in range(max_steps). The counter is a lap counter, not a progress counter, and a resumed turn is a lap. Three consecutive pauses against max_steps = 3 exhaust the cap having run no tools, then exit through raise RuntimeError("step cap exceeded"): a true statement and a misleading diagnosis. With server tools, give pauses their own counter or size max_steps knowing some laps buy nothing.

4. messages.append({"role": "assistant", "content": resp.content}) appends the block list, not the text. resp.content holds both the text block and the tool-call blocks. Keep only .text and the tool result you send next references a call the API has no record of, mistake #1 below.

5. The end_turn branch’s next(...) carries an explicit "" default. A bare next(b.text for b in resp.content if b.type == "text") raises StopIteration the moment an end_turn response contains no text block. The default converts that into an empty answer your caller can test for, something a goal predicate can catch, which an escaping StopIteration is not.

6. The except clause returns a string instead of re-raising. out, err = f"Error: {e}", True turns a tool failure into a tool_result marked is_error: true, which the model reads and can act on. Raising instead ends the run over a fixable typo, mistake #4 below.

One thing the code does not do: only one guard is implemented. The for _ in range(max_steps) is the step cap. The budget ledger and the loop detector are yours to add; a production harness has all three, and the reliability and guardrails chapter builds them.

A worked trace

Abstract descriptions hide what matters most: the exact shape of the message array. Here is a two-turn run, message by message, followed by the five ways people corrupt that array.

The question is “What’s the weather where the Eiffel Tower is?”

Turn 1 request: one message. The system and tools ride on every call and are not part of this array:

messages = [
    {"role": "user", "content": "What's the weather where the Eiffel Tower is?"},
]

Turn 1 response: stop_reason: "tool_use", containing both a text block and a tool-use block:

resp.content == [
    TextBlock(text="I'll look up the weather in Paris."),
    ToolUseBlock(id="toolu_01A", name="get_weather", input={"city": "Paris"}),
]

Turn 2 request: you append two messages, and the array is now three long. The assistant message carries both blocks verbatim, and the tool_use_id in the result matches the id the model handed you:

messages = [
    {"role": "user",      "content": "What's the weather where the Eiffel Tower is?"},
    {"role": "assistant", "content": resp.content},          # BOTH blocks, verbatim
    {"role": "user",      "content": [
        {"type": "tool_result", "tool_use_id": "toolu_01A",   # id must match
         "content": "18C and raining in Paris"},
    ]},
]

Turn 2 response: stop_reason: "end_turn", and you are done. Two model calls, one tool call, three messages.

The five mistakes, with the errors they produce

1. Appending only the text.

messages.append({"role": "assistant", "content": resp.content[0].text})   # WRONG

The tool-use block is gone, but your next message contains a tool_result referencing toolu_01A. The API sees a result for a call that never happened and rejects the request with HTTP 400:

400, tool_result block(s) provided when previous message does not contain any tool_use blocks

2. Missing a result for a parallel call. The model can request several tools in one turn. Change the question to “weather in Paris, Berlin, and Tokyo?” and the same turn comes back with three tool-use blocks in a single assistant message:

resp.content == [
    ToolUseBlock(id="toolu_01A", name="get_weather", input={"city": "Paris"}),
    ToolUseBlock(id="toolu_01B", name="get_weather", input={"city": "Berlin"}),
    ToolUseBlock(id="toolu_01C", name="get_weather", input={"city": "Tokyo"}),
]

If the Tokyo lookup throws, your except block logs it and moves on, and you send back only two results:

400, messages.2: tool_use ids were found without tool_result blocks immediately after: toolu_01C

Every tool-use block needs exactly one matching tool_result, even when the result is an error string: {"type": "tool_result", "tool_use_id": "toolu_01C", "content": "Error: ...", "is_error": True}. The block is mandatory; its content is where the failure goes.

3. Splitting results across messages.

for r in results:
    messages.append({"role": "user", "content": [r]})        # WRONG

This one is worse than an error, because it works. Nothing is rejected. But you have shown the model a conversation in which parallel calls were answered one at a time, and the model imitates the pattern it is shown: it stops emitting parallel calls. Your agent silently gets slower over a session, and no error tells you.

The slowdown factor is exactly the fan-out you gave up, the number of tool calls the model issues in a single turn. Three cities in one turn cost one round-trip; the same three one-per-turn cost three round-trips. It is 3× here because the example has three cities; a task that fanned out to eight would degrade eightfold.

4. Raising on tool failure. A ValueError propagates out of your loop and the run dies. But the model can usually fix its own bad input in one turn, and a returned message like "Error: city 'Pariss' not found. Did you mean 'Paris'?" is a better instruction than anything in your system prompt.

Retry infrastructure failures (timeouts, a service being down) in the harness; return semantic failures, the ones caused by a wrong argument, to the model.

5. No step cap. Without a step cap, a confused model loops until your budget runs out.

Stop conditions

A step cap is only the bluntest way for a run to end. A run can end six ways, and they are not interchangeable: how far you should trust the result depends on which ending fired.

flowchart TD
    L[Agent loop] --> N{Why did it stop?}
    N --> B["Goal predicate<br/>tests pass, schema valid<br/>ideal"]
    N --> A["end_turn<br/>model is satisfied<br/>acceptable"]
    N --> C["Step cap<br/>you under-scoped it"]
    N --> D["Token / dollar budget<br/>you under-scoped it"]
    N --> E["Loop detector<br/>a bug — report it"]
    N --> F["Human veto<br/>a person intervened"]

    style B fill:#2d6a4f,color:#fff
    style A fill:#40916c,color:#fff
    style C fill:#bc6c25,color:#fff
    style D fill:#bc6c25,color:#fff
    style E fill:#9d0208,color:#fff
    style F fill:#1d3557,color:#fff

Why did it stop? is the question a trace has to answer. A goal predicate, a test your own code runs to decide whether the goal was actually met, is the ideal. end_turn, model satisfied, is merely acceptable. Step cap and budget both mean you under-scoped it. The loop detector firing is a bug, to be reported and never silently swallowed. A human veto means a person intervened.

Two of the six are mechanisms, not counters, so they need defining:

• A loop detector computes a fingerprint of (tool_name, arguments) on every step, a short string such that two identical calls give the identical string, and halts when the same fingerprint recurs N times, since a model repeating itself is stuck, not working. The naive version catches only literal repeats; a read/write/read cycle and a run-tests/edit/run-tests stall both defeat it, which is why real detectors also fingerprint a snapshot of the world state and track how many steps have passed since anything measurably improved.

• A human veto is a blocking approval gate on a specific class of action, not on the agent as a whole. The agent runs freely until it proposes something irreversible, then stops and waits for a decision issued outside its context, so no amount of generated text can forge it. The reliability and guardrails chapter builds the propose-then-confirm mechanism.

Why end_turn is weak evidence

A model does not decide to finish. It produces one token at a time by scoring every word-piece it could say next and picking from that spread. One of the things it can pick is a special piece that means this response is over. end_turn is the API telling you that is the piece that came out.

So end_turn is a statement about which token the model chose, not about the state of the world. It correlates with task completion because of how the model was trained, but nothing enforces the correlation, and when it breaks, the failure is silent and confident.

Prefer a predicate your harness can check. The left column is what people settle for; the right column is the mechanical replacement:

That last row is the trap. An empty task queue with an unmet goal is a failure, and an agent that reports it as success is worse than one that crashes, since a crash gets investigated.

Stop reasons you must handle

The table works as a dispatch table for your loop: every row is a branch it needs, the third column is the bug you ship without that branch, and the fourth is what the branch should do.

These are five rows, not five stop reasons in total. The API documents six. The sixth is stop_sequence, absent here because it cannot occur unless you asked for it: it means the model emitted one of the strings you passed in stop_sequences, and this chapter’s loop passes none. If you do pass one, treat it as an end_turn that stopped where you told it to, and read stop_sequence on the response to learn which string fired.

The refusal row comes from a separate safety model screening the exchange, not from the model declining in prose, which is why it arrives as a stop_reason and can leave content empty instead of filled with an apology.

The last three rows are the ones people don’t code for, and all three fail quietly. pause_turn is the sneakiest: the response is well-formed, content is populated, and the only thing wrong is that the model was not finished.

Reactive vs. proactive

Everything so far assumed a person watching the run. Remove the person, an agent triggered by a schedule, not a message, and one failure is common: a message delivered twice to an agent that produces something different each run.

A reactive agent runs because a user sent a message. A proactive agent runs because a clock, a webhook (a request another system makes into yours the moment something happens there), or a file watcher fired.

A proactive agent is a distributed system with an LLM in it. It needs everything an ordinary scheduled job needs:

• idempotency keys, so that doing the same unit of work twice has the effect of doing it once;
• dead-letter handling: somewhere to park messages that keep failing, instead of retrying forever;
• at-least-once delivery guarantees: the queue promises a message gets through, and tolerates delivering it twice to keep that promise;
• logging and alerting loud enough that a failure nobody was watching still gets noticed.

And then non-determinism on top: the same input does not reliably produce the same output.

The specific hazard: at-least-once delivery meets a non-idempotent agent

A nightly job wakes at 02:00, pulls unanswered support tickets off a queue, and for each one runs an agent that reads the ticket, searches the docs, drafts a reply, and sends it. Nothing about that agent’s loop differs from the one above. What differs is the trigger: a failure at 02:04 is discovered at 09:00 by a customer, with no user sitting there to notice the agent answered the wrong ticket.

The collision, step by step:

1. A worker takes ticket #8412, drafts a reply, and sends it.
2. The worker dies before acknowledging the message.
3. The queue’s visibility timeout, the window during which a handed-out message is hidden from other workers, expires with no acknowledgement.
4. The ticket is redelivered. A second worker runs the same agent.
5. The customer gets two emails.

The fix is the same as in any distributed system: an idempotency key, a stable identifier for one unit of work, checked against a ledger before the irreversible act, and written to that ledger before the act, not after.

Where the LLM changes the usual answer is in deciding what the key is derived from. Derive it from the input, never from the output. The same ticket run twice produces two different drafts, so a key computed from the generated email is a different key on every retry, and the ledger never recognizes a repeat. Compute it from the ticket id and the ticket fields in one fixed normalized form, deliberately excluding anything that varies per delivery (received_at, attempt, queue_message_id). And compute it in the harness, not by asking the model, a model-generated key is just a token sequence with no stability guarantee across runs, which defeats the purpose.

The form-filling agent case study implements exactly this, ledger states included.

Harness engineering

The harness is the code around the model, and most agent quality lives in that code, not in the prompt. Every one of its responsibilities exists because no prompt instruction can substitute for it.

The diagram expands one turn of the loop into its parts. Everything inside the box is code you write; the three dotted arrows are state that no single step owns.

flowchart LR
    subgraph H["Harness — you own this"]
        P[Prompt assembly] --> API
        API[Model call] --> R[Response parsing]
        R --> AUTH{Authorize}
        AUTH --> T[Tool dispatch]
        T --> C[Context management]
        C --> P
        B[(Budget<br/>ledger)] -.-> API
        L[(Trace log)] -.-> API
        G[(Loop guard)] -.-> AUTH
    end
    M((Model)) <--> API

    style H fill:#f8f9fa,stroke:#333,color:#111827
    style M fill:#1d3557,color:#fff
    style AUTH fill:#2d6a4f,color:#fff

Follow the solid cycle once: prompt assembly builds the request in tools → system → messages order; the model call goes out; response parsing turns the returned block list into typed structures and reads stop_reason; authorize decides whether each requested call is permitted; tool dispatch runs the permitted ones; context management decides what survives into the next assembly. Then control returns to prompt assembly.

The three dotted arrows are not decoration. The budget ledger and the trace log attach to the model call, because that is the only place tokens are spent and the only place there is anything to record. The loop guard attaches to Authorize, not to the model call: a loop is detected from the (tool, args) a model asked for, so you catch it where a repeated request can still be refused, before the tool runs, not after you paid for the turn that requested it.

Authorize is the one hard boundary, not an optimization. Everything else in the loop makes the agent cheaper, faster, or more debuggable; authorization is what makes a wrong decision not matter.

A better model with a bad harness loses to a worse model with a good one. Concretely, a harness that lets tool output flood the context will hit quality collapse around turn 30 regardless of which model you use. The failure is positional, models retrieve facts reliably near the start and end of their input and unreliably from the middle, not a matter of intelligence. The LLM internals chapter derives why.

So when an agent is worse in production than in the demo, the first move is not prompt tuning. Pull the traces and check the mechanical things: cache_read_input_tokens (a deploy may have changed one byte of the cached prefix and dropped it to zero), the stop_reason distribution (a rise in max_tokens means silent truncation), tool error rates, and whether the search index the agent retrieves from was rebuilt. Prompt tuning is what you do after the diff tells you where it diverged.

When not to build an agent

Whether a system deserves an agent at all comes down to four checks. Run all four; one “no” means drop a tier.

Running the checks on one system

Take a real pipeline: “Invoice arrives → extract fields → validate totals → route to an approver.”

Complexity: no. Every invoice takes the same four steps; the variation is in layout, not procedure. The extract stage handles layout with templates: one small per-vendor rule set saying where each field sits. It is fully specifiable, so drop a tier.

Value: no. The workflow is four single calls (4×). A roughly 10-turn agent doing the same job is ~19×, so nearly five times the spend for the same output. Drop a tier again.

Viability: yes. Extracting fields from documents is something models are reliably good at today.

Cost of error: yes. A wrong total is caught by the validation stage or the human approver, and nothing has been paid out.

Two “no”s, and the first is load-bearing: the answer is a workflow.

Notice which check failed, because it names the escape hatch. The failure was complexity, and complexity is a property of the input, not of the task type. So the ~5% of invoices whose layout the templates cannot parse are exactly the population that fails check 1 on their own, and exactly the population an agent should get. The check that fails tells you where the agent belongs.

The disguised-workflow trap

A common framing describes a fixed four-step pipeline and asks for “an agent” for it. Steps one through four are the same for every input, so that is a DAG. The right build is a workflow with one LLM call per stage (cheaper, individually testable, and each stage’s failure is attributable) plus an agent escape hatch for the ~5% of inputs the pipeline cannot classify, where reasoning over an unusual case is genuinely needed. The document-processing agent case study works this through end to end and shows the pipeline coming out ~27× cheaper on model spend and more accurate, because a deterministic path has no opportunity to be creative.

Conclusion

• The test for an agent is one substitution. A workflow computes the next step in your code; an agent computes it with a forward pass over the whole history and tools. If you can draw the complete set of paths, and your code decides when the run ends, it is a workflow, even when it calls tools or a model picks a branch.
• Everything is one loop. Call the model, branch on stop_reason, run tools, append results as one message, guard, repeat. The bugs live at the exits: max_tokens, refusal, and pause_turn all arrive as HTTP 200 and are easy to mistake for the finished answer. Read stop_reason before you index content.
• Cost is quadratic and dominated by the resent prefix. A 20-turn agent runs ~58× a single call. The first lever is making the prefix cacheable, not making the agent shorter.
• Quality lives in the harness, not the prompt. Authorization, budgets, retries, loop detection, and tracing are mechanical jobs a prompt cannot do. A good harness on a weaker model beats a bad harness on a stronger one.
• Default to the lower tier. Most tasks that get called agents are workflows with a small agent escape hatch for the inputs a fixed path cannot handle.

Cheat sheet

Each row is a symptom you will observe, the mechanism that produces it, and the fix. Offloading means keeping bulky content out of the conversation and passing the model a pointer to it; compaction means replacing a run of old turns with a short summary.

Further reading

• Anthropic, Building effective agents, the workflow-versus-agent distinction and the common patterns, from the source.
• Anthropic, Tool use with the Messages API, the exact request and response shapes, tool_use/tool_result blocks, and stop_reason values used throughout this chapter.
• Anthropic, Prompt caching, the cache_control marker, minimum cacheable lengths, and the usage fields to verify a cache hit.
• LangGraph documentation, Persistence and checkpointers, durable state, resumption, and the MemorySaver-versus-durable-backend distinction.

Next: 02 — Design Patterns, the nine architectures, each with its cost derived, not asserted.