Reflect and Retry on Tool Failure

A single tool call failing is not the end of a run. A well-built agent treats the error the way a human would: read what went wrong, then try again with better arguments. The mechanism is deceptively simple — you take the exception, turn it into a tool result message, and hand the whole transcript back to the model. On its next turn the model sees the error and can correct itself. This is the “reflect and retry” loop, and it is what separates an agent that recovers from a bad JSON argument or a divide-by-zero from one that crashes on the first hiccup.

The catch is knowing when to stop. If a tool is genuinely broken (or the model keeps making the same mistake), reflecting forever just burns tokens. So each tool gets a failure budget: reflect up to max_attempts_per_tool times, then give up.

How reflection works

The model does not retry on its own — your loop makes retry possible by writing the error into the conversation:

assistant -> {"type": "tool_call", "tool": "divide", "args": {"a": 10, "b": 0}}
tool      -> {"status": "error", "content": "ZeroDivisionError: division by zero"}
assistant -> {"type": "tool_call", "tool": "divide", "args": {"a": 10, "b": 2}}   # reflected!
tool      -> {"status": "ok", "content": "5.0"}
assistant -> {"type": "final", "content": "The answer is 5.0"}

Because the failed attempt and its error are now part of messages, the model’s second turn has everything it needs to fix the call. Your job is the plumbing: call the model, run the tool, catch the exception, record it as a tool result, and stop once a tool has burned its budget.

Task

Complete reflect_and_retry(model, tools, messages, max_attempts_per_tool):

1. Loop: call model(messages), then append the returned action as {"role": "assistant", "action": action}.
2. If action["type"] == "final", return action["content"].
3. Otherwise it is a tool_call. Look up tools[name] and call it with action["args"].
   • On success, append {"role": "tool", "tool": name, "status": "ok", "content": result} and continue.
   • On failure (the tool raises, or name is not in tools), increment that tool’s failure count and append {"role": "tool", "tool": name, "status": "error", "content": f"{type(exc).__name__}: {exc}"}.
4. After appending the error, if that tool’s failure count has reached max_attempts_per_tool, raise RuntimeError. Otherwise continue so the model can reflect.

Example

def divide(args):
    return str(args["a"] / args["b"])

tools = {"divide": divide}

script = iter([
    {"type": "tool_call", "tool": "divide", "args": {"a": 10, "b": 0}},  # fails
    {"type": "tool_call", "tool": "divide", "args": {"a": 10, "b": 2}},  # reflected fix
    {"type": "final", "content": "The answer is 5.0"},
])
def model(messages):
    return next(script)

messages = [{"role": "user", "content": "What is 10 divided by 2?"}]
out = reflect_and_retry(model, tools, messages, max_attempts_per_tool=3)
# out == "The answer is 5.0"

# The error was recorded so the model could reflect:
errors = [m for m in messages if m.get("status") == "error"]
# len(errors) == 1
# errors[0]["content"] == "ZeroDivisionError: division by zero"

Constraints

• The model is the passed-in model stub only — never call a real API, network, or time.sleep.
• messages is mutated in place; both assistant actions and tool results are appended in the order they occur.
• Catch tool failures with except Exception (not BaseException), so a KeyboardInterrupt still aborts the loop.
• An unknown tool name is a failure too: it counts against that name’s budget and is recorded as an error result.
• The failure budget is per distinct tool name, counted cumulatively across the whole run — not consecutive, and not shared between tools.
• Always append the error result before raising, so the final transcript explains why the loop stopped.

Approach

Run an ordinary agent loop, but wrap the tool call in a try/except. On success, append an ok tool result and continue. On failure, convert the exception into an error tool result and append it to messages — that written-down error is the entire “reflection” mechanism, because the model’s next turn reads it and can correct its arguments. A per-tool-name counter enforces the failure budget: once a tool reaches max_attempts_per_tool failures we record the final error and raise RuntimeError instead of looping forever.

Solution

from typing import Any, Callable

def reflect_and_retry(
    model: Callable[[list[dict]], dict],
    tools: dict[str, Callable[[dict], str]],
    messages: list[dict],
    max_attempts_per_tool: int,
) -> str:
    failures: dict[str, int] = {}

    while True:
        action = model(messages)
        messages.append({"role": "assistant", "action": action})

        if action["type"] == "final":
            return action["content"]

        name = action["tool"]
        args = action.get("args", {})
        tool = tools.get(name)

        try:
            if tool is None:
                raise KeyError(f"unknown tool: {name}")
            result = tool(args)
        except Exception as exc:
            failures[name] = failures.get(name, 0) + 1
            messages.append({
                "role": "tool",
                "tool": name,
                "status": "error",
                "content": f"{type(exc).__name__}: {exc}",
            })
            if failures[name] >= max_attempts_per_tool:
                raise RuntimeError(
                    f"tool '{name}' failed {failures[name]} times; giving up"
                )
            continue  # let the model reflect on the error next turn

        messages.append({
            "role": "tool",
            "tool": name,
            "status": "ok",
            "content": result,
        })

Walkthrough

Tracing the example with max_attempts_per_tool=3:

1. model returns divide(a=10, b=0). We append it as an assistant action, then call divide — 10 / 0 raises ZeroDivisionError. We record failures["divide"] = 1 and append {"status": "error", "content": "ZeroDivisionError: division by zero"}. Since 1 < 3, we continue.
2. model runs again. Because the error is now in messages, the model “reflects” and returns divide(a=10, b=2). The call succeeds with "5.0", appended as an ok tool result.
3. model returns {"type": "final", "content": "The answer is 5.0"}. We append it and return "The answer is 5.0".

The final messages contains exactly one error entry, so errors[0]["content"] == "ZeroDivisionError: division by zero" — matching the expected output. Had divide kept failing, the third failure would push failures["divide"] to 3, and after appending that error we would raise RuntimeError.

Complexity & notes

• Time is O(T) in the number of turns T the model takes (each turn is O(1) bookkeeping plus the tool’s own cost); space is O(T) for the growing transcript plus O(number of distinct tools) for the counter.
• The order “increment, append error, then check budget” is deliberate: the transcript always ends with the error that ended the run, which is exactly what you want when debugging why an agent gave up.
• Budgeting per tool name (not one global counter) means a run touching several tools tolerates a stumble from each; a single flaky tool cannot exhaust the whole run’s patience, and one tool’s recovery does not reset another’s count.
• except Exception catches tool bugs and bad arguments but not BaseException, so KeyboardInterrupt/SystemExit still break the loop instead of being reflected on.
• Treating an unknown tool as a normal failure (raise KeyError, record it, count it) lets the model recover from a hallucinated tool name the same way it recovers from bad arguments, rather than crashing the loop with an uncaught lookup.
• In production you would layer this with the backoff retry (transient network errors) and a global step budget, and you might strip stack traces from the error content to keep prompts short — but the reflect-by-appending core stays identical.