Parsing a Tool Call from Model Output

Language models are told to respond with a JSON tool call, but they do not always cooperate. They add a sentence before it, wrap it in a Markdown ```json fence, or emit malformed text. Before your agent loop can dispatch a tool, it has to reliably pull the tool call out of that raw string. This parsing-and-validation step is where a surprising number of agents break in production.

Task

Complete parse_tool_call(text: str) -> dict:

1. Locate the first JSON object in text. It may be surrounded by prose, or fenced in a json ... (or plain ...) block. Handle both.
2. Parse that object. It must contain a string "name" and a dict "arguments" (treat a missing "arguments" as an empty dict {}).
3. On success return {"ok": True, "name": <str>, "arguments": <dict>}.
4. On any failure — no JSON object present, invalid JSON, missing/non-string name, or non-dict arguments — return {"ok": False, "error": <clear message>}. Do not raise.

To find the first object, scan for a { and track brace depth (accounting for braces inside string literals) until it balances; then hand that substring to json.loads.

Example

raw = 'Sure! Here is the call:\n```json\n{"name": "search", "arguments": {"q": "cats"}}\n```\nHope that helps.'

parse_tool_call(raw)
# -> {"ok": True, "name": "search", "arguments": {"q": "cats"}}

parse_tool_call("I cannot help with that.")
# -> {"ok": False, "error": "no JSON object found"}

Constraints

• No third-party libraries; json from the standard library only.
• The parser must never raise on bad input — every failure path returns the error dict.
• A { that appears inside a JSON string value (for example {"q": "a { b"}) must not throw off brace counting.

Approach

Ignore any fences and prose and instead scan the raw string for the first balanced {...} block, tracking brace depth while respecting string literals and escapes so braces inside values do not miscount. Hand that substring to json.loads, then validate the shape: a string name and a dict arguments (defaulting to {}). Every failure path returns a structured error dict rather than raising, so the agent loop can inspect the result and decide whether to retry.

Solution

import json
from typing import Any


def _first_json_object(text: str) -> str | None:
    """Return the first balanced {...} substring, or None."""
    start = text.find("{")
    while start != -1:
        depth = 0
        in_str = False
        escape = False
        for i in range(start, len(text)):
            c = text[i]
            if in_str:
                if escape:
                    escape = False
                elif c == "\\":
                    escape = True
                elif c == '"':
                    in_str = False
            else:
                if c == '"':
                    in_str = True
                elif c == "{":
                    depth += 1
                elif c == "}":
                    depth -= 1
                    if depth == 0:
                        return text[start : i + 1]
        # unbalanced from this '{'; try the next one
        start = text.find("{", start + 1)
    return None


def parse_tool_call(text: str) -> dict[str, Any]:
    candidate = _first_json_object(text)
    if candidate is None:
        return {"ok": False, "error": "no JSON object found"}

    try:
        obj = json.loads(candidate)
    except json.JSONDecodeError as e:
        return {"ok": False, "error": f"invalid JSON: {e}"}

    if not isinstance(obj, dict):
        return {"ok": False, "error": "top-level JSON is not an object"}

    name = obj.get("name")
    if not isinstance(name, str):
        return {"ok": False, "error": "missing or non-string 'name'"}

    arguments = obj.get("arguments", {})
    if not isinstance(arguments, dict):
        return {"ok": False, "error": "'arguments' is not an object"}

    return {"ok": True, "name": name, "arguments": arguments}

Walkthrough

On the fenced example 'Sure! Here is the call:\n```json\n{"name": "search", "arguments": {"q": "cats"}}\n```\n...':

1. _first_json_object finds the first { at the start of {"name": ...}. It walks forward: the inner { for arguments pushes depth to 2, its } drops it to 1, and the outer } drops it to 0 — returning exactly {"name": "search", "arguments": {"q": "cats"}}. The surrounding fence and prose are simply never included.
2. json.loads parses it to a dict. name is the string "search" and arguments is a dict, so both checks pass.
3. We return {"ok": True, "name": "search", "arguments": {"q": "cats"}}.

For "I cannot help with that." there is no {, so _first_json_object returns None and we return {"ok": False, "error": "no JSON object found"}.

The string-aware scan is what makes {"q": "a { b"} safe: the { inside the quoted value is seen while in_str is true, so it never increments depth.

Complexity & notes

• Time is O(n) in the length of text for a well-formed input; the outer retry over unbalanced { positions is O(n²) worst case on pathological strings full of stray unclosed braces, which real model output does not produce. Space is O(k) for the extracted substring.
• Scanning for a balanced block beats a regex: JSON is not a regular language, so nested objects defeat any \{.*\} pattern, and a greedy \{.*\} would swallow trailing braces from later prose.
• Returning a structured {"ok": False, "error": ...} instead of raising lets the caller feed the error back to the model as a tool-result and ask it to re-emit valid JSON — the standard “reflect and retry” recovery for malformed tool calls.
• Production hardening to mention in an interview: accept a top-level JSON array of calls (parallel tools), tolerate trailing commas or single quotes with a lenient second-pass parser, and validate arguments against the tool’s schema before dispatch.