Harness and Loop Engineering

The leverage in an AI system keeps moving. For a while it lived in the wording of a single prompt, then in what you packed into the context window, and now it lives in the code that runs the model in a loop. This lesson traces that shift, explains what a harness is and why most agent quality lives in it rather than in the model, and works through the most stripped-down example of loop engineering there is: the Ralph technique, a while loop that feeds an agent the same prompt until the job is done.

Three eras of leverage

The same underlying model can be mediocre or excellent depending on what you wrap around it, and over the last few years the thing worth engineering has moved outward in three steps: from the prompt, to the context, to the harness.

flowchart LR
    P["Prompt engineering<br/>~2020-2023<br/>shape one input string"]
    C["Context engineering<br/>~2023-2025<br/>shape the whole window"]
    H["Harness engineering<br/>2024-now<br/>shape the code + the loop"]
    P --> C --> H
    style P fill:#e9ecef,stroke:#333,color:#111827
    style C fill:#cfe1d0,stroke:#333,color:#111827
    style H fill:#1d3557,color:#fff

Each era did not replace the last; it wrapped it. A harness still assembles a context, and that context still contains prompts. What changed is where the marginal improvement comes from, and the table below names the unit you were tuning in each.

The prompt era

In the earliest era the model was treated as a fixed oracle and the only dial you had was the string you handed it. The craft was prompt engineering: getting a single input worded so the one completion that came back was the one you wanted. This is where the durable tricks were discovered, and they still work inside every later era. Giving a few worked examples (few-shot prompting) teaches a format from demonstration rather than description. Asking the model to reason before answering (chain-of-thought, popularized by the almost embarrassingly simple instruction “let’s think step by step”) spends output tokens to raise accuracy on anything with intermediate steps. Framing a role, pinning the output format, and stating constraints all shape that single response.

What the prompt era could not do was persist or act. A prompt has no memory of the last one, cannot call a tool, and cannot recover from its own mistake. Push it toward a task with more than a couple of steps and it degrades into a wall of increasingly desperate instructions, each patching a failure the last one caused. The ceiling was structural, not a matter of finding better words.

The context era

The next move was to stop obsessing over the sentence and start engineering everything the model could see at once. A model’s output is a function of its entire context window, the full sequence of tokens it reads before generating, so the real unit of control is that window, not the instruction buried inside it. This is context engineering: deciding what goes into the window, in what order, and what gets evicted when it fills.

The window is where retrieval, memory, and tool results all land, which is why so much of the agent stack is really context plumbing. Retrieval-augmented generation (RAG for agents) fetches the few passages worth spending tokens on instead of stuffing everything in. Memory (memory and context) decides what survives from earlier turns. Ordering matters for a mechanical reason covered in LLM internals: a stable prefix can be served from the KV cache instead of recomputed, so where you put the changing part of the context sets your cost.

Context engineering also inherited a hard limit that prompt engineering never had to face. A longer window is not a free upgrade, because quality decays with position and length, the effect derived in why quality degrades in long contexts. Attention dilutes as the field of competitors grows, and a fact buried in the middle of a huge window is recalled far worse than the same fact at the start or end. The practical consequence is that “put more in the window” stops helping and starts hurting, and the discipline becomes keeping the window small and relevant rather than large. That constraint is exactly what the next era learned to exploit.

The harness era

Once teams were managing context well, the remaining gap turned out not to be in the model or the window at all. It was in the code that assembles the context, calls the model, reads the reply, runs the tools, and decides whether to go around again. That code is the harness, and harness engineering is the current frontier because the model has become the commodity: swap in a better one and a badly built loop still fails, while a well-built loop lifts a weaker model past a stronger one running bare.

That last claim is worth stating as a rule, because it inverts the intuition most people start with. A worse model with a better harness beats a better model with a worse harness, on almost any real task, because the harness is what turns a single fallible guess into a checked, retried, bounded process. The model supplies judgment for one step; the harness supplies everything that makes a sequence of steps reliable.

What the harness actually is

The harness is the loop around the model, and agent foundations builds it in full. The short version is that every turn runs the same cycle of code, and each responsibility in it exists precisely because no prompt instruction can stand in for it.

flowchart LR
    subgraph HARNESS["Harness — code you own"]
        A[Assemble context] --> CALL[Call model]
        CALL --> PARSE[Parse response]
        PARSE --> AUTH{Authorize}
        AUTH --> DISP[Dispatch tools]
        DISP --> MANAGE[Manage context]
        MANAGE --> A
        BUD[(Budget)] -.-> CALL
        GUARD[(Loop guard)] -.-> AUTH
    end
    MODEL((Model)) <--> CALL
    style HARNESS fill:#f8f9fa,stroke:#333,color:#111827
    style MODEL fill:#1d3557,color:#fff
    style AUTH fill:#2d6a4f,color:#fff

Following the cycle once shows why these are code, not prose. Context assembly sets the cache hit rate, a mechanical property no instruction can change. Authorization is the one hard boundary, because a prompt rule is advisory and one percent non-compliance on a destructive action is not acceptable. Budget accounting has to live outside the model because the model cannot see its own token spend, and loop detection needs state across turns that the model does not reliably track. The harness is the place where “usually right” becomes “safe to run unattended.”

Loop engineering: the program is the loop

If the harness is the body of the loop, loop engineering is the design of the loop itself: how many times it runs, what carries state between iterations, and what makes it stop. It helps to see that there is not one loop but three nested inside each other, each a level coarser than the last.

flowchart TD
    subgraph TASK["Task loop — keep going until the goal is met"]
        subgraph TURN["Turn loop — one model call + its tools"]
            subgraph DECODE["Decode loop — one token at a time"]
                D[token to token]
            end
        end
    end
    style TASK fill:#f8f9fa,stroke:#333,color:#111827
    style TURN fill:#e9ecef,stroke:#333,color:#111827
    style DECODE fill:#cfe1d0,stroke:#333,color:#111827

The innermost loop is the model generating one token at a time, the autoregressive decode of the forward pass; you do not engineer it, you pay for it. The middle loop is one turn: assemble a context, call the model, run the tools it asked for. The outer loop is the one that matters most and gets the least attention: the task loop, which decides whether the whole thing is done or should run another turn. Most of what separates a toy from a working agent is in that outer loop, and the cleanest way to see it is to strip it down to almost nothing.

Ralph: the minimal loop

The most instructive piece of loop engineering is also the crudest. The Ralph technique, named by Geoffrey Huntley after Ralph Wiggum’s cheerful “I’m learning,” is a coding agent run in an infinite shell loop that feeds it the same prompt every time, letting it make one small increment of progress per iteration until the work is complete.

# Ralph: the same prompt, on repeat, until the work is done.
while :; do
  cat PROMPT.md | your-agent --yes --model sonnet
  git add -A && git commit -m "ralph iteration" || true
  grep -q "ALL TASKS COMPLETE" PROGRESS.md && break
done

That is the entire program. There is no orchestration framework, no growing conversation, no clever state machine. A fixed prompt goes in, the agent does a little work and records it, the loop commits and runs again. What makes this more than a curiosity is that it directly exploits the constraint the context era ran into, and understanding why it works is worth more than the four lines themselves.

Why a dumb loop works

Ralph inverts the usual relationship between the window and the state. Instead of accumulating a longer and longer conversation, every iteration starts from a fresh context, so the model never pays the long-context tax from why quality degrades in long contexts: each run is short, its attention is sharp, and there is no middle of a huge window for a fact to get lost in. The durable state that a normal agent keeps in its conversation instead lives in the environment, the repository on disk plus its git history, which is unbounded, inspectable, and survives a crash. The prompt is fixed because the goal is fixed; what changes between runs is the world the agent reads, not the instruction it is given.

flowchart LR
    START([loop iteration]) --> READ[Fresh context reads<br/>repo + PROGRESS.md]
    READ --> WORK[Do ONE increment]
    WORK --> TEST[Run tests / verify]
    TEST --> RECORD[Update PROGRESS.md<br/>+ git commit]
    RECORD --> DONE{All tasks<br/>complete?}
    DONE -- no --> START
    DONE -- yes --> STOP([stop])
    style READ fill:#cfe1d0,stroke:#333,color:#111827
    style DONE fill:#2d6a4f,color:#fff

Three properties do the work. Fresh context per iteration keeps every run in the high-quality regime rather than letting quality rot as a session drags on. The filesystem and git act as external memory, so progress is durable and each iteration can see exactly what the last one changed. And because the prompt is identical every time, the loop is trivially resumable: kill it, restart it, and it picks up from whatever state the files are in, since the files, not the chat, are the source of truth.

The prompt and the progress file

Ralph lives or dies on two files. The first is the fixed prompt, which has to encode not just the goal but the discipline of doing one thing, verifying it, and recording it, because the loop supplies no other control.

You are working on <project>. The repository is the source of truth, not this message.

Each run, do exactly this and nothing more:
1. Read PROGRESS.md and the code to find the FIRST unchecked task.
2. Implement ONLY that one task. Keep the change small and self-contained.
3. Run the tests. If any fail, fix them before you finish.
4. Check that task off in PROGRESS.md and append one line describing what you did.
5. If every task is checked, write "ALL TASKS COMPLETE" at the top of PROGRESS.md and stop.

Never start a second task in one run. Never rewrite work that is already checked off.

The second is the progress ledger, which is the memory the fixed prompt reads and writes each time. It carries the plan, the state, and a running log, so a brand-new context can reconstruct exactly where things stand in a single read.

# Build a URL shortener

- [x] Project scaffold and test harness
- [x] In-memory store with get/put
- [ ] Base62 id generator     <- next
- [ ] HTTP handlers
- [ ] Persistence layer

## Log
- scaffold + pytest wired, 2 tests green
- store: get/put with TTL, 5 tests green

Between them, the prompt says how to make one move and record it and the ledger says what has already been done. The agent’s own memory is deliberately thrown away every iteration, because the files remember better than a decaying window can.

The economics

Ralph trades one expensive, fragile call for many cheap, disposable ones, and the arithmetic is why it is viable rather than merely cute. A single agent asked to build a whole project in one session drifts into a huge context where quality falls and one late mistake can poison everything after it. Ralph instead spends a small, bounded context per iteration and pays for a few thousand tokens many times over.

The catch is that “many cheap iterations” is only cheap if the number of iterations stays bounded, which is exactly where an unguarded Ralph goes wrong.

Where Ralph breaks, and the guardrails

A loop that never questions itself will happily run forever, and the failure modes are specific enough to guard against directly. The first is the missing stop condition: without a check like the ALL TASKS COMPLETE sentinel, and a hard cap on iterations, the loop keeps paying for turns after the work is finished. The second is drift, where the agent starts undoing or re-doing finished tasks; the fix is the prompt discipline above, never rewriting checked-off work, plus committing every iteration so any regression is a one-line git revert. The third is no-progress oscillation, where successive iterations churn without advancing; detect it by comparing the git diff or the progress file across runs and breaking the loop when nothing has changed for a few iterations. The fourth is the most dangerous: running an agent unattended with permissions to touch anything means a single bad tool call can do real damage, so Ralph belongs in a sandbox, a container or a throwaway branch, never pointed at production with the safety off.

# A slightly less reckless Ralph: bounded, sandboxed, self-halting.
i=0
while [ $i -lt 50 ]; do
  before=$(git rev-parse HEAD)
  cat PROMPT.md | your-agent --yes
  git add -A && git commit -m "ralph $i" || true
  grep -q "ALL TASKS COMPLETE" PROGRESS.md && break
  [ "$(git rev-parse HEAD)" = "$before" ] && { echo "no progress; stopping"; break; }
  i=$((i+1))
done

Those guards are themselves loop engineering: an iteration cap is a budget, the no-progress check is a loop guard, and the sandbox is authorization. Ralph is minimal, not lawless, and the same responsibilities the harness owns reappear here in shell form.

Ralph versus a structured harness

Ralph is one point on a spectrum, and knowing when to reach for it matters more than the trick itself. Its whole advantage is that state lives in the filesystem, so it shines when the task is decomposable into small independent increments whose result is checkable from the repo, like grinding through a long list of files, tests, or migrations. It is a poor fit when steps are tightly coupled and need a decision carried in working memory rather than written to disk, when each step is expensive enough that you cannot afford to repeat one, or when a wrong action cannot simply be reverted, since a real side effect like a payment or an email leaves the filesystem an incomplete picture of the world.

The deeper point is that both are loop engineering. A production harness is a more careful loop with typed state, per-step authorization, and budgets; Ralph is the same idea compressed to a shell while and a git repo. Once you see the loop as the program, the design question is never “which framework” but “what carries state between iterations, and what makes this stop.”

In an interview

If you are asked to design or critique an agent, the strongest framing is to locate where the leverage is. Say plainly that the model is the commodity and the harness is the product: a worse model with a better loop beats a better model with a worse loop. Show that you know the three responsibilities that cannot live in the prompt, authorization, budget, and cross-turn state, and that you know the outer task loop is where reliability is won or lost. If the task is a big, decomposable grind, propose a Ralph-style loop and immediately name its guardrails, an iteration cap, a stop sentinel, per-iteration commits, and a sandbox, because proposing the loop without the guards is the tell of someone who has only read about it. The concise version to have ready: prompt engineering shaped one string, context engineering shaped the window, and harness and loop engineering shape the code that runs the model many times over, which is where the real systems are built today.

Further reading

• Geoffrey Huntley, Ralph Wiggum as a software engineer (ghuntley.com/ralph): the origin of the Ralph loop and the case for brute-force agentic iteration.
• Jason Wei et al., Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (2022) (arxiv.org/abs/2201.11903): the prompt-era result that reasoning steps can be elicited by instruction.
• Agent foundations and memory and context: the harness loop and the context management this lesson builds on.
• Why quality degrades in long contexts: the decay that makes fresh-context loops like Ralph work.