The leverage in an AI system moved outward from the prompt to the context to the code that runs the model in a loop; the model is now the commodity and the harness is the product.
Three eras of leverage
| Era | Unit you engineer | Typical moves | Wall it hit |
|---|
| Prompt | one input string | few-shot, chain-of-thought, role, output format | brittle, no memory, dies on multi-step work |
| Context | the whole window | retrieval, memory, tool results, cache-ordering, compaction | window is finite, quality decays inside it |
| Harness | the code + loop around the model | tool dispatch, authorization, retries, budget, iteration | the model itself is now the cheap part |
- Each era wraps the last: a harness still assembles a context that still contains prompts.
- Chain-of-thought: “let’s think step by step” spends output tokens to raise accuracy.
- Context window: full token sequence read before generating; output is a function of it, not the buried instruction.
- Long window is not a free upgrade: quality decays with position and length, so keep it small and relevant.
The core rule
- Harness: the code that assembles context, calls the model, parses the reply, runs tools, and decides whether to loop again.
- A worse model with a better harness beats a better model with a worse harness on almost any real task.
- Model supplies judgment for one step; the harness makes a sequence of steps reliable.
What can’t live in the prompt
- Authorization: the one hard boundary; a prompt rule is advisory, and 1% non-compliance on a destructive action is unacceptable.
- Budget: the model can’t see its own token spend, so accounting lives outside it.
- Cross-turn state / loop detection: needs state the model doesn’t reliably track.
- Context assembly sets the cache hit rate, a mechanical property no instruction can change.
Three nested loops
- Decode loop: one token at a time (autoregressive); you don’t engineer it, you pay for it.
- Turn loop: one model call plus its tools.
- Task loop: outer loop deciding done vs. run again; where reliability is won or lost.
Ralph: the minimal loop
- Ralph technique (Geoffrey Huntley): a coding agent in an infinite shell loop fed the same prompt every iteration, making one small increment until done.
while :; do
cat PROMPT.md | your-agent --yes
git add -A && git commit -m "ralph iteration" || true
grep -q "ALL TASKS COMPLETE" PROGRESS.md && break
done
Why a dumb loop works, three properties:
- Fresh context per iteration: every run is short, avoids the long-context tax, stays in the high-quality regime.
- Environment as memory: durable state lives in the repo + git history (unbounded, inspectable, survives a crash), not the conversation.
- Resumable: prompt is fixed because the goal is fixed; kill and restart, it picks up from the files (files are the source of truth).
- Two files carry it: a fixed PROMPT.md (do one task, verify, record it, never start a second or rewrite checked-off work) and a PROGRESS.md ledger (plan + state + log, plus the
ALL TASKS COMPLETE sentinel).
One session vs. Ralph
| One long session | Ralph loop |
|---|
| Context per step | grows without bound | small and fixed |
| Quality over time | decays as window fills | flat, every run fresh |
| A bad step | can corrupt the rest | one commit you can revert |
| Memory | conversation (volatile) | repo + git (durable) |
| Recovery | restart from scratch | resume from files |
Gotchas and guardrails
- Missing stop condition: add the
ALL TASKS COMPLETE sentinel plus a hard iteration cap (e.g. 50).
- Drift (undoing/re-doing finished tasks): prompt discipline never rewrites checked-off work; commit every iteration so a regression is a one-line
git revert.
- No-progress oscillation: compare the git HEAD / diff across runs, break when nothing changed.
- Unattended damage: run in a sandbox (container or throwaway branch), never at production with the safety off.
- The guards map to harness responsibilities: iteration cap = budget, no-progress check = loop guard, sandbox = authorization.
Ralph vs. structured harness
| Reach for Ralph when… | Reach for a structured harness when… |
|---|
| work splits into many small independent increments | steps are tightly coupled and order-sensitive |
| progress is fully readable from the repo | important state doesn’t live on disk |
| a bad step is cheap to revert | actions have irreversible external effects |
| you want brute-force throughput overnight | you need approvals and audit per step |
In an interview
- State plainly: the model is the commodity, the harness is the product.
- Name the three things that can’t live in the prompt: authorization, budget, cross-turn state.
- The outer task loop is where reliability is won or lost.
- For a big decomposable grind, propose Ralph and immediately name its guardrails (cap, sentinel, per-iteration commits, sandbox); proposing the loop without guards is the tell of someone who only read about it.