Form-Filling Agent

Entering 4,000 records into an API-less web form is not a scripting chore; it is an exactly-once distributed-systems problem, because you cannot make a database write and a third-party HTTP POST atomic.

The core impossibility

• No two-phase commit with a partner portal, so there is always a window where you have submitted and do not yet know if it landed.
• Submission is irreversible: no undo, no API, only a human-facing form.
• Success condition is exactly one submission per record, ever, across any crash or restart, not “4,000 submissions happened.”
• Design split: the model maps a record to fields and repairs a rejected value; ordinary harness code reads the page, fills, submits, records state. In steady state the model is out of the loop. This is a prompt chain with a cached first stage, not an agent.

Three ways to see a form

The most consequential choice. Use the accessibility tree (a11y): its identifiers are semantic (role + accessible name), so they survive a restyle and let the mapping cache hit.

• Extract with Playwright page.accessibility.snapshot(interesting_only=True), then filter to fields (textbox, combobox, checkbox, radio, listbox).

The ledger: three-state machine

The ledger is one Postgres table, keyed by idempotency key, one row per record. A boolean cannot say “we sent it and never learned the outcome,” which is what most failures produce.

• States: NONE -> PENDING -> DONE; PENDING -> UNKNOWN on timeout/ambiguous.
• PENDING and UNKNOWN never auto-advance and never auto-retry — only a human (or a portal query) resolves them.
• A set of keys fails: can’t hold “unknown,” the payload sent, the confirmation ref, or mapping_version for blast-radius queries.
• If the portal accepts its own idempotency key, use it — duplicates become impossible.

Exactly-once: ordering + one constraint

Write PENDING (with payload) before submit, write DONE after confirmation. The guarantee rests on the schema, not the code.

PENDING before submit → crash = escalate (~1 in 4,000)
DONE after only, no PENDING → duplicate (detectable)
DONE before submit → SILENT DROP (undetectable, worst)

• The claim is one atomic INSERT ... ON CONFLICT DO NOTHING against PRIMARY KEY (idempotency_key). The read before it is only an optimization.
• Read-then-write is correct at 1 worker, wrong at 10: all workers read none in the same 5 ms window and all submit. Measured: 20 trials, 20 duplicates every time.
• Silent drop is worst: a duplicate is caught by accounts payable; a DONE-but-never-entered record is found months later by the unpaid supplier.

Idempotency key rules

• Stable across runs, processes, machines, Python versions. Hash is SHA-256 over canonical JSON with sort_keys.
• Exclude: timestamps, run_id, row_number, attempt counters, ingestion metadata — any of these mints a fresh key per retry.
• Canonicalize every value first: NFC normalize, strip whitespace, coerce numbers to strings. "ü" as NFC vs NFD is two byte strings, one supplier.
• Key on supplier_id alone when the source has a real identifier; it survives a corrected legal_name. Fallback hashes all non-metadata fields but then a corrected name mints a fresh key.

Validation repair loop

• The form’s own error text is free, precise supervision authored by the party that judges you. Feed it back verbatim; don’t paraphrase.
• Loop: snapshot → fill → re-snapshot → repair, capped at 3 rounds, then human queue.
• Re-snapshot after every fill: conditional fields (e.g. VAT ID appears after country = DE) exist only after a prior answer.
• assert_all_required_filled runs in the harness (a raise), not in the prompt (a request), on re-snapshotted browser values.

Prompt injection: control vs mitigation

Page text (label, hint, errors) is authored by a third party. A mitigation lowers odds; a control removes the capability.

• Mitigation 1: quote page text as data in a tag, length-cap it.
• Mitigation 2: regex-flag instruction-shaped text — for logging/alerting, not blocking.
• The control (structural): the model emits only a permutation of existing nodes; values come from the record; submit is not a tool the model can reach. An injected “submit now” has nothing to call.
• Residual risk: a wrong-but-valid mapping passes every check; caught only by the 2% audit.

Mapping cache and economics

• Key the cache on the a11y signature sorted((role, label, required)), not DOM node ids (React useId/CDP ids are per-render → 0% hit rate looking healthy).
• Cache the template {source_field: label} (durable); resolve label -> node_id fresh each load.
• Misses = number of distinct layouts (12 here), not records → ~99.7% hit rate.
• Signature is blind to changed options/disabled (a deliberate trade); a rename is a correct miss.
• Cache-miss rate doubles as a free deploy detector for a system you don’t control — spike = portal redesign.

• Whole job: 12 map + 40 repair + 80 Haiku audit = 132 calls / 4,000 records = **$1.45**. A ReAct-loop-per-record is ~$1,850 (~1,275× more).

Latency: the browser is the bottleneck

• ~99% of wall-clock is Chrome; model call amortized to ~25 ms (0.3%). Per record ~7.4 s; job ~50 min on 10 workers.
• Big steps: submit + confirmation (42%), navigate (24%), hydration (16%), fill 22 fields (9%).
• Optimize browser reuse/navigation (hours saved), not the prompt (seconds). Reusing a browser context saves ~53 min across the job.
• networkidle is a trap (analytics/websockets never idle → 30 s timeout per record); use waitForSelector.
• 10 workers ≈ 1.4 req/s against someone’s production system — get permission, back off on 429, never solve CAPTCHAs.

Key gotchas

• Wrong field mapping: validation passes, data is wrong, every structural guard fires green. Only the 2% continuous human audit + stored payload catch it; recovery is WHERE mapping_version = 7.
• Chaos test crash point G (after DONE write) is always omitted but is the only one whose correct restart is skip — it alone tests the DONE short-circuit.
• Non-breaking space U+00A0 in a live label silently breaks label matching — raise, don’t drop the field.
• Test every guard against the adversarial case (trailing space, 10 workers, NBSP), not the happy path. Automation must be permitted by TOS and robots.txt first.