Model Debugging Playbook

A bug is an assumption you did not know you were making; debug in fixed order and stop at the first step whose number proves an assumption false. Every step outputs one sentence: named broken assumption, number that proves it, the fix, the check that would have failed if you were wrong.

The ladder (run in order)

Do not reach step 6 until the six cheaper explanations are ruled out. Most underperformance is a data, label, metric, or skew problem; the model is the residual.

0  Reproduce · seed everything · baseline   -> beats baseline?
1  Data bug or model bug?  (a overfit-8 · b label audit · c skew · d leakage)
2  Bias or variance?  (val error vs training-SET SIZE)
3  Error analysis by slice  (sort by error share)
4  Is the metric wrong?
5  Is the label wrong?  (clean TEST first)
6  Improve model: capacity -> features -> data

Step 0 — Reproduce, seed, baseline

• Seed everything, including the split seed, DataLoader workers, augmentation RNG, hash-bucket features.
• Standard error of a proportion: sqrt(p(1-p)/n). At n=2,000, p=0.90 → 0.67 pts. Unseeded split fabricates “improvements.”
• Noise floor: run same config 5 seeds. Example sd = 0.0059 for one run. You gate on a difference, whose sd is sd*sqrt(2).
• Believe nothing under the earned band:

• Gotcha: 2*sd looks like a 5% filter but always lets 15.7% of noise through (only sqrt(2)-sigma on the difference).
• Three baselines, cheapest first: majority class / global mean; simple heuristic already deployed; regularized linear or GBDT on raw features. Without a floor you cannot tell “good model” from “easy problem” from “degenerate metric” (0.940 accuracy loses to a constant at 0.942).

Step 1 — Data bug or model bug?

Assumption: the data the model sees is the data I think it sees.

• 1a Overfit 8 samples (shuffle/augmentation/dropout/weight-decay off). Must reach ~0 loss. Failing eliminates capacity, data volume, and generalization by construction → look for label misalignment, wrong loss axis, frozen params, missing zero_grad, lr=0. Loss pinned at ln(C) = uniform guessing (ln(10)=2.30).
• 1b Label audit: relabel 100 rows blind. Agreement 0.97+ = fine; 0.88 = 12% wrong, go to step 5. Use 100 not 10 (SE ≈ 3 pts).
• 1c Train/serve skew: recompute same feature, same entity, same timestamp through both paths and diff. Read the % rows differing column. One feature wildly off against a clean background = skew. Causes: different code (< vs <=), different data availability, different time semantics.
• 1d Leakage: number is implausible (0.987 where experts get 0.75).
  • Diagnostic 1: single-feature AUC scan; any raw feature above ~0.95 alone is a leak. Use max(auc, 1-auc) for direction.
  • Diagnostic 2: ablate the suspect, confirm metric lands in the plausible range (not merely drops). Ablate an ordinary feature as control.
  • Patterns: target-derived, lookahead in time, group leakage (split by entity), preprocessing fit before split.
  • Group-leakage arithmetic: 8,000 users × 5 rows, random row split → 67% of users straddle it, 99.8% of test rows have their user in train.

Step 2 — Bias or variance?

Assumption: more data would help. Plot val error vs training-set size (not vs epoch), 5–6 sizes, several seeds.

• Price the data: excess error over floor falls ×0.71 = 1/sqrt(2) per doubling = power law. Fit err(n) = e_inf + a/sqrt(n); invert n = (a/(e - e_inf))^2. Example: 3 more points of error costs 4×, 5 more costs 36× the data. e_inf = irreducible under this model → send you to features, not labeling.

Step 3 — Error analysis by slice

Assumption: the average describes every user. Overall metric is traffic-weighted, so a slice must be small to hide its failure — and small slices are the growth markets.

• Sort by error share (n*(1-acc)), not accuracy. Worked case: mobile·es = 4.0% traffic, 25.1% of all errors, accuracy 0.510 while overall reads 0.922. Fixing it moves overall 0.922 → 0.939.
• Find the cause by slicing the features, not just the metric (e.g. page_text non-null 22% vs 97%; imputer fills nulls with 0 → confident negative). Fix = missingness indicator + slice fallback, not architecture.
• Automate: fit a shallow tree on the error indicator; report worst-slice metric as a standing number.

Step 4 — Is the metric wrong?

Assumption: the metric goes up exactly when the product gets better. Five breaks:

• Imbalance: ROC-AUC x-axis is FPR (÷ negatives), so 100:1 imbalance is invisible. FPR 0.05 → precision 0.138. Report PR-AUC / precision-at-recall too.
• Wrong aggregation unit: per-row 0.970 over 20-row sessions = 0.97^20 = 0.544 session success. Report at the unit the user experiences.
• Inherited threshold: 0.5 is a default. With C_FN = 20*C_FP, optimum p* = C_FP/(C_FP+C_FN) = 1/21 = 0.048.
• Calibration (score multiplied by money/bid, not just thresholded): AUC is invariant to any monotone transform; calibration is not. Reliability table → ECE = Σ (n_b/n)|pred_b − obs_b|. Fix: isotonic or Platt on held-out split, refit every deploy.
• Proxy drift (Goodhart): offline metric up, business flat → burden of proof is on the offline metric.

Step 5 — Is the label wrong?

Assumption: the test label is the truth. Label error rate eta caps every score.

• Ceiling (binary): a perfect model scores 1 - eta. Invert a = (observed - eta)/(1 - 2*eta). Observed 0.85 at eta=0.12 → true accuracy 0.961. Observed above 1-eta = your noise estimate is wrong / test leaks. C classes: ceiling 1-eta holds; the “true accuracy” inversion is binary-only.
• Estimate eta: 200 test rows, 3 blind annotators, majority = reference. Report Cohen’s kappa.
• Human ceiling from kappa: undo chance correction p_o = kappa(1-p_e) + p_e (balance-dependent), then (1-eta)^2 + eta^2 = p_o → ceiling 1-eta. kappa 0.62 balanced-binary → ceiling 0.89, not 0.81.
• Find mislabels cheaply: rank by per-row loss / out-of-fold confident-learning; review top 100 by hand (often ~60% are test-set problems, not model errors).
• Clean TEST first: test noise caps the score and reverses rankings; cheaper by the size ratio (200,000 train vs 2,000 test = 100×). Train noise is second (partially self-correcting).

Step 6 — Capacity → features → data

Reached only when nothing was broken. Order set by information gained per hour.

• Capacity first because it is also the diagnostic: double width/depth, drop regularization, watch training error. If train error won’t fall even handicap-off → representation problem, not model size.
• Features second: the only lever that adds information; attacks the e_inf asymptote data cannot reach. Cost = point-in-time offline + online impl + parity test.
• Data last: slowest, and step 2 already priced it.
• Legitimate reorder: a starved slice jumps the queue (5,000 targeted rows beat 5,000 rows of the dominant slice).

Gotchas

• Verifying a leakage fix: the offline number must get worse and the offline↔online gap must close. Verify on the gap, never offline alone.
• A threshold tuned for model A means a different operating point on model B; ship the threshold in the model artifact.
• One class always predicted = degenerate optimum under imbalance; fix with class weights / resampling / focal loss — after checking the labels.