How to design image-privacy blurring

Find every face and readable plate in planet-scale street imagery and irreversibly blur it, in an offline batch pipeline where a miss is a legal violation and an over-blur is a grey patch of wall.

What makes it different

• False negative (FN): unblurred real face, regulatory tail. False positive (FP): over-blurred wall. Costs ~4 orders of magnitude apart, so the operating point is derived, not chosen.
• Offline batch: capture-to-publish is weeks. No latency budget; currency is cost per image (~36 PB read + written).
• mAP (mean average precision) is the wrong headline. Report a per-slice miss rate; gate on the worst cell.
• Write the annotation guideline first: if “face” is undefined, recall is undefined.

Deriving the threshold

Detector emits confidence in [0,1]; blur above threshold t. Cost-ratio rule minimizes expected cost by acting when P(face) > C_fp / (C_fp + C_fn).

• Bug in the rule was the input not the formula: C_fp was modeled linear per box; 200 small boxes ≠ 200× the harm.
• Cap (0.5%, a product decision) ≠ spend (measured, 0.40% at t=0.035). Read the area column in log space (power law).
• Dilation grows every box 15% → area ×1.15²=1.32, so 0.40% publishes as 0.53%, over budget → threshold moves to t=0.045. The dilated area is what governs.

Per-face recall is the wrong unit

• A user sees a photograph, not a face. Compound: P(≥1 miss) = 1 - (1-m)^n.
• Suburban (n=3.2, m=0.0045): 1.43% → 28.6M panoramas/yr with an unblurred face. Dense urban (n=40): 16.5%, one in six.
• Holding a dense pano to 1% needs m=2.51e-4, 18× below shipped. Unreachable by threshold tuning → recall must come from redundancy.

Multi-view association (highest leverage, pure geometry)

Same pedestrian appears in 4-6 frames (trigger every 8 m, 0.72 s apart at 40 km/h). Project a detection into neighbors using pose + depth the rig already computes for stitching.

• Inputs: 6-DoF pose, per-pixel depth (~0.25 m RMS at 12 m), equirectangular intrinsics (36.98 px/deg).
• Not tracking: reprojected boxes don’t overlap (IoU = intersection/union = 0). Association is in world coordinates.
• Depth-sensitive: a 1 m (8%) depth error displaces the box up to 230 px, 4× the face width. So reprojection is a gate, not an answer:
  1. Gate: keep frame-3 detections (down to conf 0.002) within R=1.5 m of the 3D point (1.01 m walked + 0.50 m depth noise).
  2. Promote (82%): blur the below-threshold real detection itself, zero extra area.
  3. Fall back (18%): blur the reprojected quad, dilated by depth σ. Only 0.0026 boxes/pano, 1/650th of the budget.
• Threshold propagation too (gate at score ≥0.50), else FPs paint into 3 neighbors → 56 FP/pano.
• Misses correlate across views: not 0.0045^4 but measured 6.1e-5 → k_eff = 1.80 effective independent views. Dense-urban miss 16.5% → 0.24% at zero inference cost.
• Gotcha: mis-association in a crowd is benign (neighbor needed blurring anyway); depth failure on glass is not (reflections land too deep, gate hits empty pavement — not rescued).

Models, loss, architecture

Only two learned models. Most recall is geometry + pipeline, not modeling.

• Content gate: tiny binary classifier at 1/16 res, labels free from box labels, recall must be 0.9999.
• Detector: anchor-free single-stage + feature pyramid, classes {face, plate}. L = L_cls + λ_box·L_box.
  • Focal loss (α=0.25, γ=2.0) over cross-entropy: imbalance ~58,000:1 (87,296 locations vs ~1.5 objects); easy negatives swamp positives.
  • GIoU box term: plain IoU is flat at 0 for disjoint boxes; GIoU keeps a gradient.
  • λ_box low (~0.3 vs usual 2.0): localization is nearly worthless, dilate at blur time.

Metadata (time, geo, exposure) is deliberately withheld from the model: it buys mAP out of the worst cell (low-light, low-density). Used only outside the model (tone-mapping, association).

Small-object problem

Face 0.16 m spans ~9.17/D degrees × 36.98 px/deg: 5 m→68 px, 15 m→23 px, 30 m→11 px, 50 m→7 px (below floor). The 8-16 px band is 20% of in-scope faces but 62% of misses.

• Anchor assignment floor: anchors {32,64,128,256,512}. An 11 px face vs 32 px anchor has IoU=11²/32²=0.118 < 0.4 → trained as background. Nothing 8-22 px ever becomes positive; floor at ~23 px (a face at 15 m). Fix: anchor-free + center sampling (positive if inside the box, no IoU test).
• Pyramid level: at stride s a face f px wide is f/s cells. An 11 px face is 0.69 cells at P4 → needs P2 (stride 4). Full head at P2 = 619 GFLOP (3× the other 4 levels). Fix: cheap P2 head (depthwise-separable, 128 ch, 2 layers) = 9.2 GFLOP → 8-16 px recall 0.42 → 0.89.

Data and labels

• Cost: drawing $0.060/box, scanning $1.47/panorama (5:1). So annotate tiles not panoramas (6 s vs 4 min) and reweight by sampling probability; pre-annotate with the model (1.2 s correct vs 7 s draw).
• Blind 5% control (no proposals): measures label-process recall = 1,285/1,412 = 0.910. Model-assisted labels are 9% incomplete in exactly the hard band → control must be permanent, evaluation-only.
• Active learning: multi-view disagreement yields 214 hard examples/1,000 labeled (vs 12 random), free and self-labeling. Working mix: 50% multi-view, 30% slice-targeted, 20% random.
• Synthetic: downscale+JPEG, paste (needs Poisson blending or the seam becomes the shortcut), exposure jitter, motion smear. Diagnose leakage on real hard examples only.
• Split by capture_run_id (not random): a face is in 4-6 frames, random tile split leaks 99.84% of siblings → reads 0.9987 vs true 0.9955 (3.5× optimistic). Also hold out whole cities and the last six weeks.

Metrics to report

mAP misleads: averages recall levels you never operate at, prices localization (loose box is free here), and hides the one risky size band. Model B (mAP 0.541, 4.2 misses) beats Model A (mAP 0.612, 18.8 misses).

• Precision at shipped point is 0.26 (14 FP vs ~5 true/pano): 3 of 4 blurred regions are empty, priced into the area budget.
• Match rule: IoU ≥ 0.35, greedy one-to-one, global coords. Loose side is free; tight side (0.59× width covers a third of the face) is caught by effective recall, not IoU.
• Slice grid: skin-tone gap is an interaction, not a main effect. Marginal 1.24 pts hides daylight 0.12 vs night 2.47 (Monk scale 8-10). Cause is physical (auto-exposure crushes darker faces into the noise floor in low light). Fixes: worst-cell gate, exposure augmentation, slice-targeted labels, tone mapping → night cell 0.9661 → 0.9904.
• No A/B test: the control arm is published unblurred imagery (the harm). Substitutes: reweighting (publish weights), shadow scoring over past captures, staged rollout by capture run. Alarm on the association fallback rate (daily, free) since audit taps before association (74× off).

Redaction must be irreversible

• Gaussian blur is a convolution, transfer exp(-2π²σ²f²) — never zero, so Wiener deconvolution divides it back. At 40 dB SNR, σ=2 px, a 14 px eye is recoverable. Blur attenuates; it does not redact.
• Correct primitive: decimate to block means (many-to-one, no inverse) + zero-mean noise + re-encode (requantize DCT).
• Fix the block size, not a fixed 8×8 grid: a fixed grid is 1.9:1 on an 11 px face and the identity function on an 8 px one — weakest where risk is highest.

n = clamp(1, 8, floor(w / 6))     # w = dilated box width
ratio = (w/n)² ≥ 36:1 at every face size   # 6 px floor; 8-block cap avoids one flat rectangle

• Noise must be correlated across frames of the same identity: the face is in 4-6 published frames and averaging independent redactions reduces noise by √k.

Scale, cost, gotchas

• Compute ~$110k/yr ($55/M panoramas): detector 86%, decode 13%, re-encode 1% (84% pass through untouched). I/O is the real constraint: 36 PB each way / 30 days = 13.9 GB/s sustained.
• Content gate is a cascade: its recall bounds the system’s, so run it tighter than the detector. At 0.9999 it skips 61% → 2.55× cheaper (not the naive 4.5× at 0.994).
• Tiling: 1024², overlap 192 → 128 tiles. Largest face (340 px) exceeds overlap → run NMS in global coords, union cross-boundary boxes (don’t just add overlap: +56% compute).
• Reflections: recall 0.712 (64× worse) — out of training distribution; mine glass surfaces. Motion smear at dusk = 37 px (55% of a 68 px face) — capture-side fix (cap exposure 1/500 s).