How to design personalized headshots

Personalized headshots train a tiny per-user model on 10-20 selfies, then generate ~40 professional shots; because there is one model artifact per customer, quality, cost, storage, and privacy become a single problem that collapses onto “how many parameters does a user get, and who stores them.”

Core definitions

• Fine-tuning: continue training the base model’s own weights; you end with a whole new model (5.2 GB/user, not shareable).
• Adapter: freeze all original weights, add a small new file (42 MB, shareable).
• Trigger token <tok>: a rare meaningless string (e.g. sks) that becomes a handle for this person.
• Catastrophic forgetting: training hard on a narrow task destroys base competence; every fix is a regularizer.
• Two quality axes, they trade off monotonically with training steps: identity preservation (recognizably them) vs prompt following (suit, backdrop, pose). Pick the knee, not the max.
• Storage quoted in fp16 = 2 bytes/param. Speed quoted in MFU (fraction of a card’s peak; H100 peaks ~990 TFLOP/s).

The personalization ladder

• Storage spans ~160,000x; identity spans only 1.7x. That asymmetry is why nobody ships full fine-tunes.
• LoRA saving = 2r/d: add dW = B·A (rank r); ratio 2r/d = 32/2048 = 1.56% at r=16, independent of how many layers you touch.
• LoRA’s win is memory, not FLOPs: still forward + backprop through all 2.6B (~23% cheaper, not 100x). Frozen shared base = ~26 training jobs / ~1,570 adapters per 80 GB card.
• Textual inversion ceiling 0.44 (one point in text space can’t hold a face). Encoder-based ceiling 0.52 (recognition embeddings are built for invariance, so they discard glasses/hair/lighting). Ship encoder-based as instant preview, LoRA as the delivered set.

Choosing a rung

latency < 10s required?  --yes--> encoder-based (0 params, 1 KB)
        --no--> identity top complaint?  --no--> textual inversion (32 KB)
                --yes--> storage budget?  tight -> LoRA r=4 (5.2 MB)
                                          normal -> LoRA r=16 (42 MB)  [default]
                                          unbounded -> DreamBooth (5.2 GB, NOT batchable)

Economics (per user, LoRA r=16)

• H100 $2.50/GPU-hr; object storage $0.023/GB-mo; egress $0.09/GB.
• Generation dominates GPU: 1024px = 26.8 TFLOP/pass; 30 steps x2 (CFG) = 60 passes = 1,608 TFLOP/image; generate 48, deliver best 40.
• Co-batch 8 users: training cost 2.2x cheaper, latency 3.7x longer. Tiers: free 20 min, paid 5 min, instant preview 10 s.
• Marginal cost ~$0.49 vs $29 price = 98% gross margin, but compute is only 4.6% of contribution ($10.66/user).
• Halving GPU bill = +2.3% contribution; cutting refunds 9%→5% = +10.9%. The identity gate is worth ~5x any compute optimization.

Multi-tenancy and serving

• Hot-swap a 42 MB adapter: 42 ms from object store vs 4,060 ms to generate one image = 0.02% of a request. Swap cost is a rounding error.
• Keep adapters unmerged: merging bakes identity into W, pins batch=1. Unmerged costs +1.56% but lets 64 users share one base GEMM = 1.9x throughput. Best trade in the system.
• Cache tiers: VRAM (LRU + 30-min TTL) → NVMe (7 days) → object store (delete at 90 days + audit record). Hit rate >0.9 by construction (bursty 20-min sessions).
• Gotcha: adapter_index must be correct; a mis-routed row is a biometric leak, not a quality bug. Assert and test it.

Metrics and the knee

• ArcFace cos is meaningless without anchors: different people ~0.02, same person ~0.65, verification threshold 0.36. Our LoRA 0.68 reads “above the same-person anchor.”
• Step sweep (r=16): 400→2000 steps, identity 0.41→0.72, adherence 0.79→0.54, background leak 3%→58%. Knee at 1,200 steps.
• Rank is a regularizer, not just capacity: rank 4 behaves like fewer steps; rank 64 = more memorization (leak 44%), worse.
• Identity gate (cos ≥ 0.45) drops ~14%, so generate 48 for 40. Costs $0.041, saves $0.78 in refunds = 19:1, the highest-return component.

Gotchas and failure modes

• Every failure traces to the missing loss term: nothing ranks “bone structure” above “wallpaper,” so <tok> learns the training set’s joint distribution, not the person.
• Background/clothing overfit fixes, ranked (objective-changing beats data-changing): 1) caption nuisance variables, 2) face-masked loss, 3) prior preservation (~200 base “person” images), 4) lower rank/fewer steps, 5) augmentation (least).
• Class bleed (multi-person prompt returns clones) and prompt collapse (framing baked in): fix with prior preservation.
• Demographic bias: ArcFace is itself biased on darker skin, so report download rate and refund rate by bucket (human-judged), never ArcFace by bucket. Calibrate gate thresholds against a human study, never aggregate the slice away.
• Biometric-derived data: deletion must reach source photos, adapter, VRAM + NVMe caches, and outputs, and emit an audit record. Uploads must never contribute a gradient to the shared base, or deletion is unachievable. Consent must be specific and withdrawable. Get this wrong and there is no product.