How to design personalized feed ranking

Viewer identity plus session context in, ~25 ordered posts out, drawn from ~3,000 candidates that a per-user social graph and recommendation indexes produce.

Why a feed is not a catalog

• Supply is endogenous: creators adapt to last week’s ranker, so ranking is an input to next week’s candidate distribution.
• Candidate set is a graph query (your follows/groups), small and expiring, not a global pool.
• Objective is contested: viewer, producer, platform want different things; disagreement shows up as a term.
• Watch-time trap is worse here: an engagement-maximizing feed changes what gets written.

The objective and the proxy gap

• The viewer’s want (“leave better off”) has no logged event, so the score is written on proxies.
• Engagement and satisfaction rank content nearly in reverse: Spearman −0.55 across content types; the two highest-engagement types (outrage, bait) are the two lowest-satisfaction.
• Within one type, more engaging is mildly more satisfying (rho +0.35); flip across types is Simpson’s paradox.
• Four blocks, each because the one above is insufficient:

Value(u,p) =  sum_k w_k · P_k(engagement k)      logged for free
            + w_s · S(worth your time)           had to survey for it
            - sum_j c_j · N_j(hide/report/unfollow)
            - d(p) · Value_positive              integrity demotion

Candidate generation and serving

• In-network inventory is tiny (~300 posts median over a 72h window), so scan it exhaustively; no ANN needed. The real problem is fill for thin-graph users (18% of DAU need 75–90% out-of-network).
• Out-of-network: ANN over a global embedding index + topic/entity inverted indexes.
• Index turns over 33%/day (500M posts/day, 72h life); a nightly rebuild hides ~42% of the day’s engagement. Use incremental-insert HNSW with tombstones.
• Fanout hybrid: push (fanout-on-write) the long tail into per-user inboxes; pull the head. ~50k big accounts fit an 80 MB in-process hot set, killing the celebrity write burst (100M writes = 33s of global write budget) and the pull straggler math (0.99^200 = 0.134).

Recency: fit it, don’t pick it

• Confounded if measured raw (ranker shows good posts sooner). De-confound: measure age against interaction rate inside one score decile.
• Fit rate(t) = rate0 · exp(−lambda·t). Global half-life ~24h, but per-type half-lives span 4.1h to 284h (46× spread); one global number is wrong.
• Argue about the exchange rate: a day-old post must be exp(lambda·t) as good to hold its slot (2.0× global, 56× breaking news, 1.23× life event).
• Don’t multiply decay onto the score (model already saw age); feed age, log(age), age×type as features. Keep an explicit multiplier only for retrieval tie-breaking.
• Freshness trap: young posts (worth most) have null count features. Measuring per-post rates is arithmetically impossible (±0.02 needs ~120 impr/post ≈ 30% of inventory). Fixes: count-free cold-start path, fixed 3–6% exploration budget (Thompson sampling), maturity gate (k+alpha·prior)/(n+alpha).

Multi-task ranking: the combine is where designs break

• One shared-bottom network (optionally mixture of experts), 11 heads; one model not eleven because sparse embedding tables are 95% of memory. p(comment) and p(hide) peak on the same divisive content.
• 11 heads: click, dwell>10s, like / comment, reshare, dwell>60s / hide, see-fewer, unfollow, report / survey.
• Calibration is a launch gate (linear combine: a 1.4× overconfident head silently becomes a 1.4× weight edit). AUC/NDCG are only diagnostics.
• Heads live on base rates ~3,000× apart (survey 0.44 vs report 0.00015). Weighting raw probabilities makes a “balanced” vector 91.7% survey and inverts stated preferences.
• Fix: weight lifts, not probabilities. A lift = prediction / its base rate, unitless and addable:

Value = sum_k w_k·(p_k/p̄_k)  −  sum_j c_j·(n_j/n̄_j)

• Normalize positives to 1.000, negatives to 0.400; base-rate post scores 0.600.
• Share constraint holds only at the mean; ranking happens in the tail. Clip each lift at p99/base (≈3×) or bait wins anyway.
• Weights cannot be learned (no long-term-welfare label); set by humans, checked by 8–12 week holdouts (~20 evals/year). Keep 8–12 terms in a reviewed config; re-baselining base rates is a weight change.

Integrity, diversity, and the bad dashboard

• Engagement rises monotonically with p_violating up to the removal line (2.4× across servable buckets), so an engagement maximizer sits against the removal step. Fix with a continuous demotion ramp d(p) (starts 0.20, floors 0.85, ~5.9× at p=0.80) that beats the 2.4× gradient. Apply only when Value > 0 (else it promotes negatives). Demotion lets a low-precision classifier (0.14 precision at 0.20) do useful work.
• Diversity is a slate property no pointwise score expresses; greedy re-selection pass fills 25 slots, forgoing ~11% of pointwise value (ads interleave alone 4.9%). Author cap 3/25 also backstops the feedback loop (s_v ≤ 0.12).
• Dashboard that says ship: every weekly metric up (+5.4% interactions/session), every slow/asked metric down (survey −2.2pp, hides +18%, reports +6%). Supply response (bait 2.1%→5.6%) is invisible below population scale. Use long-term + reverse holdouts, guardrails as independent blockers, CUPED for the retention arm.

Failure modes (mechanism to control)

Gotchas

• Content encoders run at write, never read: 500M/day is a batch job, 200B impressions/day is 400× and impossible in-request.
• Serving bottleneck is memory traffic, not FLOPs: ~60 sparse embedding lookups/candidate, ~1.07 TB/s random access; route candidates to the shard holding their author row.
• Train with strictly temporal split (random leaks a post’s own counts) and mature labels (24h for comment/reshare, 1h censors 40%).
• Hourly continual updates train on the prior model’s own choices; the exploration budget is the only counterfactual source.
• No LLM on the read path ($3.2M/day); confine LLMs to offline labeling. No full session RL (reward is the same delayed proxy; use bandits + supervised heads).