How to design an event recommender

Recommend the local events (concerts, meetups, classes) a user will actually attend, where the item is permanently cold: an event has no history before it happens and zero value after.

The core fact

• Cold start never decays here: the item is destroyed before it warms up, so it is the normal case, not an edge case.
• Information and value move in opposite directions. Median event: 0 RSVPs at creation, ~9 at peak-value day (7 out), 34 at start, negative after.
• Collaborative filtering fails: matrix factorization needs ~10·d ≈ 640 interactions for d=64 vectors; you have ~9, underdetermined by 7x, so the fit returns the regularizer’s prior mean = global popularity. Only 0.3% of catalog ever reaches 640.
• The one collaborative signal that survives: social graph × current RSVP list (exists from the first RSVP). Lift vs baseline: 1 friend 4.7x, 2 friends 7.9x, 3+ friends 11.2x.

Two heads, one trained late

The delayed label (attendance) lands 9 days after the recommendation. Split it:

score = p_rsvp(u,e) · p_attend_given_rsvp(u,e)

• p_rsvp (marginal): retrains hourly on same-day labels, tracks the fast catalog.
• p_attend_given_rsvp (conditional): retrains weekly on 9-day-old labels; stale is fine because follow-through rates are stable.
• Train on RSVP directly = trap: free/distant events are over-RSVPed. RSVPs +9% but attendance −4.8%.
• Nothing else enters the score (distance already lives in both heads; a post-hoc multiplier double-counts and decalibrates).

Labels and their gotchas

• Objective is attendance, not RSVP. P(attend given RSVP): free 0.41 / paid 0.88; <5mi 0.71 / >15mi 0.44; alone 0.51 / 2+ friends 0.79.
• Pointwise logistic loss (need the calibrated scale for a quality floor and a notification threshold), not pairwise/listwise.
• Negatives: impressed-but-not-RSVPed (780:1) + ~20% random never-impressed candidates. Both downsampling and injection distort odds by a constant factor; apply the exact correction (the notification threshold reads an absolute probability).

Features: geometry and time carry it

134 features, 64 are one content-tower vector, so ~70 hand-built numbers, no learned item ID.

• Raw lat/long is bad: trees split axis-aligned; a “within 5mi” disc needs ~48 leaves and moves per user. Fix: Haversine distance (one scalar, one split). General rule: precompute translation-invariant differences.
• H3 grid, three resolutions for three jobs: res 4 = user×event cross (coarse so it stays learnable), res 5 = retrieval ring, res 7 = travel-time matrix + “desirable neighborhood.” H3 chosen over geohash (equal-area, one neighbor distance).
• Distance: no law fits (implied tau runs 4.4–20.1; it is a mixture of transport modes). Bucket at 2/5/10/20/40 mi; prefer precomputed travel time (cell-to-cell matrix, ~30MB/metro, live routing = 18M calls/s, impossible).
• Time: skip cyclic encoding; bucket hour-of-week (168 buckets, ~1.2M RSVPs each — enough data beats a smoothing prior).
• Shrinkage for thin metro×category×hour cells: blend up cell → metro×hour-of-week → hour-of-week → global with pseudocount m, keep n/(n+m). Fit m per level (empirical Bayes). Median user’s 3 RSVPs keep only 13% — “learn the city, then nudge.”
• Route collaborative machinery to persistent entities: organizer (400 events vs 9), series, venue, category. Most reusable idea.

Retrieval is geometry, not a model

3M events → ~15,000 candidates (selectivity 0.50%). Geo is an exact predicate, so partition on H3, do not use an ANN/HNSW index (an event 40mi out is a wrong answer, not a worse one; post-filter gives e^-2.5 = 8.2% empty feeds).

resolve loc → H3 res 5 (ring) + res 7 (travel)
k-ring 4 = 61 cells → guarantees 47.6 km ≥ 40.2 km
merge posting lists by start time
exact haversine ≤ 25 mi per id
stop at 21 days OR 15,000 in-radius
hard filters (sold-out, cancelled, RSVPed, started) at request time

• Ring size (1.5k − 0.5)·R: k=3 covers only 85% (not enough), k=4 covers 118%. Worst-placed user sits at the vertex (circumradius R), and the ring union is a notched star (1.5R through the notch, not spacing √3·R).
• Capacity is a gate, not a feature: sold-out read live from the authoritative counter (one 15k-key Redis MGET, ~6ms); a 15-min stale store serves sold-out events for 15 min on exactly the events that sell fastest. Fill rate stays a feature; sold out is a boolean.

Model, training, metrics

• Ranker: GBDT (LightGBM), ~300 trees, depth 8. Tabular + 200M rows + near-monotone → trees beat neural nets. Scoring ≈ 36M comparisons ≈ 36ms; feature fetch is the expensive part, not the model.
• Temporal split mandatory: a random split leaks fill_rate/rsvp_velocity (NDCG 0.412 leaky vs 0.317 temporal, and the better-looking leaky model lost online: −1.8% vs +2.4%).
• Imbalance (0.128% positive): weight if you can afford the rows (exact objective), downsample + correct if not.
• Offline: Recall@25 (gate), NDCG@25 (graded 3/2/1/0), ECE (gate). Only ~2% of impressions (random half of slot 25) is clean, unbiased eval data.
• Online decision metric: attended events per user per 30 days, reads out in ~44 days (7 lead + 30 window + 7 burn-in). Sample size is easy (~122k/arm); the experiment queue is scarce, so run arms in parallel. Watch capacity interference (compare 5% vs 50% arms). CUPED cuts variance ~35% but buys no time.

Key numbers

• 40M MAU, 3M live events, ~15,000 per metro, p99 250ms (p50 138 / p99 220).
• Median event lifetime 17 days; median user 3 lifetime RSVPs, 41% of MAU zero.
• Total cost ~$264k/yr; feature store dominates (users + friends, not events — events are 7GB of 256GB).

Top failure modes