How to design similar-item recommendations

Define “similar” as substitutable for this trip, and the sessions pick the signal, the market picks the shard, and retrieval collapses into a library.

The problem

• Anchor listing: the page being read; module fills the strip below with 10 ranked listings.
• Catalog: 5.0M listings, ~1,400 markets (city/destination). Median market 900 listings; mean 3,600; p99 42,000; largest 380,000.
• Traffic 300M sessions/month, ~10 views each, renders on 40%. Latency p99 80 ms (below fold). Drives 11% of bookings.
• Two medians: market-weighted 900 (“most markets are small”), listing-weighted ~15,000 (where the median request/pair lands). Weight by what you count; they differ 17x. Price serving and negatives at ~15,000.

What “similar” means

• Only substitutability has a booking as its optimum, and it is not recoverable from attributes: only 34% of top-decile co-view pairs are also top-decile visually similar. Train on sessions.

Objective

• sub(A,B) = P(books B | viewed A this session, did not book A, B was available). “This session” fixes the trip; “did not book A” makes it substitution not complement; “B available” removes the booked-out confound.
• Learn one embedding v per listing so substitutes sit near under cosine (dot product / both lengths, -1..1).
• L2-normalize at export → unit vectors, so serving dot product is cosine. Blending unit vectors shortens them; anything that blends must re-normalize.
• No calibration: score is a retrieval key, only order within one market matters, nothing multiplies it by money.

Data: the session

• Session = ordered listing-page views, cut on 30 min idle or a booking. Not the account lifetime (different trips → different intent). P(same market) 0.91 within session vs 0.34 across.
• Two labels: co-view (both viewed, symmetric, precision 0.34) and viewed→booked (asymmetric, B dominated A, precision 0.78).

• Train on views for volume: ~10 views → C(10,2)=45 pairs → 1.35e10 co-view vs 1.94e8 view→book pairs/month (69x more, 2.3x lower precision).
• No dates on ~38% of views → degrade filter to a bookable-soon bit (one free 2-night window in 60 days, selectivity ~0.90); don’t fabricate dates.

Model: skip-gram + negative sampling

• word2vec: session = sentence, listing = word. Positive = within 5 positions same session; negative = sampled outside session. Output d=32 unit vectors.
• Loss (window 5, N=5 negatives): L = -Σ[ log σ(v_c·v_l) + Σ_n log σ(-v_c·v_n) ], σ = 1/(1+e^-z). Dot here is a logit on unnormalized params; normalize only at export.
• d=32 on purpose: within-market signal is low-dimensional (price, capacity, style, sub-neighbourhood); larger d buys memorization + hubness, and 32 fits the whole index on one machine.
• Booking as global context: add booked listing as context for every position in a booking session → recall@100 0.37 → 0.44 (+19%). Rare label injected as a different kind of pair, not re-weighted away.

Negative sampling decides the model

• P(global negative same-market) = 15,000/5M = 0.003 → 99.7% separable by market alone → those terms teach nothing.

• Global-only collapses within-market 4x; same-market-only can’t tell Lisbon from Paris. Mix, weight toward same-market.

Cold start: content tower

• New listings: 8% of inventory, 0.4% of co-occurrences (20x under-rep). No sessions → random vector (worse than useless).
• Item content tower: small net mapping creation-time attrs (H3 cell, price/capacity/beds/type buckets, amenity multi-hot, instant-book, host tenure, off-the-shelf photo + review-text embeddings) into the same 32-dim space, same objective. recall 0.29 (66% of warm 0.44); beats the nearest-3 averaging heuristic (0.14).
• Blend with shrinkage: v = normalize(a·v_beh + (1-a)·v_content), a = n/(n+40). m=40 = recall crossover; a=0.11 at n=5, 0.50 at 40, 0.71 at 100.
• normalize is load-bearing: unnormalized blend shortens (worst at a=0.5), a right-direction cold listing scores 0.672 vs warm 1.0 → fewer impressions → n stays low → death spiral. Antipodal inputs cancel to the zero vector at n=40; handle it explicitly (never pass a zero vector to the index).

Constraints reshape retrieval

• Unavailable listing has value exactly zero: upside 0, downside 17 pts extra abandonment (23% vs 6% baseline). No score buys it back → hard filter, not a feature.
• Pre-filter, not post-filter. Selectivities multiply (assume independence): dates 0.34 × price 0.40 × capacity 0.55 = 0.075. Post-filtering top 100 → 7.5 survivors; tail query (0.0009) → E[survivors]=0.36, P(empty)=e^-0.36=70%.
• Market = free shard key (nobody substitutes across cities). Scans 15,000 not 5M (333x smaller): ~35 µs median request, ~0.9 ms largest market. ANN only pays above ~5M candidates/query → no ANN index here.
• Transposed calendar: bitmap per day over listings (625 KB each, 228 MB for 365 days, resident), not calendar per listing. “Free Jul 4,5,6” = three bitwise ANDs (~2 µs in-shard). Availability becomes a cheap pre-filter.

Surface decides the target

• Post-booking wants the user-level cross-session signal rejected everywhere else. Same logs, different cut.

Metrics

• Offline recall ladder (held-out last-viewed→booked, k drawn from ~1,120 eligible set): random 0.021 → attribute NN 0.11 → content tower 0.29 → skip-gram global-only 0.09 → +same-market 1:1 0.37 → +booking context 0.44 → +content blend 0.46 → +CSLS 0.48. k must not exceed the candidate set, or recall@k is not a metric.
• Ladder only sees ranker-chosen listings → rewards geographic collapse. Check the randomized slot: cosine-only 0.22 vs after MMR+caps 0.29 recall@10; the unbiased number tracks the online +9.4%.
• Online: bookings per user on a user-randomized A/B test. Module CTR and attributed bookings both call a zero-gain A→B substitution a win (cannibalization). Sizing: n ≈ 16σ²/δ² → ~4.7M users/arm (σ=0.26, 1% MDE). Halving MDE costs 4x users.
• Guardrails (supply has no vote): cancellation, review score, host exposure Gini/top-1% share, new-host time-to-first-booking, anchor-to-booked distance, median/p1 eligible-set size. Interference: treatment consumes inventory control needed (real at 50%, negligible at 5%); detect via 5% vs 50% arms.

Failure modes

• CSLS: sim'(x,y) = 2·cos(x,y) - r_k(x) - r_k(y), r_k = mean cosine to k=20 NN, precomputed at build. Penalizes hubs; top-1% share of slots 34% → 16%, recall 0.44 → 0.48 (both improve).
• Cold-slot reservation: 1 slot in 10 → cost 0.12% of platform bookings, buys new-host time-to-first-booking 31 d → 11 d (2.8x) and the only unbiased eval data.
• Near-dup key = building geohash + capacity + photo perceptual hash (not host id).
• Stale availability: flip the bit synchronously on the booking write; nightly rebuild from source of truth.

Serving and cost

• Whole index < 1 GB (640 MB embeddings + 228 MB bitmaps) → every node a full replica: no sharding, fan-out, or consistency protocol. A library, not a distributed system.
• Latency: 0.5 ms p50, 1.2 ms p99 vs 80 ms budget (~65x headroom).
• Path: infer trip → resolve shard → constraint bitmaps AND → eligible set (~1,120) → exact dot products → CSLS → MMR re-rank + caps → 1 randomized cold slot → 10 listings.
• Cost ~$104k/year, almost all standby capacity (24 nodes = $63k doing <2 cores of work). Bound by experiment time, nothing technical.

Rejected alternatives

• Attribute/visual similarity: only 34% of co-view pairs look alike; visual measures the photographer.
• User-level co-occurrence: P(same market) 0.91 → 0.34 across sessions; learns a fact about the person.
• Co-bookings only: 0.78 precision but 69x fewer pairs → use as global context instead.
• Global random negatives: 99.7% market-separable → within-market recall 0.09.
• Large d (256/512): buys hubness + memory; signal is low-dimensional.
• HNSW over 5M + metadata filter: tail selectivity 0.09% collapses pre-filtered traversal; largest market is a 0.9 ms exact scan.
• Availability as a feature: value identically zero, nothing to trade.
• Cross-encoder reranker: MMR + caps overwrite fine score order; recall@100 already 0.48.