How to design a video-streaming service

A YouTube-scale video-on-demand service reads ~306 bytes for every byte written, so it is a CDN with an upload pipeline attached: delivery is the design, transcoding is a rounding error.

The ratio that decides everything

• Ingest: 500 k uploads/day, 12-min mean, 8 Mbps mezzanine = 720 MB/video = 0.36 PB/day (~33 Gbps).
• Egress: 110 PB/day. Read amplification = 110 / 0.36 ≈ 306:1.
• One PB/day spread evenly = 92.6 Gbps.
• Consequences: it’s a CDN not a database; a 10% codec win = $80 M/yr (egress is $800 M/yr); upload/transcode are cheap; storage tiering is a real budget line.
• Mezzanine: creator’s master file, never served, kept 90 days as the source for every rendition.

Core terms

• Codec: coder+decoder pair (H.264, AV1); same quality, different byte counts.
• Bitrate: compressed bits per second of playback (Mbps).
• Rendition / rung: the video re-encoded at one resolution+bitrate.
• Segment: independently downloadable 6-second piece.
• Manifest: ~2 KB text file naming every quality + segment URL; player fetches it first.
• Transcode: decode to raw pixels, re-encode at another resolution/bitrate/codec; most compute-hungry op.

Numbers to know

• Bitrate ladder (720-s video, size = Mbps × 720 / 8):

• Ladder (414 TB/day) costs more than the source (360 TB/day). Rungs cost storage, not compute (extra rungs add only ~80%: pixels halve each step, geometric sum ≈ 2× first term).
• Zipf views (s=1): top-m share = ln(m)/ln(K). Top 10 M videos (2%) = 80.5% of views; other 490 M (“tail”) average ~0.4 views/day.

Upload: resumable, 8 MB chunks

• Hazard rate p=0.002/s: survival = exp(-p·t). A 720 MB single-request upload (~1,152 s) succeeds only ~10%; retry restarts from zero.
• Optimal chunk: S* = rate × sqrt(2h/p) = 0.625 × sqrt(2·0.15/0.002) ≈ 7.66 MB → 8 MB (90 chunks). Overhead flat 4–16 MB; scales as 1/sqrt(p).
• Server dictates chunk_size; 409 returns committed_through resume point; per-chunk hashes make re-sends idempotent; ingest terminated regionally.
• Result: 1.3% wasted bytes vs 291% single-request. Ledger = 45 M commits/day (~450/s), its own store with short TTL.

Transcode DAG

• DAG (no cycles → terminates; any ready task runs anywhere). Serial head (assemble+verify+probe codec/fps/HDR/rotation) → fan-out (30-s units × 5 rungs = 120 tasks) → fan-in stitch + verify GOP → package → publish.
• Work unit 30 s = 5×6 s segments (whole segments → no extra keyframes).
• Cost: 1080p = 1/0.35 = 2.86 core-s/out-s; ladder 2.86×1.80 + 0.10 decode = 5.25 core-s/source-s. 12-min video = 3,780 core-s ≈ 1.05 core-hr.
• Fleet: 1.89 B core-s/day ÷ 2.21 M (32-core box, 80%) = 854 boxes mean → 1,281 at 1.5× (queue absorbs peaks into publish latency, so no peak factor).
• Wall clock: 63 min serial → 86 s on 120 machines (longest task = 30 s of 1080p ≈ 86 s).
• Software encode beats GPU/fixed-function: hardware needs ~15% more bits = $120 M/yr, 8.9×–23× any compute saving. (Hardware right for live only.)

Storage tiering + JIT

• Bills by GB-month: hot $0.023, cold $0.004. All-hot baseline = 414 PB = $114 M/yr.
• Head: every rung hot. Tail: keep two rungs cold (1080p + 360p), generate the rest on demand. Total ~$13 M/yr, saves ~$101 M.
• Keep 1080p (encoding runs downhill only — can derive lower, never higher) + 360p (cheap floor). Policy is derivability, NOT popularity.
• JIT transcode with a FAST preset (~0.29 core-s/s), on 6-s segments (1.7 core-s each, fits startup buffer). ~4,510 boxes, ~$47 M/yr → net save ~$54 M. Archival preset would need 16,291 boxes ($171 M) and lose money — preset is a function of expected read count.
• Reuse divisor ≈ 1 (a cached JIT result is one rung of one video, essentially no reuse).

CDN economics

• Edge hit ratio: tail copy serves ~0.056 requests (first-access miss, no cache fixes it); head ~1.6/day → ~1. Blended 0.805.
• Origin: 110 × 0.195 = 21.45 PB/day = 1,986 Gbps mean. Size = mean × 2 (peak) / 0.8 (NIC) = 4,965 machines (2.5× the mean). Tail = 19.5% of views, 100% of the origin problem.
• Bigger edge cache buys nothing (first-access, not capacity, miss). Fix = fewer caches: 4 regional shields at 30-day TTL. Reuse scales as TTL / shields; break-even ≈ 7.4 days; 30/4 = 7.5 sits on the knife edge (blended tail hit ~0.5%, not 66.6%).
• Head guards: prefetch to all 50 PoPs on publish (~41 GB); request coalescing at shield turns 50 simultaneous misses (hot key) into one origin fetch.

Adaptive bitrate (ABR)

• Client picks the rendition (buffer occupancy, throughput estimate, viewport size — state moves at ~100 ms, faster than a 150 ms round trip); server only publishes the manifest and guarantees alignment.
• Alignment: every rung places IDR keyframes at the same presentation timestamps → clean mid-stream switches. Forces IDR onto a fixed grid.
• Segment length: mean frame cost (10 + G − 1)/G. 6-s (G=180) = 1.050 vs 2-s (G=60) = 1.150. 2-s costs 9.5% more bits ($76 M/yr) for latency nobody perceives on recorded video. Ship 6 s on-demand, 2 s live.

Gotchas

• Segments are NOT DB rows (600 objects/video × 500 k = 300 M rows/day) — address by convention {video}/{rung}/seg{n}.m4s.
• Pick one byte base: MB (10^6) vs MiB mismatch bites on the last chunk.
• Dedup (exact hash, catches byte-identical only) ≠ copyright match (perceptual fingerprint/pHash, survives re-encode). Dedup: reference-count bytes, never change observable timing (membership oracle leak).
• Copyright: never gate publication (publish first, claim later); threshold set by reviewer headcount not a curve (1% FP = 5,000 claims/day = 83 reviewers); false-positive owner ≠ false-negative owner.
• Origin NIC sizing = mean → ×peak → ÷utilization → boxes; skipping the middle two under-buys by 2.5×.