How to design a video-streaming service

In this lesson, we’ll design a video-on-demand service at YouTube scale, and we’ll do it by chasing a single number until it dictates every part of the system. That number: the service ships roughly 300 bytes to viewers for every byte a creator uploads. Follow that ratio end to end and it decides the whole infrastructure. By the end you’ll be able to size each stage from the workload, say which stage is a rounding error and which one is the design, and defend every trade-off from one measured figure.

We size four things:

1. the upload path, which has to survive a phone losing signal;
2. the compute that turns one uploaded file into several streamable qualities;
3. the storage that keeps those qualities;
4. the network that delivers them.

Here is where the arithmetic lands, so you know what we are driving at: the delivery network is the design, and the encoding cluster is a rounding error beside it. A 10% improvement in video compression is worth more than the entire compute budget. Every section below is that claim, made concrete one stage at a time.

Two words to fix first

Two terms carry the economics, so let’s fix them before we lean on them. A codec is a matched pair of algorithms (a coder and a decoder) that squeeze video into a compact byte stream and expand it back into pictures. H.264 and AV1 are codecs. Different codecs hit the same picture quality with different numbers of bytes, and the whole economic argument here turns on that difference.

A bitrate is how many bits of compressed stream one second of playback costs, in megabits per second (Mbps). A 12-minute video at 4.5 Mbps spends 4.5 million bits on each of its 720 seconds. Higher bitrate means a better picture and more bytes to ship, and at this scale, bytes to ship is the only cost that matters.

The input and the output

Input: one file, a 12-minute recording of about 720 MB, pushed over a link that may drop mid-transfer.

Output: a set of renditions: the same video re-encoded at several picture qualities, from 240 pixels tall up to full 1080p. Each rendition is cut into six-second segments: small, independently downloadable pieces a player can fetch one at a time. Alongside them sits a manifest, a ~2 KB text file naming every quality and the URL of every segment. A player downloads the manifest first, then pulls segments in order, re-deciding which quality it can afford before each one.

Everything in this design is one of four jobs: getting the file in, converting it into that set, storing the set, or shipping segments out.

Why the ratio decides everything

Read the ratio the other way and the shape of the system appears. It reads about 306 bytes for every byte written, and a system that reads 306x more than it writes is a content delivery network (CDN) (a fleet of caching servers placed close to viewers, so most requests are answered near the user instead of from your own datacenter) with an upload pipeline attached.

That split sorts every component for us. Everything expensive is downstream of the ratio: the egress bill (egress is bytes leaving your network toward users, which is what a cloud provider actually charges for), the codec team, and storage tiering. The origin (your own authoritative copy, the thing the caches fall back to) serves almost nothing.

Everything hard is upstream of it: an upload that has to survive a mobile link, and a transcode fan-out that turns one file into five. Transcoding is decoding a video back into raw pixels and re-encoding it at a different resolution, bitrate, or codec, and it is the single most compute-hungry operation in the system. Hold onto that tension: the hard part and the expensive part are not the same part, and the whole design is arranged around telling them apart.

Scope

Search over this corpus (speech recognition, transcripts, multimodal retrieval) belongs to the video-search chapter. The home feed (candidate generation and ranking) belongs to the video-recommender chapter. Neither is re-derived here. This design owns the bytes: getting them in, transforming them, storing them, getting them out. Comments, live streaming, and monetization are out of scope too.

The estimation techniques used throughout, and the egress figure extended here, are in the estimation chapter. Every number borrowed from it is restated where it is used.

The decision that shapes everything

Let’s compute the ratio properly, because one number holds up the rest of the design: how many bytes go out for every byte that comes in.

A few units first. A petabyte (PB) is a million gigabytes, the natural unit for a day’s traffic. Gigabits per second (Gbps) is a rate, not a quantity. One petabyte spread evenly across a day is 92.6 Gbps (a PB is 10^15 bytes, a byte is 8 bits, a day is 86,400 s). Every “PB/day → Gbps” conversion below is a multiply by 92.6. The mezzanine is the master file the creator uploaded, high quality, large, never served to a viewer, kept only as the source for every rendition.

Assume 500 k uploads/day, 12-minute mean, 8 Mbps mezzanine (720 MB per video). That is 360 TB/day of ingest = 0.36 PB/day ≈ 33 Gbps. Egress, from the estimation chapter, is 110 PB/day.

Read amplification = 110 / 0.36 ≈ 306. The read-to-write ratio of this system is 306 to 1, and that single figure reorders every decision below.

What actually breaks

Four failures, roughly in order of how often you see them.

• An upload dies at 80% on a train and the client starts over from zero.
• A transcode job stalls on one rendition and the video publishes with a hole in its bitrate ladder, the fixed set of quality levels, each with its own resolution and bitrate, produced for every video. A hole means one quality level is simply missing.
• A video goes viral before the caching network has a copy, and the origin absorbs a step change of terabits per second.
• A copyright claim fires on a false positive (the matcher says “this is someone else’s content” when it is not) and a creator is demonetized by a pipeline nobody can explain.

Those four failures are what the requirements below exist to bound. Let’s write them down.

Requirements

Functional

• Upload a video from a browser or phone, over a link that may drop.
• Transcode into a rendition ladder, package it for adaptive streaming, and publish it. Packaging is the step after encoding that cuts each rendition into segments and writes the manifest describing them.
• Stream with adaptive bitrate, so the picture does not stall when the network dips.
• Produce thumbnails (the still shown before playback), storyboards (a grid of small frames shown while scrubbing the timeline), captions, and a per-video manifest.
• Detect duplicate uploads, and match every upload against a copyright reference set.

Non-functional

p50 means the median (half of cases are faster); p99 would be where the slowest 1% begin. An SLA (service level agreement) is the availability you contractually promise: 99.99% permits about 53 minutes of downtime a year, 99.9% about 8.8 hours.

The asymmetry between the two availability rows is deliberate. A viewer who gets a playback error simply leaves, so playback must be 99.99%. A failed upload is retried, usually by a human watching a progress bar, so uploads can be a full order of magnitude less available. That gap licenses the split common to read-heavy designs: a fat read path served from caches at the network edge (the caching machines nearest the viewer) and a thin write path that accepts work into a queue and answers immediately. The URL-shortener chapter reaches the same split; the difference here is that this read path moves 110 PB a day.

Back of the envelope

Two quantities get spent by every later section, so we compute them once here: what one video costs to store after it is encoded at every quality, and how unevenly views spread across the catalogue. The first sets the storage bill; the second decides which of that storage is worth keeping fast.

What one video costs to store

A rung is one step of the ladder: one resolution at one bitrate. Converting each rung’s bitrate to megabytes for a 720-second video (Mbps × 720 / 8):

At 500 k uploads/day that is 414 TB/day of renditions against 360 TB/day of source: the ladder costs more than the original. Every rung you add is a permanent storage commitment, not a compute one. The mezzanine, retained 90 days for future re-encodes, adds about 32 PB.

How views spread across the catalogue

The intuition is that attention is wildly lopsided: a few videos get watched constantly and the vast majority almost never. Views follow Zipf’s law, which makes that precise: rank videos by popularity and the view count at rank r falls off roughly as 1/r, so rank 100 gets about a hundredth of rank 1’s views. With exponent s = 1, the share of all views captured by the top m of K videos is just ln(m) / ln(K), a ratio of two logarithms and no other input.

Put the real numbers in. For K = 500 M and m = 10 M: ln(10M) / ln(500M) = 16.12 / 20.03 ≈ 0.805.

The top 10 million videos (2% of the corpus) take 80.5% of views. The other 490 million average about 0.4 views a day each (the remaining 19.5% of ~1 B daily views, spread over 490 M videos). That last number drives the storage-tiering and CDN-economics deep dives: it is why the long tail cannot usefully be cached near viewers, and why keeping its full ladder in fast storage is paying rent on bytes nobody reads. Throughout, head means the top 10 million and tail means the other 490 million. That head/tail split is the lever behind nearly every cost decision from here on.

API sketch

Three important decisions are visible in the request shapes alone: the client announces an upload, pushes it in pieces, says it is finished, and later a player fetches the manifest and then segments.

POST creates, PUT writes a named thing, GET reads. Status codes: 201 created, 202 accepted-but-not-finished, 204 done with nothing to return, 409 conflict (the server disagrees with the client’s idea of the current state). sha256 is a cryptographic hash: run it over bytes and you get a 32-byte fingerprint effectively unique to that content. The ingest_url is a presigned upload URL: a temporary, signed, single-purpose address that lets the client write bytes straight to storage without the storage system ever seeing the user’s credentials. master.m3u8 is a manifest in the HLS (HTTP Live Streaming) text format; .m4s is one media segment.

POST /v1/uploads
  {"bytes": 720000000, "sha256": "...", "duration_s": 720, "title": "..."}
  201 {"upload_id": "u_9f3", "chunk_size": 8000000,
       "ingest_url": "https://ingest-eu.../u_9f3"}

PUT  /v1/uploads/u_9f3/chunks/{n}
  Content-Range: bytes 40000000-47999999/720000000
  204  committed
  409  {"committed_through": 39999999}      <- the resume point

POST /v1/uploads/u_9f3/complete
  202 {"video_id": "v_7Kq", "state": "transcoding", "ladder": []}

GET  /v1/videos/v_7Kq/master.m3u8           the ABR manifest, ~2 KB, cacheable
GET  /cdn/v_7Kq/720p/seg00042.m4s           a 6-second segment

Four deliberate choices:

• The byte literals are decimal and match the derivations exactly. 720000000 and 8000000 are 720 MB and 8 MB with MB = 10^6 bytes. The binary neighbours (754,974,720 for 720 MiB, 8,388,608 for 8 MiB) divide to the same 90 chunks, which is exactly why mixing them survives review and then bites: a client sizing buffers in MiB against a server counting MB disagrees about the last chunk’s length. Pick one base.
• The server dictates chunk_size. Chunking means splitting the file into fixed-size pieces uploaded as separate requests, so a dropped connection costs one piece, not the whole file. The size is derived from the link’s failure rate below, and the server can vary it per network type, a phone on cellular gets a different number from a desktop on fibre. If clients choose, the worst-behaved clients choose worst.
• The 409 carries the resume point instead of requiring a separate probe. A reconnected client that must ask twice doubles round trips in exactly the situation where round trips are unreliable.
• complete returns 202 with an empty ladder. Publication is asynchronous by construction (86 seconds at best, below), and an API that pretends otherwise forces every client to hold a connection open through it.

Data model

The real question is what goes in a database and what does not. The bytes of the video live in object storage: a service that stores arbitrarily large blobs under a string key, with no query language and effectively unlimited capacity (Amazon S3 is the familiar example). The small mutable facts live in a conventional database.

videos
  video_id PK, owner_id, duration_ms, state,
  mezzanine_key  -- object key, deleted after 90 days
  content_hash   -- sha256 of uploaded bytes, for dedup
  -- state: uploading | transcoding | live | blocked

renditions      -- one row per (video, rung), written by the transcode DAG
  video_id, rung, state, bitrate, key, segment_count

uploads         -- the resume ledger, hot for hours then cold
  upload_id PK, video_id, total_bytes, committed_through, chunk_hashes[]

segments        -- NOT a table. Object keys by convention: {video}/{rung}/seg{n}.m4s

Segments are deliberately not in a database. A 12-minute video at five rungs is 720/6 × 5 = 600 objects, and at 500 k uploads a day that is 300 million rows a day of pure naming. Address them by convention instead, so the manifest is generated arithmetically from duration and rung and the storage layer never gets a row per object.

Every remaining query is a point lookup (one row by its identifier, never a range scan or a join), which is the exact shape a key-value store is built for. Spread it across machines with consistent hashing: hash each video_id and each server onto one circle, and let each key belong to the first server clockwise from it, so adding a server relocates only the keys between it and its neighbour. The key-value store chapter builds the store and the consistent-hashing chapter builds the ring.

The exception is the uploads table: written 45 million times a day (below) and read almost never afterwards. It wants a short TTL (time-to-live, the store deletes each row automatically once it reaches a set age) and its own store, so a flood of upload bookkeeping cannot slow the table playback depends on.

High-level architecture

Two independent paths meet only in storage: a write path from creator to published video, and a read path from viewer back to whichever cache still has the bytes.

flowchart TD
    C["Client<br/>8 MB chunks · resumable"] --> IN["Ingest service<br/>regional, near the uploader"]
    IN --> OBJ[("Object store: mezzanine<br/>authoritative copy")]
    IN --> LED[("Upload ledger<br/>committed_through")]
    IN -->|"complete"| Q[["Transcode queue<br/>priority by creator tier"]]

    Q --> DAG["Transcode DAG<br/>most compute-hungry box, yet not the constraint"]
    DAG --> REND[("Renditions<br/>hot + cold classes")]
    DAG --> PKG["Package: CMAF segments,<br/>manifest, thumbnails"]
    PKG --> PUB["Publish: state = live"]

    OBJ -.-> FP["Fingerprint + copyright<br/>separate pipeline, hours; can demonetize"]
    OBJ -.-> ASR["ASR / captions<br/>feeds video search"]
    OBJ -.-> EMB["Item embeddings<br/>feeds recommender"]

    V["Viewer"] --> EDGE["Edge PoP (50)<br/>read cache, nearest viewer"]
    EDGE -->|"miss"| SH["Regional shield (4)<br/>read cache, shared by edges"]
    SH -->|"miss"| ORG["Origin + JIT transcoder<br/>NIC-bound"]
    ORG --> REND

The mezzanine in the object store is the authoritative copy; every rendition, segment, manifest, and thumbnail is derived and can be rebuilt from it, which is exactly why the tiering deep dive is allowed to throw rungs away. The edge PoPs and regional shields are read capacity: they answer reads without asking the authority. The origin is bound by its network cards, not its cores. And the transcode DAG, by far the most compute-hungry box on the page, is not what the design bends around: compute is not the constraint.

The write path

The client chunks the file into 8 MB pieces and uploads them resumably. They land at an ingest service that is regional (near the uploader) so the many small commit round trips do not each cross an ocean. Ingest writes the assembled bytes into the object store as the mezzanine, and maintains an upload ledger whose one important field is committed_through: the byte offset the server can prove it has.

On complete, ingest places a job on a transcode queue ordered by creator tier, so a large channel’s video is not stuck behind a backlog of hobbyist uploads. The job runs on the transcode DAG (directed acyclic graph, a set of tasks with arrows showing which must finish before which, and no cycles). Its output goes two places: renditions to storage, split into hot (fast, expensive, immediately readable) and cold (cheaper, higher latency) classes; and a packaging step that produces CMAF segments, a manifest, and thumbnails. CMAF (Common Media Application Format) is the container that lets one set of segment files serve both common streaming protocols instead of storing two copies. Only when packaging finishes does Publish flip the video’s row from transcoding to live.

Three pipelines hang off the mezzanine and none may delay publication (hence the dotted arrows): fingerprint-and-copyright matching, whose latency budget is hours; speech recognition for captions and search; and item embeddings (fixed-length vectors summarizing a video so similar videos land near each other) for the recommender. None is designed here.

The read path, three cache layers deep

A player asks an edge PoP (point of presence, one of ~50 small cache clusters in major metros). On a miss (the segment is not in that cache) the edge asks a regional shield: a larger cache tier shared by many edges, so an object fetched once for one edge is already present for the next. There are only 4. On a further miss the shield asks the origin, which is both the authoritative store of renditions and a just-in-time (JIT) transcoder that can generate a missing quality level on the spot instead of reading a stored one.

Deep dive 1: resumable upload, and where 8 MB comes from

We start on the hard-but-cheap side of the system, the upload, and derive the one magic number in it, 8 MB, from the physics of a flaky link. First, a warning about vocabulary: three different chunkings live in this system, and conflating them is the fastest way to confuse yourself:

They are independent, and the only constraint between them is that the transcode work unit must be a whole number of media segments (30 = 5 × 6) so splitting the encode across machines forces no keyframe the ladder did not already need.

Why a single request does not work

The intuition is that a long transfer is not just a bit riskier than a short one, it is exponentially riskier, because every second is a fresh chance to fail. A hazard rate is the probability per unit time that a live connection dies; if it is constant at p per second, the chance of surviving t seconds is exp(-p·t), so a transfer twice as long is far worse than twice as risky. Let’s put numbers on it.

Take a 5 Mbps mobile uplink (0.625 MB/s of goodput, useful throughput after overhead and retransmission) and p = 0.002 per second (one drop per 500 s). A 720 MB file takes ~1,152 s, expects 1,152 × 0.002 ≈ 2.3 drops, and finishes in one request with probability exp(-2.3) ≈ 10%. A single-request upload succeeds one time in ten, and retrying does not help because a retry starts from zero.

Costing that retry is where the obvious model goes wrong. The tempting shortcut, “a failed attempt gets about halfway, so it wastes half a file”, is wrong for a whole file. A failure conditioned on dying before the deadline lands earlier than the midpoint, because deaths are spread exponentially and most happen early. The conditional mean here is E[T | T < L] ≈ 372 s, or 32.3% of the file, not 50%. So expected bytes transferred are about 3.9× the file, not 5.5×.

That /2 shortcut is the right approximation for a chunk and the wrong one for a whole file: it holds only when the attempt is short compared with the mean time between drops (1/p = 500 s). A chunk satisfies that (p·t ≈ 0.026); a 1,152-second file does not. Either way, chunking is not optional.

Where the size comes from

Chunk size is a two-term trade-off. Small chunks pay a fixed per-chunk cost h (request framing, a durable offset commit, a metadata write), about 150 ms each, scaling as 1/S. Large chunks pay retry waste: a drop discards whatever chunk was in flight, scaling as S. A 1/S term plus an S term has a minimum where the two are equal, giving the closed form:

S* = rate × sqrt(2h / p) = 0.625 × sqrt(2 × 0.15 / 0.002) = 0.625 × 12.25 ≈ 7.66 MB  →  8 MB

Checking the curve at several sizes shows why the exact value barely matters. Total overhead is nearly flat from 4 to 16 MB, where handshake cost and retry waste cross:

The single-request bottom row is computed differently on purpose: its p·t = 2.3 is far outside the “half the chunk is lost” regime, so its waste comes from the conditional mean above (9 failed attempts × 372 s ≈ 3,350 s). Applying the small-p·t shortcut to that row gives the commonly quoted +451%, which is the wrong formula for it.

The closed form gives the whole sensitivity. Chunk size scales as 1/sqrt(p): quadruple the drop rate and the chunk halves. A satellite link at p = 0.02 wants ~2.4 MB; a datacenter copy with p ≈ 0 wants chunks as large as the object store will take.

At 8 MB, a 720 MB upload is 90 chunks, each retried with probability p·t ≈ 0.026, so ~2.3 retries and about half a chunk lost each: 1.3% of bytes re-sent, against 291% for the single-request design. Three server-side requirements make that work:

• A durable committed_through offset, written on every chunk commit. That is 500,000 × 90 = 45 M commits/day ≈ 450/s, trivial for a key-value store, and the reason the ledger is its own table with a short expiry instead of a column on videos.
• Per-chunk content hashes, so re-sending a chunk is idempotent. Idempotent means doing it twice has the same effect as once. Without the hash, a chunk whose 204 was lost gets written twice, and a client resuming from a stale offset silently corrupts the file. The hash makes the duplicate a no-op and makes real corruption detectable at complete.
• Ingest terminated regionally. A cross-continent upload pays ~150 ms of round-trip time on every commit, which at 90 chunks is 13.5 s of pure waiting bolted onto an already slow operation.

Deep dive 2: transcoding as a DAG

Now the compute stage, where we answer two separate questions people routinely mix up: how many machines the fleet needs (throughput) and how long one creator waits (wall clock). Modelling the work as a DAG answers both, because the DAG has two useful properties: no cycles means it is guaranteed to terminate, and any task whose inputs are ready can run immediately on any machine.

Compute is counted in core-seconds (one CPU core busy for one second; a core-hour is 3,600). Two accounting terms matter: offered load is how much work arrives, in core-seconds/day; capacity is how much a machine can absorb. Fleet size is offered load ÷ capacity, after a peak factor and a utilization target (a queue served at exactly its arrival rate has unbounded waiting time, so you leave headroom; 80% is conventional), never the raw mean divided by a rated maximum.

The ladder’s compute cost

Encoder work scales with pixel count. Summing width × height across the five rungs gives 1.80× the 1080p pixel count alone, pixels roughly halve at each step down, and a geometric series with ratio near ½ sums to about twice its first term. So adding four rungs below 1080p costs 80% more, not 400% more. Nobody agonizes over ladder length for that reason; it is storage that scales with rung count.

Turning pixels into machines needs two terms. A preset is the encoder’s speed-vs-quality setting: a slow archival preset searches harder and produces a smaller file at the same quality; a fast preset gives up sooner and emits a bigger file. A realtime multiplier expresses encoder speed against playback: “0.35× realtime” means one core produces 0.35 s of finished video per second of work. Decoding the mezzanine back to raw frames is far cheaper, so it is done once and shared by all five encodes.

At the archival preset (1080p at 0.35× realtime, decode at 10× realtime):

• 1080p costs 1/0.35 = 2.86 core-s per output second; the whole ladder 2.86 × 1.80 = 5.15; plus 0.10 for the shared decode = 5.25 core-s per source second.
• A 12-minute video: 720 × 5.25 = 3,780 core-s = 1.05 core-hours.
• Offered load: 500,000 × 3,780 ≈ 1.89 B core-s/day. A 32-core box at 80% supplies 32 × 0.8 × 86,400 ≈ 2.21 M core-s/day, so 854 boxes exactly meet the mean.

854 is demand, not a fleet. A fleet sized at exactly the offered load has zero spare capacity, so it can hold a backlog level but never work it down. Any hour that runs long stays long forever. Provision 1.5× = 1,281 boxes. This is the one tier where you do not multiply by a peak factor: uploads are bursty, but transcoding sits behind a queue, and a queue converts a spike into publish latency, not machines. The 1.5× buys the drain rate after a spike plus two priority lanes. 1,281 is the number every cost below is computed from.

Why a DAG and not a loop

Fleet size says nothing about how long one creator waits: that is wall clock, answered by cutting the video into pieces encoded in parallel. Sixty-three minutes on one core becomes 86 seconds on 120 machines: 24 work units of 30 s × 5 rungs = 120 independent encode tasks, and the longest single task is 30 s of 1080p at 2.86× ≈ 86 s.

flowchart LR
    A["Assemble chunks<br/>verify sha256"] --> B["Probe: codec, fps,<br/>HDR, rotation"]
    B --> C["Split into 30 s units<br/>on segment boundaries"]
    C --> D1["encode 240p ×24"]
    C --> D2["encode 360p ×24"]
    C --> D3["encode 480p ×24"]
    C --> D4["encode 720p ×24"]
    C --> D5["encode 1080p ×24"]
    B --> AU["Audio: AAC + Opus"]
    B --> TH["Storyboard frames"]
    D1 --> S["Stitch per rung<br/>verify GOP alignment"]
    D2 --> S
    D3 --> S
    D4 --> S
    D5 --> S
    AU --> S
    S --> P["Package CMAF<br/>+ manifest + thumbnails"]
    TH --> P
    P --> PUB["Publish"]

The serial head assembles the chunks, verifies the sha256 against what the client declared (catching a corrupted upload before it is encoded), then probes four facts: the arriving codec, the frames per second, whether it carries HDR (high dynamic range, an extended brightness/colour range that must be tone-mapped down for rungs that do not support it), and whether the phone recorded it sideways. Those facts set every downstream parameter, which is why nothing else may start first.

The fan-out splits the video into 30-second units on segment boundaries; each rung gets 24 independent encode tasks (5 × 24 = 120). Two side branches leave the probe directly because they do not depend on the video encode: audio is encoded twice (AAC and Opus) so every client has a format it supports, and storyboard frames are produced.

The fan-in stitches each rung’s 24 pieces back into one rendition and verifies GOP alignment. A GOP (group of pictures) is the run of frames between one full self-contained frame and the next; that self-contained frame is an IDR (instantaneous decoder refresh) frame, the only place a player can start decoding or switch quality. Alignment means every rung placed its IDR frames at the same instants. Then packaging runs, and only after it does the video publish.

Three properties the DAG buys, and one it costs:

• Every encode task is independently retryable. A lost machine costs 30 s of work, not 63 minutes, and the failure does not propagate.
• Rungs publish independently. If 1080p is late, the video goes live with the four that finished and the manifest is rewritten when the fifth lands.
• Work units align to segment boundaries (30 = 5 × 6), so parallel encoding forces zero extra keyframes. Choose 25-second units and every boundary is an unplanned IDR, pure bitrate waste.
• The cost: per-unit rate control cannot see across boundaries. Rate control is the encoder budgeting bits so the whole video lands near its target, spending more on fast scenes. Split across 24 machines, each budgets blind to the others. The fix is a two-pass encode: one pass measures how hard each part is, then a shared target is set before the second pass encodes. Without it you get roughly 2% of quality drift at the seams.

CPU or GPU, and why the transcode bill is not the bill

“Why not GPUs, they are much faster at video?” The answer turns on which bill you optimize.

Three terms. A fixed-function encoder is a silicon block implementing one codec directly, far faster and more power-efficient than software, but it cannot search harder (there is no loop to slow down), so its output is larger at the same quality. NVDEC is the corresponding hardware decoder, present on parts that have no encoder at all. VMAF (Video Multimethod Assessment Fusion) is the standard perceptual quality score; “15% more bits for the same VMAF” means the hardware encode needs a 15%-bigger file to look equally good.

Hardware video encode lives on T4 / L4 / L40S-class parts at about $1.00/GPU-hour, not the ~$2.00 (A100) or ~$2.50 (H100) training-accelerator rates quoted elsewhere on this site, because those parts decode in hardware and encode in software, so borrowing their rate pays for tensor cores this job never touches. A GPU doing the whole ladder at 12× realtime is 60 GPU-s per video.

Price both sides the same way or the answer swings 2×. Two defensible readings:

• (a) Both as owned fleets at 80% utilization, mean demand: 854 CPU boxes at $9.0 M/year against 435 GPUs at $3.8 M/year, a $5.2 M saving.
• (b) The 1,281-box CPU fleet actually provisioned, against GPU hours rented as consumed: $13.5 M against $3.0 M, a $10.4 M saving.

Now the other side of the trade. Fixed-function output is ~15% larger, and 15% of the $800 M/year egress bill is $120 M/year. Against the two readings that is 23× and 11.5× the compute saving. Pricing the encode part generously (at the low $1.00 rate) only flatters hardware, and it still loses; at the $2.00 training rate the ratios are 88× and 16.3×.

It has to lose, because the compute saving can never exceed the entire CPU bill, so the ratio can never fall below 120 M / 13.5 M ≈ 8.9: even a free encode part would leave the egress penalty nine times the whole thing it saves. That is the 306:1 read amplification at work: anything that trades output size for input cost loses by roughly that factor. Hardware encoding is right for live streaming, where the deadline is realtime and no slow preset is available; it is wrong for on-demand, where a file is encoded once and read a million times.

Deep dive 3: storage tiering, and generating the tail on demand

The view distribution is worth about a hundred million dollars a year in storage, if part of the saving is spent back on compute and the trade still pays.

Storage bills by the GB-month. The hot class here costs $0.023/GB-month and reads immediately; the cold class $0.004 and is slower to first byte. The baseline (every rung of every video in the hot class) is 500 M × 828 MB = 414 PB, which at $0.023 is $114 M/year. And 490 million of those videos average 0.4 views a day each. (This is the same tiering argument the estimation chapter makes for cold photos.)

Which rungs the tail keeps — and why it is not the popular ones

Head videos keep every rung hot. Tail videos keep only two rungs, cold, and any other rung a viewer asks for is generated at request time. Which two is not a popularity question:

Those two carry 41% of playback seconds, and they are deliberately not the two that carry the most (720p + 1080p, at 57%). The reason is a one-way property: encoding runs downhill only. From a stored 1080p you can generate 720p, 480p, and 240p, shrinking a picture only throws information away. From a stored 720p you can generate nothing above it. So the top rung is the only irreplaceable one, and 360p is a cheap floor so the commonest bandwidth-constrained request costs no compute. Read the table as a policy about derivability, not popularity: “keep the two most popular rungs” sounds right and picks the wrong pair.

Pricing the policy: head (10 M videos, full ladder, hot) is ~$190 k/month; tail (490 M videos, two rungs = 468 MB each, cold) is ~$917 k/month. Total ≈ $13 M/year, saving ~$101 M against all-hot.

What it costs back: just-in-time transcoding

The saving is only real if generating the missing rungs on demand costs less than storing them. JIT transcoding encodes a rung when a viewer asks for it, from a rendition you did keep, then caches the result in case another viewer wants the same rung of the same video. That last clause assumes reuse, and there is almost none.

About 59% of tail playback wants a rung the tail did not store (everything except 360p and 1080p). Weighted by pixel cost and normalized over the requests that need JIT, a missing rung averages ~0.33× a full 1080p encode. At a JIT preset ~5× faster than archival, plus one shared decode, that is ~0.29 core-s per encoded second.

The reuse question is the trap. A cached JIT result is one rung of one video, not a video, so a cached 480p does nothing for the next viewer who wants 720p. The right divisor is requests per (video, rung) pair over the 7-day origin TTL: 0.398 × rung-share × 7 = 0.95 for 720p and below that for everything else. Every one is under 1, so there is essentially no reuse and the honest divisor is 1.

With divisor 1, the tail’s ~34.5 B playback-seconds/day of missing rungs cost 34.5 B × 0.29 ≈ 10 B core-s/day = ~4,510 boxes, ~$47 M/year, leaving a net saving of ~$54 M, where the per-video divisor (2.79) would have wrongly claimed $84 M. There is no peak multiple on the JIT fleet, because it has a valve the transcode queue lacks: the failure-modes section rate-limits JIT per client and falls back to a stored rung, so a spike degrades quality instead of building a queue. Apply the same 1.5× if you would rather not lean on that valve (6,765 boxes, ~$71 M, net ~$30 M).

One number matters more than the result: the preset. Reuse the slow archival preset for JIT and the fleet jumps to ~16,291 boxes and ~$171 M/year, 70% more than the whole storage saving, turning a $54 M win into a $70 M loss. They are different jobs: an archival encode is read a million times, so its slow search is repaid a million times over; a JIT encode is read about once. The preset is a function of the expected read count and nothing else.

JIT is viable because of the segment. Transcoding a whole 12-minute rendition on the play path would be 720 × 0.29 ≈ 208 core-s, and no viewer waits three and a half core-minutes to start. But the media is already cut into 6-second segments, so the real unit of work is 6 × 0.29 ≈ 1.7 core-s, under half a second on four cores, inside a player’s startup buffer, and paid only for the segments actually watched.

Deep dive 4: CDN economics

Cache hit ratio here is computed, not assumed, and the number forces an extra tier into the design. A cache hit ratio is the fraction of requests a cache answers from its own copy; every miss is paid for upstream. A NIC is a network interface card, whose bandwidth limits how many bytes one machine can push.

Computing the edge hit ratio

A cached copy is worth keeping only if more than one request arrives for it before it expires. Spread the tail’s 195 M daily views across 50 PoPs and 490 M videos, over a 7-day TTL: one cached copy serves about 0.0557 requests, far under one. Essentially every tail request at an edge PoP is a miss, no matter how large the PoP or how long the TTL. The head is the opposite: 805 M views over 10 M videos across 50 PoPs is ~1.6 views per video per PoP per day, giving a hit ratio near one. Blended, the edge hit ratio is 0.805.

Sizing the origin from the miss rate

Origin therefore ships 110 × 0.195 = 21.45 PB/day = 1,986 Gbps. But that is a daily mean of offered load, and a fleet cannot be sized on a mean. Views peak at roughly 2× the average in prime time, and NICs are planned at 80% of rated speed, so the origin is 1,986 × 2 / 0.8 ≈ 4,965 machines doing nothing but feeding the CDN. The gap from 1,986 to 4,965 is exactly 2.5× (2× peak × 1/0.8 headroom); treating offered load as capacity always flatters the design. The tail is 19.5% of views and 100% of the origin problem.

Why a bigger edge cache buys nothing

In the URL-shortener chapter, more cache memory bought hit rate, because misses there were capacity misses: the object had been requested before and evicted. Here more memory buys nothing, because the miss is a first-access miss: the object was never requested at that PoP at all, and no cache can hold what nobody asked for.

The fix is not a bigger cache but fewer independent caches, so each sees more traffic and the first access happens sooner. With 4 regional shields at a 30-day TTL, one copy serves 195M/4/490M × 30 ≈ 2.99 requests per video, a 66.6% tail hit ratio, cutting origin transfer to ~7.16 PB/day and saving about $52 M/year in origin-to-CDN transfer at $0.01/GB.

What a cached copy actually is

Read that $52 M as an optimistic bound, because 2.99 is per video and a cached object is one segment of one rung. Multiplying by each rung’s playback share: 720p clears 2.99 × 0.34 = 1.02 requests per copy and every other rung is below 1, so the blended tail hit ratio is about half a percent, not 66.6%, and the $52 M is closer to $0.4 M.

The two answers are far apart because the design sits on a knife edge. Requests per copy scale as 0.398 × (TTL / shields) × rung-share, so clearing one request per copy on the largest rung needs TTL / shields ≥ 1 / (0.398 × 0.34) ≈ 7.4 days. This design runs at 30 / 4 = 7.5, right on break-even. The lever is the real finding: reuse scales as TTL / shields, so the tier that pays is fewer shields or a much longer TTL, not more shields. A single shield at 30 days clears ~4.1 requests per copy on 720p (a 76% hit ratio); four shields would need a 120-day TTL to match it.

The shield still earns its place on the head, through request coalescing on a viral object. Two more consequences of the same distribution:

• Prefetch the head. A large channel’s video is predictably about to be watched everywhere, so pushing it to all 50 PoPs at publish costs 828 MB × 50 ≈ 41 GB and removes the cold-start miss on exactly the videos that generate the most traffic.
• A viral video hits the origin as a step function. Between publication and the first edge fill, every PoP misses independently and asks at once, the caching form of the hot key problem, where one object is so much more popular that spreading data across machines cannot help (the consistent-hashing chapter proves partitioning cannot fix it). The guard is request coalescing at the shield: many simultaneous misses for the same object trigger exactly one fetch to origin, and the rest wait on that single result, so 50 misses become one.

Deep dive 5: adaptive bitrate, where the server does almost nothing

Adaptive bitrate (ABR) publishes the video at several qualities at once, cut into aligned segments, and lets the player re-choose a quality before every segment, so a viewer whose train enters a tunnel drops from 1080p to 360p and keeps playing instead of freezing. The interesting decision lives on the client; the one number the server owns is segment length.

The client picks the rendition and the server merely publishes a list, because of where the deciding information lives and how fast it moves. The client’s control loop runs once per 6-second segment. Its inputs change on a ~100 ms scale: buffer occupancy (seconds of video already downloaded but not shown, the cushion that absorbs a dip, and the single most informative variable), the player’s own throughput estimate, and viewport size (how many pixels the video is actually shown in, which is why sending 1080p to a thumbnail-sized player is pure waste). Shipping that state to a server costs a ~150 ms round trip plus ≥1 s of telemetry batching. A server-side decider would be at least two control periods behind, and blind to the two variables that matter most. Put the controller where the state is. The manifest is ~2 KB, static, and identical for every viewer, so it caches at the edge like any other object.

The one thing the server must get right

Every rung must place its keyframes at the same presentation timestamps (the timeline positions where frames are meant to be shown), so segment n of the 480p rendition and segment n of the 1080p rendition cover the same six seconds. Without that, a mid-stream switch produces a visible gap or a repeated frame. This is why the transcoder forces IDR frames onto a fixed grid instead of onto scene cuts, and it costs real bitrate, because a forced keyframe in the middle of a static shot is bits spent on nothing.

That cost turns segment duration into an economic decision. A keyframe encodes a whole picture from scratch; an inter frame encodes only what changed and is roughly ten times cheaper in bits. With a forced keyframe at every boundary and 30 fps, the mean frame cost with a GOP of G frames is (10 + G − 1) / G: 1.050 at 6-second segments (G = 180) versus 1.150 at 2-second (G = 60). Two-second segments cost about 9.5% more bits, $76 M a year on the egress bill, and triple the request count (360 segments per rendition instead of 120), to buy a switching latency nobody watching a recorded video can perceive. Ship 6 seconds for on-demand. Ship 2 seconds or less for live, where segment duration is a floor on end-to-end latency and the economics invert.

Dedup and copyright, which are different problems

Deduplication (dedup) means noticing an uploaded file is byte-for-byte identical to one you already store and keeping one copy. Copyright matching means noticing an upload contains the same content as a protected work even though not one byte agrees, because it was re-encoded, cropped, or re-recorded off a screen. The first is a hash lookup and nearly free; the second needs a different technique, a different latency budget, and a different attitude to being wrong.

Exact dedup: cheap, three ways to get it wrong

Assume 3% of uploads are byte-identical re-uploads (15,000/day). That avoids ~12.4 TB/day (4.5 PB/year) of storage and about 38 boxes off the 1,281-box transcode fleet. Worth doing, with three constraints:

• Reference-count the stored bytes. Once two video_ids point at one object, deleting a video must decrement a counter, not unlink the object, or the second uploader’s video disappears when the first deletes theirs.
• Never let dedup change the observable timing of an upload. If a duplicate completes instantly, anyone can test whether a specific file already exists by uploading a copy and watching the clock, a membership oracle, and against private videos a privacy breach. Run the full upload every time and deduplicate on the storage side only.
• This catches literal re-uploads and nothing else. A re-encode produces a different byte string, so exact hashing has zero recall (the share of true matches found) against what people usually mean by “duplicate”.

Copyright matching, where the error budget is a headcount

That case needs perceptual fingerprints: compact summaries of what the audio and video look and sound like, designed so two encodes of the same content produce nearly the same fingerprint. A pHash (perceptual hash) is the video half: a short code from a frame’s coarse structure that survives re-encoding.

Assume 100 M reference assets, 5-minute mean, fingerprinted at 40 B/s (32 B audio hash + 8 B video pHash). The index is ~1.2 TB answering ~3,600 lookups/s, small by the standards of everything else here, so retrieval is not the hard part.

The error budget is the hard part, and it is set by a headcount, not a model. Precision is the share of raised claims that are correct. At a 1% false-positive rate you generate 5,000 wrongful claims a day and need ~83 reviewers (at 60 appeals per reviewer per day); at 0.1% you generate 500 and need ~9. The threshold is read off the staffing plan, not a validation curve. Three separations follow:

• Copyright matching must not gate publication. A reference scan takes minutes to hours; a publish takes 86 seconds. Publish first and claim later, gating only accounts with a history of claims.
• It needs a different retry policy from the transcode DAG. A failed encode simply retries. A failed match must never silently pass, because “we scanned and found no match” and “the scan did not run” have very different legal meanings.
• Its false positives are a product decision and its false negatives (missed infringement) are a legal one. Those errors have different owners, so the thresholds belong in configuration either owner can move, not baked into the model.

Bottlenecks and scaling

The surprising one is the origin NIC. A correct CDN design still needs about five thousand machines just to feed it, because the long tail is a first-access miss and no amount of edge memory fixes a first access. Note how the number is built: mean offered load, then a peak factor, then a utilization target, then a box count. Skip the middle two steps and you under-buy by 2.5×.

Failure modes

Almost every guard here is a design decision made in advance, not a runbook step taken during the incident.

Alternatives rejected

Several are not wrong so much as right for a different product.

• A single POST with the whole file. One endpoint, no ledger, no resume logic. Rejected: it succeeds only 10% of the time on a 5 Mbps link dropping once per 500 s, and retrying from zero transfers an expected 3.9× the file. Chunking is the difference between working and not working.
• GPU or fixed-function encoding for the main ladder. $5.2 M–$10.4 M/year cheaper and far lower wall clock. Rejected: hardware encoders need ~15% more bits for the same quality, and 15% of $800 M is $120 M, 11.5×–23× the compute saving, 8.9× even if the encoders were free. Correct for live streaming, where realtime is a hard deadline.
• Full ladder hot for every video. No JIT fleet, no promotion logic, uniform latency. Rejected at $114 M/year against $13 M, because 490 M videos average 0.4 views/day. JIT costs $47 M of that back (net $54 M), and only if JIT uses a fast preset; the archival preset exceeds the whole storage saving by 70%.
• Server-side bitrate selection. One consistent, client-independent policy. Rejected: the deciding state (buffer, throughput, display size) lives on the client and moves faster than a 150 ms round trip plus telemetry can track. The server would be two control periods behind and blind to the two most important variables.
• Two-second segments for on-demand. Adapt and start faster. Rejected on bits: 3× the keyframes is a 9.5% penalty, $76 M/year, for switching latency nobody watching a recording perceives. Right for live, where segment duration sets a floor on glass-to-glass latency (camera lens to viewer’s screen).
• One database row per media segment. Queryable, a home for per-segment metadata. Rejected at 300 M rows/day of pure naming, all determined by duration and rung. Address segments by convention.
• Perceptual dedup as a storage mechanism. It catches re-encodes, which exact hashing misses. Rejected: two perceptually identical files may have different rights holders and edits, so collapsing them into one stored object is a legal problem, not a storage saving. The perceptual pipeline exists, but its output is a claim, not a pointer.

The assumption ledger

Every design is a set of assumptions, and the diagram is only correct relative to them. Sort each into three bins. State it: you may pick a number, and being wrong costs only a re-derivation. Ask it: the answer changes a policy or threshold. Load-bearing: if it is wrong the design is invalid, a box appears or disappears, not just the count inside it. The test is to move the assumption an order of magnitude each way and ask whether the set of boxes changes.

Conclusion

The whole design falls out of one measured ratio: 306 bytes read for every byte written.

• It is a CDN with an upload pipeline attached. Egress dominates the bill ($800 M/year), so the only lever that matters at scale is codec efficiency, which is why software encoding beats hardware despite being slower and pricier in compute, and why the transcode cluster, the most compute-hungry component, is not the constraint.
• The write path is built for a link that drops. Chunked, resumable upload at 8 MB turns a 10% single-request success rate into 1.3% wasted bytes, and a 120-way DAG turns 63 minutes of serial encode into an 86-second publish.
• Storage and delivery are governed by the Zipf tail. Head videos stay hot; the tail keeps only the irreplaceable top rung plus a cheap floor and generates the rest just-in-time, but only with a fast preset, and only because the media is already cut into cheap 6-second segments. The tail is also a first-access miss, so no cache fixes it; the origin still needs ~5,000 NIC-bound machines, and shield reuse lives or dies on TTL / shields.
• The client owns adaptation; the server owns alignment. The player picks the quality; the server only guarantees keyframes line up across rungs and ships 6-second segments.

One line to remember: at 306 bytes read per byte written, this is a delivery network with an upload pipeline bolted on, so every dollar and every design choice bends toward moving fewer bytes to viewers, never toward saving compute.

Further reading

• RFC 8216, HTTP Live Streaming (HLS): the manifest and segment format sketched in the API section.
• ISO/IEC 23009-1, MPEG-DASH: the other major adaptive-streaming standard, and why CMAF exists to serve both from one set of files.
• Netflix, “Toward a Practical Perceptual Video Quality Metric” (VMAF): the perceptual score behind the “15% more bits” comparison.
• Huang et al., “A Buffer-Based Approach to Rate Adaptation,” SIGCOMM 2014: why buffer occupancy drives the client’s ABR decision.
• Spiteri, Urgaonkar, Sitaraman, “BOLA: Near-Optimal Bitrate Adaptation for Online Videos,” INFOCOM 2016: a formal client-side ABR controller.
• Related chapters on this site: the egress estimate extended here is in the estimation chapter; the consistent-hashing chapter partitions the metadata and explains the viral-video hot key; video search owns retrieval over this corpus and the video recommender owns the feed.