14 — Design YouTube

“Users upload video. Other users watch it. Build that.”

This is the infrastructure design for a video-on-demand service at YouTube scale, derived end to end from one measured ratio: the service ships roughly three hundred bytes to viewers for every one byte a creator uploads.

By the end you will be able to size four things from first principles:

1. the upload path, which has to survive a phone losing signal;
2. the compute that converts one uploaded file into five streamable qualities;
3. the storage that keeps those qualities;
4. the network that delivers them.

You will also be able to say, with arithmetic rather than adjectives, why the delivery network is the design and the encoding cluster is a rounding error beside it. The test that matters: close the chapter and explain to someone else why a 10% improvement in video compression is worth more than the entire compute budget.

Two words to fix before anything else

Video vocabulary is the main thing that makes this chapter look harder than it is. Two terms carry most of the weight, and both appear in the next paragraph.

A codec is a matched pair of algorithms — a coder and a decoder — that squeeze video into a compact stream of bytes and expand it back into pictures. H.264 and AV1 are codecs. Different codecs hit the same picture quality with different numbers of bytes, and the whole economic argument of this chapter turns on that difference.

A bitrate is how many bits of compressed stream one second of playback costs, written in megabits per second (Mbps). A 12-minute video at 4.5 Mbps is 4.5 million bits for each of its 720 seconds. Higher bitrate means a better picture and more bytes to ship, and “more bytes to ship” is the only cost that matters at this scale.

The concrete input and the concrete output, before any mechanism

In goes one file: a 12-minute recording of about 720 MB, pushed over a link that will drop mid-transfer.

Out comes a set of renditions — the same video re-encoded at several picture qualities, from a 240-pixel-tall postage stamp up to full 1080p high definition. Each rendition is cut into six-second segments, meaning small independently downloadable pieces of video that a player can fetch one at a time.

Alongside the segments sits a manifest: a text file of roughly 2 KB that names every available quality and the URL of every segment. A viewer’s player downloads the manifest first, then pulls segments in order, re-deciding which quality it can afford before each one.

Everything in this chapter is one of four jobs: getting the file in, converting it into that set, storing the set, or shipping segments out.

Why the ratio decides everything

You read 306 bytes for every byte you write. That makes this a content delivery network (CDN) — a fleet of caching servers placed close to viewers so most requests are answered near the user instead of from your own datacenter — with an upload pipeline bolted to the side.

Everything expensive is downstream of that ratio. The egress bill, where egress means bytes leaving your network toward users, which is what a cloud provider actually charges for. The codec team. The storage tiering. And the fact that the origin — your own authoritative copy of the bytes, the thing the caches fall back to — serves almost nothing.

Everything hard is upstream of it. An upload that has to survive a mobile link, and a transcode fan-out that turns one file into five. Transcoding is decoding a video back into raw pixels and re-encoding it at a different resolution, bitrate or codec. It is the single most compute-hungry operation in the system.

Scope, and where candidates lose it

Half the answer people give to “design YouTube” is not in this chapter, so the boundary is worth stating at the top.

Search over this corpus — automatic speech recognition (ASR), transcripts, multimodal retrieval — is ml/04. The home feed, candidate generation and ranking, is ml/06. Neither is re-derived here, and neither should be re-derived in an interview when the question is infrastructure. This chapter owns the bytes: getting them in, transforming them, storing them, getting them out.

The four ways candidates lose this question: a single POST /upload with no resume story; “we transcode it” offered as a whole answer; keeping every rendition of every video permanently in fast storage; and describing adaptive bitrate as something the server does. Adaptive bitrate, abbreviated ABR, is the mechanism by which playback quality changes mid-video as the network improves or degrades — and Deep dive 5 adaptive bitrate where the server does almost nothing shows it is almost entirely a client behaviour.

The estimation technique used throughout, and the egress figure this chapter extends, are in chapter 02. Nothing in that chapter is required reading first: every number borrowed from it is restated here where it is used.

1. Framing: what decision, and what breaks

One number holds up the rest of the chapter — how many bytes go out for every byte that comes in — and it carries four consequences you can state before you have drawn a single box.

Three terms first, because they appear in every block below.

A petabyte (PB) is a million gigabytes. It is the natural unit for a day’s traffic here.

Gigabits per second (Gbps) is a rate, not a quantity, and converting between the two is the first line of arithmetic below. One petabyte spread evenly across a day is 92.6 Gbps: a petabyte is 10^15 bytes, a byte is 8 bits, and a day is 86,400 seconds, so 10^15 x 8 / 86,400 bits per second is 92.6 billion of them. Every “PB/day to Gbps” step in this chapter is a multiplication by that 92.6.

The mezzanine is the industry name for the master file the creator uploaded — high quality, large, never served to a viewer, kept only as the source from which every viewable rendition is generated.

The arithmetic runs in four moves — PB/day to Gbps, the traffic assumptions, one uploaded video, a day’s ingest — and ends by dividing the day’s outbound bytes by its inbound bytes.

1 PB/day expressed in Gbps, used throughout
  1,000,000,000,000,000 x 8 / 86,400 / 1,000,000,000  =  92.6

assume  500 k uploads/day, mean 12 minutes, 8 Mbps mezzanine
        (same corpus and upload rate as ml/04 and ml/06)

bytes per uploaded video, MB
  720 x 8 / 8                            =  720
ingest, TB/day
  500,000 x 720 / 1,000,000              =  360
ingest, PB/day
  360 / 1,000                            =  0.36
ingest, Gbps
  0.36 x 92.6                            =  33.3

egress, PB/day, from ch 02 estimation 4                  110
read amplification
  110 / 0.36                             =  306

The read-to-write ratio of this system is three hundred and six to one. Say that number out loud in the first two minutes of the interview, because it reorders everything that follows.

Here are those four consequences, each with the arithmetic that forces it.

What actually breaks

Four failures, roughly in order of how often you will see them.

An upload dies at 80% on a train and the client starts over from zero.

A transcode job stalls on one rendition and the video publishes with a hole in its bitrate ladder. The ladder is the fixed set of quality levels — each with its own resolution and its own bitrate target — that the encoder produces for every video. A hole in it means one of those quality levels is simply missing.

A video goes viral before the caching network has a copy, and the origin absorbs a two-terabit-per-second step change in load.

A copyright claim fires on a false positive — the matcher says “this is someone else’s content” when it is not — and a creator is demonetized by a pipeline nobody can explain.

2. Requirements

What the system must do takes ninety seconds to state; the seven measurable promises underneath it are where the design is actually decided.

Functional

The capabilities the product must have, none of them controversial:

• Upload a video from a browser or a phone, over a link that will drop.
• Transcode into a rendition ladder, package it for adaptive streaming, and publish it. Packaging is the step after encoding that cuts each rendition into segments and writes the manifest describing them.
• Stream with adaptive bitrate, so the picture must not stall when the network dips.
• Produce thumbnails (the still image shown before playback), storyboards (a grid of small frames the player shows while you scrub the timeline), captions, and a per-video manifest.
• Detect duplicate uploads, and match every upload against a copyright reference set.
• Out of scope, said explicitly: search (ml/04), recommendation (ml/06), comments, live streaming, monetization.

Non-functional — the rows that decide the design

Each row below is a promise with a number attached, and the third column says where the number comes from rather than asserting it. The two rows that decide the architecture are the last two.

Two conventions used in it. p50 means the median: half of all cases are faster than the stated number, and p99 would mean the slowest one percent begin there. An SLA, or service level agreement, is the availability figure you contractually promise — 99.99% permits about 53 minutes of downtime a year, 99.9% permits about 8.8 hours.

The asymmetry between the two availability rows is deliberate, and it is the single most useful thing in the table.

Playback must be available 99.99% of the time because a viewer who gets an error simply leaves. Uploads can be a full order of magnitude less available — 99.9% — because a failed upload is retried, usually by a human who is still sitting on the page watching a progress bar.

That gap licenses a split you will see in every read-heavy design: a fat read path served from caches at the network edge — the caching machines physically nearest the viewer — and a thin write path that accepts work into a queue and answers immediately.

Chapter 08 reaches the same split for URL shortening, where the read path is a single key lookup. The difference here is that this read path moves 110 petabytes a day.

3. Back of the envelope

Two quantities get spent by every later section: how many bytes one video costs to store once it has been encoded at every quality, and how unevenly views are spread across the catalogue.

What one video costs to store

A rung is one step of the bitrate ladder — one resolution at one bitrate, such as “720p at 2.5 megabits per second”.

The block below converts each rung’s bitrate into megabytes for a 12-minute video. The recipe is the same on every line: multiply the bitrate in Mbps by the 720-second duration to get megabits, then divide by 8 to get megabytes. So 720p at 2.5 Mbps is 720 x 2.5 = 1,800 megabits, which is 1,800 / 8 = 225 MB. The total on the full ladder line, set against the 720 MB source, is the point.

bytes per 12-minute video by rendition, in MB (720 s x Mbps / 8)
  240p  :   720 x 0.3 / 8                =     27
  360p  :   720 x 0.7 / 8                =     63
  480p  :   720 x 1.2 / 8                =    108
  720p  :   720 x 2.5 / 8                =    225
  1080p :   720 x 4.5 / 8                =    405
full ladder
  27 + 63 + 108 + 225 + 405              =    828

renditions written per day, TB
  500,000 x 828 / 1,000,000              =  414
mezzanine retained 90 days for future re-encodes, PB
  360 x 90 / 1,000                       =  32.4

414 TB/day of renditions against 360 TB/day of source: the ladder costs more than the original. Every rung you add is a permanent storage commitment, not a compute one.

How views spread across the catalogue

The second quantity is how views distribute across videos, and the standard model for that is Zipf’s law: rank every video by popularity, and the view count of the video at rank r falls off roughly as 1/r. The video at rank 100 gets about a hundredth of the views of the video at rank 1.

With the exponent s = 1 used here, that 1/r shape has a convenient consequence. Total views across K videos are proportional to 1 + 1/2 + 1/3 + ... + 1/K, and that sum is very close to ln(K). The same sum stopped at m is close to ln(m). So the share of all views captured by the top m videos out of K is just ln(m) / ln(K), a ratio of two natural logarithms — no other input needed.

That formula, and nothing else, produces the split below: what fraction of views the top 10 million of 500 million videos get, and what is left over for everybody else.

Zipf s = 1 over K = 500 M videos: the top m cover ln(m) / ln(K)
  ln 500,000,000 = 20.03 and ln 10,000,000 = 16.12
  16.12 / 20.03                          =  0.805

tail views/day, of ch 02's 1 B views/day
  1,000,000,000 x 0.195                  =  195,000,000
per tail video, per day
  195,000,000 / 490,000,000              =  0.398

The top 10 million videos — 2% of the corpus — take 80.5% of views, and the other 490 million average four tenths of a view a day each. That last number drives Deep dive 3 storage tiering and generating the tail on demand and Deep dive 4 cdn economics extended: it is why the long tail of unpopular videos cannot usefully be cached near viewers, and why keeping its full ladder in fast storage is paying rent on bytes nobody reads. Throughout, head means those top 10 million videos and tail means the other 490 million.

4. API sketch

Three of the design’s most important decisions are visible in the request and response shapes alone. The sketch is a sequence: the client announces the upload, pushes it in pieces, says it is finished, and later a player fetches the manifest and then segments.

A few notations, glossed once.

POST creates something, PUT writes a specific named thing, and GET reads. The three-digit numbers are HTTP status codes: 201 means created, 202 means accepted-but-not-finished, 204 means done with nothing to return, and 409 means conflict — the server disagrees with the client’s idea of the current state.

sha256 is a cryptographic hash. Run it over a byte string and you get a fixed 32-byte fingerprint that is, for practical purposes, unique to that exact content.

The ingest_url returned by the first call is a presigned upload URL — a temporary, signed, single-purpose address that authorizes exactly this upload to exactly this storage location. It lets the client write bytes directly to storage without the storage system ever seeing the user’s credentials.

master.m3u8 is a manifest in the widely used HLS (HTTP Live Streaming) text format, and .m4s is one media segment file.

POST /v1/uploads
  {"bytes": 720000000, "sha256": "...", "duration_s": 720, "title": "..."}
  201 {"upload_id": "u_9f3", "chunk_size": 8000000,
       "ingest_url": "https://ingest-eu.../u_9f3"}

PUT  /v1/uploads/u_9f3/chunks/{n}
  Content-Range: bytes 40000000-47999999/720000000
  204  committed
  409  {"committed_through": 39999999}      <- the resume point

POST /v1/uploads/u_9f3/complete
  202 {"video_id": "v_7Kq", "state": "transcoding", "ladder": []}

GET  /v1/videos/v_7Kq/master.m3u8           the ABR manifest, ~2 KB, cacheable
GET  /cdn/v_7Kq/720p/seg00042.m4s           a 6-second segment

Four deliberate choices are worth defending, because each one is a place candidates get it wrong.

• The byte literals are decimal, and they match the derivations exactly. 720000000 and 8000000 are the same 720 MB and 8 MB used everywhere in this chapter, where MB means 10^6 bytes. The binary neighbours — 754974720 for 720 MiB and 8388608 for 8 MiB — divide to the same 90 chunks, which is exactly why mixing them survives review and then bites: the sketch is the one place a reader copies literal integers into code, and a client that sizes its buffers in MiB against a server counting MB disagrees about the last chunk’s length. Pick one base, and pick the one the arithmetic above used.
• The server dictates chunk_size; the client does not choose it. Chunking here means splitting the file into fixed-size pieces and uploading them as separate requests, so that a dropped connection costs one piece rather than the whole file. Deep dive 1 resumable upload and where 8 mb comes from derives the size from the link’s failure rate, and the server can vary it per network type — a phone on cellular gets a different number from a desktop on fibre. Let clients pick and the worst clients pick worst.
• The 409 response carries the resume point rather than requiring a separate probe. The failure case is a client that reconnected and does not know how far it got; making it ask twice doubles round trips in exactly the situation where round trips are unreliable.
• complete returns 202 with an empty ladder. Publication is asynchronous by construction — Deep dive 2 transcoding as a dag shows 86 seconds of wall clock at best — and an API that pretends otherwise forces every client to hold a connection open through it.

5. Data model

The data model is a question of what goes in a database and, more importantly, what does not.

The dividing line is this. The bytes of the video live in object storage — a service that stores arbitrarily large blobs under a string name called an object key, with no query language, no joins, and effectively unlimited capacity. Amazon S3 is the familiar example. The small mutable facts about a video live in a conventional database.

The sketch below is three real tables and one deliberate absence: videos holds one row per video, renditions one row per quality level, uploads the bookkeeping that makes a resume possible, and segments is written in as a comment precisely because it is not a table.

videos
  video_id BIGINT PK, owner_id, duration_ms, state,
  mezzanine_key TEXT,          -- object key, deleted after 90 days
  content_hash  BYTES(32)      -- sha256 of the uploaded bytes, for dedup
                               -- (dedup: storing one copy when two uploads
                               --  are byte-identical. Priced in section 12)
  -- state: uploading | transcoding | live | blocked

renditions                     -- one row per (video, rung), written by the DAG
  video_id, rung, state, bitrate, key, segment_count

uploads                        -- the resume ledger, hot for hours then cold
  upload_id PK, video_id, total_bytes, committed_through, chunk_hashes[]

segments                       -- NOT a table. Object keys by convention:
                               -- {video}/{rung}/seg{n}.m4s

What is deliberately not in a database is the segments. A 12-minute video at five rungs is 720 / 6 x 5 = 600 objects, and at 500 k uploads a day that is 300 million rows a day of pure naming. Address them by convention, so the manifest is generated arithmetically from duration and rung and the storage layer never gets a row per object.

Every query left over is a point lookup — fetch exactly one row by its identifier, never a range scan and never a join across tables. That is the exact shape a key-value store is built for.

Spread that store across machines with consistent hashing: hash every video_id onto a circle, hash every server onto the same circle, and let each key belong to the first server clockwise from it. Adding a server then relocates only the keys sitting between it and its neighbour, rather than reshuffling every key in the system.

Chapter 06 builds the store and chapter 05 builds the ring. The paragraph above is all this chapter needs from either.

The exception is the uploads table, written 45 million times a day (Deep dive 1 resumable upload and where 8 mb comes from) and read almost never afterwards. It wants a short TTL — a time-to-live, meaning the store deletes each row automatically once it reaches a set age — and it wants to live in a different store from videos, so that a flood of upload bookkeeping cannot slow down the table playback depends on.

6. High-level architecture

Every component fits on one page as two independent paths that meet only in storage: a write path running down the left, from a creator’s client to a published video, and a read path running along the bottom, from a viewer back to whichever cache still has the bytes.

flowchart TD
    C["Client<br/>8 MB chunks · resumable"] --> IN["Ingest service<br/>regional, near the uploader"]
    IN --> OBJ[("Object store<br/>mezzanine")]
    IN --> LED[("Upload ledger<br/>committed_through")]
    IN -->|"complete"| Q[["Transcode queue<br/>priority by creator tier"]]

    Q --> DAG["Transcode DAG<br/>section 8"]
    DAG --> REND[("Renditions<br/>hot + cold classes")]
    DAG --> PKG["Package: CMAF segments,<br/>manifest, thumbnails"]
    PKG --> PUB["Publish: state = live"]

    OBJ -.-> FP["Fingerprint + copyright<br/>separate pipeline, hours"]
    OBJ -.-> ASR["ASR / captions<br/>ml/04"]
    OBJ -.-> EMB["Item embeddings<br/>ml/06"]

    V["Viewer"] --> EDGE["Edge PoP<br/>50 locations"]
    EDGE -->|"miss"| SH["Regional shield<br/>4 locations"]
    SH -->|"miss"| ORG["Origin + JIT transcoder"]
    ORG --> REND

    style OBJ fill:#1d3557,color:#fff
    style EDGE fill:#2d6a4f,color:#fff
    style SH fill:#2d6a4f,color:#fff
    style ORG fill:#bc6c25,color:#fff
    style FP fill:#9d0208,color:#fff

Reading the colours

The colours are ch 01’s published key, and in a chapter about bytes they are worth reading first.

Blue is the authoritative copy of the data. Here that is the mezzanine in the object store. Every rendition, segment, manifest and thumbnail below it is derived and can be rebuilt from it — which is the whole reason Deep dive 3 storage tiering and generating the tail on demand is allowed to throw rungs away.

Green is read capacity: the edge PoPs and the regional shields answer reads without asking the authority.

Orange is a box forced by something other than processor time. That is the origin — 4,965 machines of it in Deep dive 4 cdn economics extended, sized by network card and not by core.

Red is the one path whose mistakes cannot be taken back: a copyright claim against a creator.

The transcode DAG is deliberately uncoloured despite being by far the most processor-hungry box on the page. The key has no colour for processor time, and the fact that the single hungriest component in the chapter does not qualify for one is the chapter’s thesis stated in a legend.

The write path

The client chunks the file into 8 MB pieces and uploads them resumably, meaning it can stop and restart at a chunk boundary without losing what it already sent.

Those chunks land at an ingest service that is regional — physically near the uploader — so that the many small commit round trips do not each cross an ocean.

The ingest service does two things with them. It writes the assembled bytes into the object store as the mezzanine master. And it maintains an upload ledger, a tiny durable record whose only important field is committed_through: the byte offset the server can prove it has.

When the client calls complete, the ingest service places a job on a transcode queue ordered by creator tier, so a large channel’s video is not stuck behind a backlog of hobbyist uploads.

That job runs on the transcode DAG — a directed acyclic graph, meaning a set of tasks with arrows showing which must finish before which, and no cycles. Deep dive 2 transcoding as a dag derives it.

The DAG’s output goes two places. The renditions go to storage split into hot and cold classes: hot meaning fast, expensive, immediately readable storage, cold meaning cheaper storage with worse latency. Deep dive 3 storage tiering and generating the tail on demand turns that one distinction into a hundred million dollars a year.

In parallel the step drawn as Package: CMAF segments, manifest, thumbnails produces exactly those three outputs. CMAF, the Common Media Application Format, is the standard container that lets one set of segment files serve both of the common streaming protocols, instead of forcing you to store two copies of everything.

Only when packaging finishes does the last node, Publish: state = live, flip the video’s row from transcoding to live and make it visible.

The three dotted branches

Three pipelines hang off the mezzanine and none of them is allowed to delay publication. That is why they are drawn with dotted arrows.

Fingerprint and copyright matching is a separate pipeline whose latency budget is hours rather than seconds (Dedup and copyright which are different problems).

The branch drawn as ASR / captions ml/04 runs automatic speech recognition, transcribing the spoken audio into text, and feeds the search system in ml/04.

Item embeddings — fixed-length numeric vectors that summarize a video so that similar videos land near each other — feed the recommender in ml/06.

None of the three is designed in this chapter.

The read path, which is three cache layers deep

A viewer’s player asks an edge PoP: a point of presence, one of roughly 50 small clusters of cache servers placed in major metropolitan areas.

On a miss — meaning the requested segment is not in that cache — the edge asks a regional shield. A shield is a second, larger cache tier that sits behind many edges and that all of them share, so an object fetched once on behalf of one edge is already present for the next. There are only 4 of them, and Deep dive 4 cdn economics extended shows why that count is the whole argument for the tier.

On a further miss the shield asks the origin, which is two things at once: the authoritative store of renditions, and a just-in-time (JIT) transcoder — a service that can generate a missing quality level on the spot rather than reading one that was stored in advance.

The five claims this diagram owes you

Five things are asserted in that picture without proof, and each gets its own section to earn it: the upload is chunked and resumable (Deep dive 1 resumable upload and where 8 mb comes from), transcoding is a fan-out DAG (Deep dive 2 transcoding as a dag), storage is tiered and part of the ladder is generated on demand (Deep dive 3 storage tiering and generating the tail on demand), the shield tier exists because the tail cannot be cached at an edge (Deep dive 4 cdn economics extended), and copyright matching must not gate publication (Dedup and copyright which are different problems).

7. Deep dive 1: resumable upload, and where 8 MB comes from

The upload chunk size is derived from the physics of a mobile connection rather than picked as a round number, and the derivation lands on a closed form you can reproduce under questioning. Before it, one piece of vocabulary hygiene.

Three different chunkings live in this system and conflating them wastes an hour of interview time. State them once, at the start:

They are independent, and the only constraint between them is that the transcode work unit must be a whole number of media segments — 30 = 5 x 6 — so splitting the encode across machines forces no keyframe the ladder did not already need.

Why a single request does not work

The argument for chunking is not “large files are large” — it is that the probability of a long transfer completing at all falls exponentially with its duration. Two terms carry that argument. Goodput is the useful application-level throughput actually achieved, after protocol overhead and retransmission, as opposed to the headline link speed. A hazard rate is the probability per unit time that a connection dies given that it is still alive; if that rate is constant at p per second, the chance of surviving t seconds is exp(-p x t), which is why a transfer twice as long is far worse than twice as risky.

assume  a mobile uplink at 5 Mbps effective goodput, and a connection that
        drops with hazard p = 0.002 per second, i.e. one drop per 500 s

goodput, MB/s
  5 / 8                                  =  0.625
seconds to push a 720 MB file
  720 / 0.625                            =  1,152
expected drops in that window
  1,152 x 0.002                          =  2.30
probability of finishing in one request, exp(-2.30)
  0.100

A single-request upload of a 12-minute video over a real phone connection succeeds one time in ten, and the retry does not help because a retry starts from zero.

Costing that retry is where the obvious model goes wrong. The tempting shortcut is “a failed attempt gets about halfway, so each one wastes half a file”. That is wrong here, and the block below does it properly.

The reason is a conditioning effect. You are not asking “when does a connection die?” — you are asking “when does it die given that it died before the transfer finished?” Those are different questions. Deaths are spread exponentially over all time, so most of them happen early; restricting attention to the ones that happen before 1,152 seconds pulls the average death time earlier still. E[T | T < L] is the notation for that: the expected death time T, conditioned on T landing before the deadline L. The share comes out at 32.3%, not 50%, and the expected bytes move with it.

expected attempts, 1 / exp(-2.30)                        10
how far a FAILED attempt gets is not half the file: it is the
mean of an exponential CONDITIONED on dying before 1,152 s
  E[T | T < L] = (1/p)(1 - e^-pL) - L e^-pL, over (1 - e^-pL)
  numerator    = 500 x 0.900 - 1,152 x 0.0999   =  335
  denominator  = 0.900
  E[T | T < L]                           =  372 s
as a share of the file
  372 / 1,152                            =  0.323
expected bytes, 9 failures at 0.323 plus one full success
  9 x 0.323 + 1                          =  3.9

Expected bytes transferred are 3.9x the file, not 5.5x.

The /2 that gives 5.5 is the right approximation for a chunk and the wrong one for a whole file. A failure is spread evenly over the attempt only when the attempt is short compared with the mean time between drops, 1/p = 500 s. A chunk satisfies that easily — p x t = 0.0256, so it holds to three decimal places. A 1,152-second file does not.

The chunked arithmetic further down is unaffected; only the single-request comparison moves. Either way, chunking is not optional.

Where the size comes from

The chunk size is the solution to a two-term optimization, and both terms need naming before the algebra.

Small chunks pay a fixed per-chunk cost. Request framing, a durable commit of the offset, the server’s metadata write. Call that h; it totals about 150 milliseconds per chunk. Halve the chunk size and you double the number of chunks, so this cost scales as 1/S where S is the chunk size.

Large chunks pay retry waste. A connection drop discards whatever chunk was in flight, so the bigger the chunk the more you lose each time. This cost scales as S.

A 1/S term plus an S term has a minimum, and you find it by differentiating and setting the derivative to zero. The block below writes out the total-time expression, then does exactly that.

fixed cost per chunk, h                                  0.15 s

with chunk size S MB, chunk time t = S / 0.625, total time is
    720/0.625  +  (720/S) x h  +  (720/0.625) x (exp(p x t) - 1)/2
      base         per-chunk           retry waste

to second order the waste term is 720 x p x S / (2 x 0.625^2), so
    dT/dS = 0   ->   S* = 0.625 x sqrt(2h / p)
  2 x 0.15 / 0.002                       =  150
  150^0.5                                =  12.25
  0.625 x 12.25                          =  7.66

The algebra says 7.66 MB. The table below checks that answer by brute force at six chunk sizes, so you can see the shape of the curve rather than trusting one derivative.

Read the columns like this. Chunks is 720 / S. Handshake is that count times 0.15 s. Retry waste is the seconds lost to drops: chunk count times the chance a chunk dies, exp(p x t) - 1, times the half-chunk you lose on average, which works out to chunks x (exp(p x t) - 1) / 2 x t where t = S / 0.625 is how long one chunk takes. Total adds both of those to the 1,152 seconds the transfer would take on a perfect link — that 1,152 is the floor and it never appears as a column. Overhead is the total divided by 1,152, minus one.

Going down the table, handshake falls and retry waste rises; they cross near 8 MB, and the Total column barely moves between 4 MB and 16 MB.

The bottom row is computed differently from the ones above it, and it has to be. Every chunked row has p x t well under 0.03, so “half the chunk is lost” holds to three decimal places and the retry-waste column follows the formula just given. The single-request row has p x t = 2.30, far outside that regime, so it takes its waste from the conditional mean derived above instead: 9 failed attempts, each losing 372 seconds, is 9 x 372 = 3,350 seconds. Apply the small-p x t formula to that row anyway and you get the familiar 5,192 and +451% — a number you will see quoted, and now know is the wrong tool applied to the wrong row.

Eight megabytes, and the curve is flat between 4 and 16, which is the useful part of the answer.

The optimum sits where handshake cost and retry waste are equal — 13.5 s against 14.9 s. That is not a coincidence. It is what dT/dS = 0 means for a sum of a 1/S term and an S term: at the minimum, the two terms contribute equal amounts.

Say the closed form out loud: S* = rate x sqrt(2h/p). It gives you the whole sensitivity for free. Chunk size scales as 1/sqrt(p), so quadrupling the drop rate halves the chunk: a satellite link at p = 0.02 wants 2.4 MB, and a datacenter copy with p near zero wants chunks as large as the object store will take.

Now price the 8 MB choice. The block below asks what fraction of the file gets re-sent when a 720 MB upload is cut into 90 chunks, and the last line is the number to remember.

chunks in a 720 MB upload at 8 MB
  720 / 8                                =  90
probability a given chunk needs a retry, p x t with t = 12.8
  12.8 x 0.002                           =  0.0256
expected retries per upload
  90 x 0.0256                            =  2.30
bytes re-sent, half a chunk each on average, MB
  2.30 x 4                               =  9.2
as a share of the file
  9.2 / 720                              =  0.0128

1.3% of bytes re-sent, against 291% for the single-request design. Three server-side requirements make that work, and each is a place a naive implementation fails.

• A durable committed_through offset per upload, written on every chunk commit. The load is 500,000 x 90 = 45,000,000 commits/day, which at the standing rounding of 100,000 seconds per day is 45,000,000 / 100,000 = 450 per second. That is trivial for a key-value store, and it is the reason the upload ledger is its own table with a short expiry rather than a column on the videos row.
• Per-chunk content hashes, so that re-sending a chunk is idempotent. Idempotent means doing the operation twice has exactly the same effect as doing it once. Without the hash, a chunk that committed but whose 204 response was lost in transit gets written a second time, and a client resuming from a stale offset silently corrupts the file. The hash makes the duplicate write a no-op and makes any real corruption detectable at complete.
• Ingest is terminated regionally, close to the uploader. A cross-continent upload pays about 150 ms of round-trip time — RTT, the time for a packet to reach the far end and its acknowledgement to come back (ch 02) — on every chunk commit, which at 90 chunks is 13.5 seconds of pure waiting bolted onto an already slow operation.

8. Deep dive 2: transcoding as a DAG

Sizing the encoding cluster from first principles is also what settles the question interviewers reach for immediately — whether to do this on graphics hardware — and it starts with why the work is organised as a graph of independent tasks rather than a single long job. The DAG named in High level architecture earns a section of its own because of two properties: no cycles means the whole thing is guaranteed to terminate, and any task whose inputs are ready can run right now, on any machine.

The unit for all of it is the core-second: one CPU core kept busy for one second. A core-hour is 3,600 of those. Counting compute this way lets you add up work regardless of how many cores a machine has, and then divide by machines only at the end.

Two accounting terms are then used with deliberate precision, because mixing them is the most common sizing error in this chapter. Offered load is how much work arrives, in core-seconds per day. Capacity is how much work a machine can absorb, in the same units. A fleet size is offered load divided by capacity — and only after you have applied a peak factor and a utilization target, never the raw mean divided by a rated maximum.

The utilization target is why a 32-core box is counted at 80% and not 100%. A queue served at its exact arrival rate has unbounded waiting time; leaving headroom is what keeps latency finite, and 80% is the conventional place to leave it.

The ladder, and the compute it implies

The ladder’s compute cost is set by total pixel count — an encoder’s work scales with how many pixels it has to look at — so the first question is how much a five-rung ladder costs relative to encoding the top rung alone.

The block below multiplies width by height for each rung, sums them, and divides by the 1080p number. The ratio on the last line is what you carry forward.

pixels per frame, by rung
  240p  :    426 x 240                   =    102,240
  360p  :    640 x 360                   =    230,400
  480p  :    854 x 480                   =    409,920
  720p  :   1280 x 720                   =    921,600
  1080p :   1920 x 1080                  =  2,073,600
sum
  102,240 + 230,400 + 409,920 + 921,600 + 2,073,600  =  3,737,760
the whole ladder against the top rung alone
  3,737,760 / 2,073,600                  =  1.80

Adding four rungs below 1080p costs 80% more, not 400% more, because pixel count roughly halves at each step and a geometric series with ratio near one half sums to about twice its first term. Nobody agonizes over ladder length for that reason: the marginal rung is cheap in compute. It is storage that scales with rung count, which is Deep dive 3 storage tiering and generating the tail on demand.

Turning pixels into machines needs two more terms.

A preset is the named speed-versus-quality setting an encoder runs at. A slow archival preset spends far more computation searching for a compact representation, and produces a smaller file at the same visual quality. A fast preset gives up on that search and emits a bigger file sooner.

A realtime multiplier expresses encoder speed as a ratio to playback speed. “0.35x realtime” means the encoder produces 0.35 seconds of finished video per second of work, so one processor core needs about three seconds of work per second of video.

Decoding — turning the compressed mezzanine back into raw frames — is far cheaper than encoding. That is why it is done once and its output shared by all five encodes, which is the 1 / 10 line below.

The block runs from one core’s speed all the way to a fleet size. Follow it in four moves: cost per second of 1080p, cost per second of the whole ladder, cost of one 12-minute video, then a day’s worth of videos divided by what one box supplies.

assume  one core encodes 1080p30 at 0.35x realtime at the archival preset,
        and decodes 1080p30 at 10x realtime

core-seconds per second of 1080p output
  1 / 0.35                               =  2.86
the whole ladder, scaled by the pixel ratio
  2.86 x 1.80                            =  5.15
decode the mezzanine once, shared by all five encodes
  1 / 10                                 =  0.10
per second of source
  5.15 + 0.10                            =  5.25
per 12-minute video, core-seconds
  720 x 5.25                             =  3,780
core-hours
  3,780 / 3,600                          =  1.05

core-seconds/day at 500 k uploads -- OFFERED LOAD, daily mean
  500,000 x 3,780                        =  1,890,000,000
one 32-core box at 80% utilization -- CAPACITY, core-seconds/day
  32 x 0.80 x 86,400                     =  2,211,840
boxes that exactly meet the mean
  1,890,000,000 / 2,211,840              =  854
provisioning factor: draining a backlog, reruns, lane isolation  1.5
transcode fleet
  854 x 1.5                              =  1,281

854 is the mean demand. The transcode fleet is 1,281, and the two numbers are not interchangeable. A fleet sized at exactly the offered load has zero spare capacity, so it can never work a backlog down — only hold it level. Any hour that runs long stays long forever.

This is still the one tier where you do not multiply by a peak factor. Uploads are bursty, but transcoding sits behind a queue, and a queue converts a traffic spike into publish latency rather than into machines. What the 1.5x buys instead is the drain rate after a spike, plus two priority lanes so a large channel is not stuck behind a backlog of hobbyist uploads. The p99 publish latency absorbs whatever is left.

1,281 is the number every cost below is computed from.

Why it is a DAG and not a loop

Fleet size answers “how many machines”; it says nothing about how long one creator waits. That is a wall-clock question, and it is answered by cutting the video into independent pieces and encoding them at the same time on different machines.

one core, start to finish, in minutes
  3,780 / 60                             =  63
split into 30-second work units, units per video
  720 / 30                               =  24
encode tasks, units x rungs
  24 x 5                                 =  120
wall clock if all 120 run concurrently, seconds
(the longest single task is 30 s of 1080p)
  30 x 2.86                              =  86

Sixty-three minutes on one core becomes 86 seconds on 120, which is the difference between a publish latency creators tolerate and one they do not.

The graph below is that plan drawn out. Read it left to right: one serial head, a five-way fan-out in the middle where all the parallelism lives, and a fan-in back to a single publish. The prose after it walks each stage.

flowchart LR
    A["Assemble chunks<br/>verify sha256"] --> B["Probe: codec, fps,<br/>HDR, rotation"]
    B --> C["Split into 30 s units<br/>on segment boundaries"]
    C --> D1["encode 240p x24"]
    C --> D2["encode 360p x24"]
    C --> D3["encode 480p x24"]
    C --> D4["encode 720p x24"]
    C --> D5["encode 1080p x24"]
    B --> AU["Audio: AAC + Opus"]
    B --> TH["Storyboard frames"]
    D1 --> S["Stitch per rung<br/>verify GOP alignment"]
    D2 --> S
    D3 --> S
    D4 --> S
    D5 --> S
    AU --> S
    S --> P["Package CMAF<br/>+ manifest + thumbnails"]
    TH --> P
    P --> PUB["Publish"]

The serial head: assemble, then probe. The first task assembles the chunks and verifies the sha256 fingerprint of the reassembled file against the one the client declared. That is the moment a corrupted upload is caught rather than encoded.

Next comes Probe: codec, fps, HDR, rotation, which reads four facts out of the file: what codec it arrived in, how many frames per second (fps) it runs at, whether it carries HDR (high dynamic range — an extended brightness and colour range that has to be tone-mapped, meaning squeezed back into the ordinary range, for any rung that does not support it), and whether the phone recorded it sideways. Those four facts determine every parameter of everything downstream, which is why nothing else may start first.

The fan-out is where the parallelism is. The file is split into 30-second units on segment boundaries. Each rung then gets its own row of tasks — all five rows are drawn, 240p through 1080p, each with twenty-four independent 30-second encode tasks, which is where 5 x 24 = 120 comes from. Those 120 tasks are what run at the same time on different machines.

Two side branches leave the probe directly, because they do not depend on the video encode at all. Audio: AAC + Opus encodes the sound twice, in two formats, so that every client has one it supports. Storyboard frames produces the scrub-preview thumbnails from Requirements.

The fan-in is a stitch per rung. Each stitch concatenates that rung’s 24 encoded pieces back into one continuous rendition and verifies GOP alignment. A GOP, or group of pictures, is the run of frames between one full self-contained frame and the next. That self-contained frame is called an IDR (instantaneous decoder refresh) frame, and it is the only place a player can start decoding or switch quality. Verifying alignment means checking that every rung placed its IDR frames at the same instants.

The last step, Package CMAF + manifest + thumbnails, is the same packaging stage as in High level architecture. Only after it does the video publish.

The graph is drawn without colour on purpose. Ch 01’s key marks data roles — authority, read capacity, byte-bound, irreversible — and a task graph has none of them. Every box here is processor time, which is the one thing the key deliberately has no colour for.

Three properties the DAG buys, and one it costs:

• Every encode task is independently retryable. A machine lost mid-video costs 30 seconds of work, not 63 minutes, and the failure does not propagate.
• Rungs publish independently. If 1080p is late, the video goes live with the four rungs that finished and the manifest is rewritten when the fifth lands.
• Work units align to segment boundaries — 30 = 5 x 6 — so parallel encoding forces zero extra keyframes. Choose 25-second units and every boundary is an unplanned IDR, pure bitrate waste on the most expensive resource in the system.
• The cost: per-unit rate control cannot see across boundaries. Rate control is the encoder’s budgeting logic — it decides how many bits to spend on each part of the video so the whole thing lands near its bitrate target, spending more on a fast-moving scene and less on a static one. Split the video across 24 machines and each one budgets in ignorance of the other 23. The fix is a two-pass encode: one pass measures how hard each part of the video is, then a shared bitrate target is set from the whole video’s measurements before the second pass actually encodes. Without it you get roughly 2% of quality drift at the seams. That is the honest answer to “why not just split it”.

CPU or GPU, and why the transcode bill is not the bill

Every interviewer asks the same question here — “why not do this on GPUs, they are much faster at video?” — and the answer is a lesson about which bill you are optimizing.

Three terms first.

A fixed-function encoder is a dedicated silicon block that implements one video codec directly in hardware. It is enormously faster and more power-efficient than software. It also cannot be told to search harder, because there is no software loop to slow down, so its output file is larger at the same visual quality.

NVDEC is the corresponding hardware decoder block. It is present on parts that have no encoder at all, which is exactly the trap in the pricing below.

VMAF (Video Multimethod Assessment Fusion) is the standard perceptual quality score, used to compare two encodes fairly. “15% more bits for the same VMAF” means the hardware encoder needs a file 15% bigger to look equally good to a human eye.

Start with the per-video cost on each side. The block below prices one video’s encode on CPU, then on a hardware encode part, and the two bottom lines are the ones to compare.

cost per core-hour on a 32-core box at $1.20/hour
  1.20 / 32                              =  0.0375
software encode, per video
  1.05 x 0.0375                          =  0.0394

assume  fixed-function encoders emit the whole 5-rung ladder at 12x
        realtime from a single decode, on an L4-class encode part at
        $1.00/GPU-hour -- NOT this repo's $2.50 H100 or $2.00 A100 rate,
        because those parts carry NVDEC only and cannot do this job at all

GPU-seconds per video
  720 / 12                               =  60
GPU-hours
  60 / 3,600                             =  0.0167
at $1.00/GPU-hour
  0.0167 x 1.00                          =  0.0167

State the part, not just the rate. Hardware video encode lives on T4 / L4 / L40S-class parts. The datacenter training accelerators priced elsewhere in this repo decode in hardware and encode in software, so borrowing their $2.00 or $2.50 rate would be paying for tensor cores this calculation never touches. $1.00/GPU-hour is if anything high for the class, which understates the hardware saving and therefore flatters the conclusion this subsection reaches — which is why the subsection ends by pricing the encoder at zero and checking the conclusion again.

Two accounting choices, and why you must name yours

Both sides have to be priced the same way, and the answer moves by a factor of two depending on which way you pick. Pick one and say which out loud.

Two independent choices move the number.

Choice one: is the CPU side quoted at mean demand or at the fleet you actually provision? 854 boxes is the mean demand. 1,281 is the fleet that exists, because you provisioned 1.5x. Quoting 854 prices machines you would not own.

Choice two: is the GPU side an owned fleet or rented hours? An owned fleet means you buy enough GPUs to absorb the daily load at 80% utilization and pay for them 24 hours a day whether or not work arrives — that is 435 GPUs. Rented hours means you pay only for the GPU-seconds you consume, so the idle hours cost nothing. The second is cheaper on paper and only honest if you can actually rent that way.

The table below runs both choices in every combination, plus the free-encoder bound at the end. The last column is what matters: the $120 M/year egress penalty divided by that row’s compute saving. Every row is above 8, which is why none of these choices changes the verdict.

Two of those rows are internally consistent readings you could defend in an interview, and the block below shows their arithmetic in full.

(a) both as owned fleets at the same 80% utilization, quoted at mean
    demand -- applying the same 1.5x to both scales the saving by 1.5
    and lands the final ratio at 15.5, which changes nothing:
  CPU, $/year      854 x 1.20 x 24 x 365    =  8,977,248
  GPU-seconds/day  500,000 x 60             =  30,000,000
  one GPU at 80%, GPU-seconds/day
    86,400 x 0.80                           =  69,120
  GPUs             30,000,000 / 69,120      =  434  ->  435
  GPU, $/year      435 x 1.00 x 24 x 365    =  3,810,600
  transcode saving
    8,977,248 - 3,810,600                   =  5,166,648

(b) CPU as the 1,281-box fleet actually provisioned above, GPU rented by
    the hour it consumes:
  CPU, $/year      1,281 x 1.20 x 24 x 365  =  13,465,872
  GPU-hours/year   30,000,000 x 365 / 3,600 =   3,041,667
  GPU, $/year      3,041,667 x 1.00         =   3,041,667
  transcode saving
    13,465,872 - 3,041,667                  =  10,424,205

The GPU wins by somewhere between $5.2 M and $10.4 M a year on compute. Now price the other side of the trade — the bits that extra-large hardware output costs you every time somebody watches.

hardware encoders need roughly 15% more bits for the same VMAF
CDN egress, $/year, from ch 02 estimation 4              800,000,000
the bitrate penalty
  800,000,000 x 0.15                     =  120,000,000
against the transcode saving, reading (a)
  120,000,000 / 5,166,648                =  23.2
against the transcode saving, reading (b)
  120,000,000 / 10,424,205               =  11.5

The egress penalty is 11.5 to 23 times the compute saving, so software encoding still wins by an order of magnitude even on the reading most favourable to hardware.

Note which direction the encoder’s price pushes that range. At a training accelerator’s $2.00/GPU-hour the two readings give 88x and 16.3x. Halving the rate to the L4-class $1.00 roughly doubles the hardware saving on both, which narrows the range to 23.2 and 11.5. Pricing the part generously is the honest way to argue a case you expect to win, and it still loses.

One line says why it must lose. The compute saving can never exceed the entire CPU bill, so the ratio can never fall below 120,000,000 / 13,465,872 = 8.9 — even an encode part that cost nothing at all would leave the egress penalty nine times the whole thing it saves.

That is the 306:1 read amplification doing its work. In a system that reads 306 bytes for every byte written, anything that trades output size for input cost loses by roughly that factor.

Hardware encoding is right for live streaming, where the deadline is realtime and a slow preset is not available at all. It is wrong for on-demand, where a file is encoded once and read a million times.

9. Deep dive 3: storage tiering, and generating the tail on demand

The view distribution from Back of the envelope is worth a hundred million dollars a year — provided part of the saving is spent back on compute and the trade still pays, which is what this dive checks.

The pricing unit is the GB-month: cloud object storage bills you for holding one gigabyte for one month. So a corpus’s annual storage cost is its size in gigabytes, times the monthly rate, times twelve.

A storage class is the tier you place an object in. The hot class here costs $0.023 per GB-month and reads immediately. The cold class costs $0.004 and is slower to first byte. Moving an object between them is a policy decision, not an engineering one — which is why the saving below is available to anybody willing to write the policy.

Start with the bill you would pay for doing nothing clever: every rung of every video in the hot class.

corpus 500 M videos, full ladder resident, in PB
  500,000,000 x 828 / 1,000,000,000      =  414
hot object storage at $0.023/GB-month, $/month
  414,000,000 x 0.023                    =  9,522,000
$/year
  9,522,000 x 12                         =  114,264,000

$114 M a year to keep every rung of every video in the hot class, and Back of the envelope already established that 490 million of those videos average four tenths of a view a day each. This is the same tiering argument ch 02 makes for cold photos, with this workload’s numbers filled in.

Which rungs the tail keeps, and why it is not the popular ones

The policy that follows is asymmetric. Head videos keep every rung in the hot class. Tail videos keep only two rungs, in the cold class, and any other rung a viewer asks for is generated at request time.

Which two is not a popularity question, and this is the place the obvious answer is wrong. The middle column of the table below is how much each rung is actually watched; the right column is the policy. Notice that they disagree.

Those two carry 41% of playback seconds, and they are deliberately not the two that carry the most. The two most-played rungs are 720p and 1080p, at 57% between them. Picking by playback share would keep exactly those — dropping 360p, which is the cheap floor, and keeping 720p, which is the one rung you never need to store at all.

The reason is a one-way property. Encoding runs downhill only. From a stored 1080p you can generate 720p, 480p and 240p, because shrinking a picture only throws information away. From a stored 720p you can generate nothing above it — the extra detail is gone and no amount of compute invents it back.

So losing the top rung means the highest quality is gone for good rather than merely slow, while losing any lower rung means it is merely slow. That is why the top rung is kept: it is the only irreplaceable one. The second kept rung, 360p, is a cheap floor, so the commonest bandwidth-constrained request costs no compute at all.

The 34% of playback that 720p carries is not lost. It is generated, which is exactly what the rest of this section prices.

Read the table as a policy about derivability, not popularity, and say so out loud, because “keep the two most popular rungs” is a sentence that sounds right and picks the wrong pair.

Now price the policy. The block below sizes the head and the tail separately — head at full ladder and hot prices, tail at two rungs and cold prices — and subtracts the total from the $114 M above.

tail footprint per video, MB
  405 + 63                               =  468

head, 10 M videos, full ladder, hot class:
  petabytes
    10,000,000 x 828 / 1,000,000,000     =  8.28
  dollars per month at $0.023/GB-month
    8,280,000 x 0.023                    =  190,440

tail, 490 M videos, two rungs, cold class at $0.004/GB-month:
  petabytes
    490,000,000 x 468 / 1,000,000,000    =  229.3
  dollars per month
    229,300,000 x 0.004                  =  917,200

total $/month
  190,440 + 917,200                      =  1,107,640
total $/year
  1,107,640 x 12                         =  13,291,680
saved against all-hot
  114,264,000 - 13,291,680               =  100,972,320

A hundred and one million dollars a year, from one Zipf curve and two storage classes. The cold-class price is ch 02’s “roughly one fifth” ratio made concrete.

What it costs back: just-in-time transcoding

The saving above is only real if generating the missing rungs on demand costs less than storing them, so price that compute and subtract it.

Just-in-time (JIT) transcoding means encoding a rung at the moment a viewer requests it, from a rendition you did keep, serving the result, and caching it in case another viewer wants the same rung of the same video. That last clause is a hope, and the arithmetic below declines to grant it.

The first block converts “some playback needs a rung we did not store” into core-seconds. It runs in four moves. First, what share of tail playback wants a missing rung. Second, how expensive each missing rung is relative to a full 1080p encode — that is the pixel fraction, since encode cost tracks pixel count. Third, a weighted average of those fractions across the playback mix, normalised so it is an average over the requests that need JIT rather than over all requests. Fourth, the resulting per-second cost, at a fast preset plus one decode.

share of tail playback wanting a rung the tail does not keep
  1 - 0.18 - 0.23                        =  0.59

pixel fraction of each missing rung, relative to 1080p
  240p    102,240 / 2,073,600            =  0.049
  480p    409,920 / 2,073,600            =  0.198
  720p    921,600 / 2,073,600            =  0.444
weighted by playback share
  0.04 x 0.049                           =  0.00196
  0.21 x 0.198                           =  0.04158
  0.34 x 0.444                           =  0.15096
  0.00196 + 0.04158 + 0.15096            =  0.1945
normalised by the 0.59 that needs JIT
  0.1945 / 0.59                          =  0.330

core-seconds per encoded second, archival preset
  2.86 x 0.330                           =  0.944
at a JIT preset roughly 5x faster than the archival one
  0.944 / 5                              =  0.189
plus one decode of the stored 1080p rendition
  0.189 + 0.10                           =  0.289

tail views/day wanting a missing rung
  195,000,000 x 0.59                     =  115,050,000
playback-seconds, at ch 02's 5-minute mean watch
  115,050,000 x 300                      =  34,515,000,000

Now say what a cached JIT result actually is, because the divisor depends on it and the chapter is about to divide by the wrong thing.

A JIT result is one rung of one video, not a video. The block above already split the missing playback across three separate rungs, so a cached 480p result does nothing for the next viewer who wants 720p.

That changes the reuse question. It is not “how many views does this video get in seven days”. It is “how many views does this (video, rung) pair get in seven days” — and the two differ by the rung’s own share of playback. The block below computes both so you can see the gap.

views per tail video per day (section 3)                 0.398
per VIDEO, over a 7-day origin TTL
  0.398 x 7                              =  2.79
per (VIDEO, RUNG), which is what one JIT result is
  720p   0.398 x 0.34 x 7                =  0.95
  480p   0.398 x 0.21 x 7                =  0.59
  240p   0.398 x 0.04 x 7                =  0.11

Every one of those is below 1, so there is essentially no reuse and the honest divisor is 1, not 2.79. A cached result asked for less than once before it expires is serving the request that created it and nothing else.

With the divisor settled, the fleet falls out. The block below converts playback seconds into core-seconds, divides by the same per-box capacity used for the transcode fleet, prices it, and subtracts it from the storage saving.

seconds actually encoded: playback seconds over a reuse
divisor of 1, so the two are the same number
  34,515,000,000 / 1                     =  34,515,000,000
core-seconds/day -- OFFERED LOAD, daily mean
  34,515,000,000 x 0.289                 =   9,974,835,000
boxes at the same 80%-utilized CAPACITY per box
  9,974,835,000 / 2,211,840              =  4,510
$/year
  4,510 x 1.20 x 24 x 365                =  47,409,120
net saving
  100,972,320 - 47,409,120               =  53,563,200

Fifty-four million dollars a year, net — and the per-video divisor would have said eighty-four.

Quote 4,510 boxes and $54 M. If you want to argue the optimistic reading, say what would have to be true for it: the tail would need enough segment-level locality to lift a (video, rung) above one request per TTL. Segment granularity pushes the other way, because a 12-minute video at 6-second segments is 120 objects per rung, and a 5-minute mean watch touches only about half of them.

The tiering decision survives either reading — $54 M is still a very large number for two storage classes and a policy. The fleet does not survive: 1,616 boxes against 4,510 is the difference between one rack row and three.

Note what 4,510 is and is not. It is mean demand divided by a box’s capacity, with no provisioning multiple on top, unlike the 1.5x the transcode fleet gets. That is deliberate rather than an omission. The JIT path has a valve the transcode queue does not: Failure modes rate-limits it per client and falls back to a stored rung, so a spike degrades quality instead of building a queue. If you would rather not lean on that valve, apply the same 1.5x — 6,765 boxes, $71 M/year, and a net saving of $30 M instead of $54 M. Say which one you are quoting.

One more number is worth more than the result itself, because it is where this design is usually gotten wrong. The block below asks what happens if the JIT path reuses the same slow archival preset the publish pipeline uses.

if the JIT path reused the archival preset instead
  0.944 + 0.10                           =  1.044
against the fast preset
  1.044 / 0.289                          =  3.61
boxes
  34,515,000,000 x 1.044 / 2,211,840     =  16,291
$/year
  16,291 x 1.20 x 24 x 365               =  171,250,992
as a share of the storage saving
  171,250,992 / 100,972,320              =  1.70

Reusing the archival preset for on-demand work does not eat into the saving, it exceeds it by 70% — a $54 M win becomes a $70 M loss.

They are different jobs. An archival encode is read a million times, so every byte the slow search saves is repaid a million times over; it deserves the slow preset. A JIT encode is read about once, so the slow search is repaid once and deserves the fastest preset that still looks acceptable.

The preset is a function of the expected read count and nothing else. Once you compute that read count per rung rather than per video, the margin for getting the preset wrong disappears entirely.

Why JIT is viable at all is the segment. Transcoding a whole 12-minute rendition on the play path would be 720 x 0.289 = 208 core-seconds, and no viewer waits three and a half core-minutes for a video to start.

But the media is already cut into 6-second segments, so the real unit of work is 6 x 0.289 = 1.7 core-seconds — under half a second on four cores, comfortably inside a player’s startup buffer, and paid only for the segments actually watched.

10. Deep dive 4: CDN economics, extended

Cache hit ratio is usually assumed; here it is computed — and the number it produces forces a whole extra tier of infrastructure into the design, whose value then turns entirely on a definition the arithmetic never states: what one cached copy actually is.

Three terms carry the section. A cache hit ratio is the fraction of requests a cache answers from its own copy; every miss is a request that must be paid for further upstream. A TTL is the same time-to-live idea Data model applied to database rows, pointed at a cache instead: how long a cached copy is allowed to stay before it is discarded. A NIC is a network interface card, the piece of hardware whose bandwidth ultimately limits how many bytes one machine can push.

Computing the edge hit ratio instead of assuming it

Chapter 02 establishes the delivery figures reused here: 110 PB/day of egress, $2.2 M/day, $800 M/year, and the conclusion that edge hit ratio is the most valuable metric in the system. What it does not do is say what that ratio can actually be.

Hit ratio is not a tuning parameter. The view distribution and the cache TTL determine it, and you can compute it in four lines.

The logic of those lines is worth stating before you read them. A cached copy is worth keeping only if more than one request arrives for it before it expires — a copy fetched once and then thrown away has saved nothing. So the block below counts requests per cached copy: spread the tail’s daily views across 50 PoPs, then across 490 million videos, then multiply by the 7-day TTL. Compare the result against 1.

edge PoPs                                                     50
tail views per PoP per day
  195,000,000 / 50                       =  3,900,000
per tail video, per PoP, per day
  3,900,000 / 490,000,000                =  0.00796
requests one cached copy serves inside a 7-day edge TTL
  0.00796 x 7                            =  0.0557
  -- and a copy is one RUNG of one video, so the real figure is
     this times that rung's playback share: 0.019 for 720p,
     0.002 for 240p. Both round to zero; the conclusion is
     unchanged here, which is why this granularity survives
     unstated until the shield tier below, where it does not

blended edge hit ratio
  0.805 x 1.00 + 0.195 x 0.00            =  0.805
origin egress, PB/day
  110 x 0.195                            =  21.45
in Gbps -- OFFERED LOAD, and a daily MEAN
  21.45 x 92.6                           =  1,986
prime-time peak, at 2x the daily mean
  1,986 x 2                              =  3,972
usable NIC CAPACITY per machine, 80% of 1 Gbps          0.8
machines of origin NIC
  3,972 / 0.8                            =  4,965

0.0557 requests per cached copy is far under one, so essentially every tail request at an edge PoP is a miss — no matter how large the PoP or how long the TTL.

The head is the opposite. 10 million videos taking 805 million views a day, spread across 50 PoPs, is 805,000,000 / 50 / 10,000,000 = 1.6 views per video per PoP per day, and a week’s TTL turns that into a hit ratio near one.

Sizing the origin from the miss rate

Two terabits a second out of origin — but that is a daily mean of offered load, and a fleet cannot be sized on a mean.

Views peak at roughly twice the daily average in prime time, and nobody plans a network card at 100% of its rated speed. So the fleet is 4,965 machines of origin NIC doing nothing but feeding the caching network, not the 1,986 you get by spending the mean straight as though it were capacity.

The gap between those two figures is exactly 2.5x — the 2x peak times the 1/0.8 utilization headroom — and the direction of the error is the usual one: treating offered load as capacity always flatters the design. The tail is 19.5% of views and 100% of the origin problem.

Why a bigger edge cache buys nothing

Note what this does to an intuition that holds almost everywhere else.

In ch 08, a URL shortener, buying more cache memory bought hit rate logarithmically. Misses there were capacity misses: the object had been requested before and was evicted to make room, so a bigger cache would have kept it.

Here more memory buys nothing, because the miss is a first-access miss — an object that was never requested at that PoP at all. No cache of any size could have held something nobody asked for.

The fix for a first-access miss is not a bigger cache, it is fewer independent caches. Fewer caches means each one sees more of the traffic, so the first access happens sooner and the copy it creates gets reused. The block below runs the same requests-per-copy arithmetic with 4 shields instead of 50 PoPs, and a 30-day TTL instead of 7.

regional shields                                               4
tail views per shield per day
  195,000,000 / 4                        =  48,750,000
per tail video, per shield, per day
  48,750,000 / 490,000,000               =  0.0995
requests one copy serves inside a 30-day shield TTL, PER VIDEO
  0.0995 x 30                            =  2.99
shield hit ratio on the tail, per video
  1 - 1 / 2.99                           =  0.666
origin egress behind the shield, PB/day
  21.45 x 0.334                          =  7.16
in Gbps, daily mean
  7.16 x 92.6                            =  663
machines of origin NIC, same 2x peak and 80% NIC
  663 x 2 / 0.8                          =  1,658
machines of NIC no longer needed
  4,965 - 1,658                          =  3,307

origin-to-CDN transfer at $0.01/GB, without a shield, $/day
  21,450,000 x 0.01                      =  214,500
$/year
  214,500 x 365                          =  78,292,500
with the shield, $/year
  7,160,000 x 0.01 x 365                 =  26,134,000
saved
  78,292,500 - 26,134,000                =  52,158,500

What a cached copy actually is, and why it swings the answer 100x

Read that $52 M as the optimistic bound, because 2.99 is requests per video and a cached object is one segment of one rung.

This is the same granularity slip Deep dive 3 storage tiering and generating the tail on demand makes about JIT results. At the edge it was harmless — 0.0557 and 0.019 both round to zero. At the shield it is load-bearing, because 2.99 is above 1 and the per-rung figures mostly are not. The block below multiplies the per-video figure by each rung’s playback share to get the real one.

requests one copy serves in 30 days, PER (VIDEO, RUNG)
  720p    2.99 x 0.34                    =  1.02
  1080p   2.99 x 0.23                    =  0.69
  480p    2.99 x 0.21                    =  0.63
  360p    2.99 x 0.18                    =  0.54
  240p    2.99 x 0.04                    =  0.12
hit ratio per rung, 1 - 1/n where n > 1 and zero otherwise
  720p only                              =  0.015
blended over the playback mix
  0.34 x 0.015                           =  0.005

Under per-rung accounting the shield’s tail hit ratio is half a percent, not 66.6%, and the $52 M is roughly $0.4 M.

One line of algebra explains why the two answers are so far apart. Requests per copy are 0.398 x TTL / shields x rung share. Clearing the one-request-per-copy line on the largest rung therefore needs TTL / shields >= 1 / (0.398 x 0.34) = 7.4 days.

This design runs at 30 / 4 = 7.5 days. It is sitting essentially on the break-even point, which is why the answer swings by two orders of magnitude on a definition. And segment granularity pushes it further the wrong way, since a 5-minute watch of a 12-minute video touches only about half of that rung’s 120 segment objects.

Quote the pessimistic number and quote the lever with it, because the lever is the real finding.

Reuse scales as TTL / shields. So the tier that pays is not four shields at 30 days; it is fewer shields, or a much longer shield TTL, or both.

Run the numbers on that. A single shield at a 30-day TTL clears 0.398 x 30 x 0.34 = 4.1 requests per copy on 720p, which is a 76% hit ratio. Four shields need a 59-day TTL just to reach two requests per copy, and 120 days — four times the TTL, because there are four times the caches — to match what one shield gets at 30.

That is a sharper version of the lesson the edge taught. The fix for a first-access miss is fewer independent caches, and “fewer” has to be measured against the object you are actually caching.

The shield tier still earns its place on the head, through request coalescing on a viral object — the second bullet below. But its tail economics are a function of TTL / shields and have to be re-derived whenever either number moves.

Two more consequences of the same view distribution:

• Prefetch the head rather than waiting for it to be pulled. A video from a large channel is predictably about to be watched everywhere, so pushing it to all 50 PoPs at publish time costs 828 MB x 50 = 41 GB and removes the cold-start miss on exactly the videos that generate the most traffic.
• A viral video arrives at the origin as a step function. In the window between publication and the first edge fill, every PoP misses independently and they all ask at once. This is the caching form of the hot key problem: one object so much more popular than the rest that no amount of spreading data across machines helps, because every copy of the problem is a copy of the same key (ch 05 proves partitioning cannot fix it). The guard is request coalescing at the shield: when many simultaneous requests arrive for an object the shield does not have, it issues exactly one fetch to origin and makes the others wait on that single result, so 50 misses become one.

11. Deep dive 5: adaptive bitrate, where the server does almost nothing

Adaptive bitrate (ABR) streaming means the video is published at several qualities at once, cut into aligned segments, and the player re-chooses a quality before every single segment — so a viewer whose train enters a tunnel drops from 1080p to 360p and keeps playing rather than freezing. The interesting decision lives on the client; the one number the server owns is how long a segment should be.

The client picks the rendition and the server merely publishes a list. That division of labour is not a convention someone chose. It falls out of where the deciding information lives and how fast that information moves.

The block below is not arithmetic but a comparison of three timescales — how often the decision has to be made, how fast the inputs to it change, and how long it would take to ship those inputs to a server.

the client's control loop period, one segment                6 s
the state it decides on -- buffer occupancy, throughput estimate,
decoded-frame drops, viewport size -- changes on a 100 ms scale
the cost of shipping that state to a server that decides instead:
  cross-continent RTT (ch 02)                              150 ms
  plus telemetry batching and aggregation                  >= 1 s

Three of those terms are worth stating plainly.

Buffer occupancy is how many seconds of video the player has already downloaded but not yet shown. It is the cushion that absorbs a network dip, and it is the single most informative variable in the decision.

A throughput estimate is the player’s own running measurement of how fast segments are arriving.

Viewport size is how many pixels the video is actually being displayed in, which is why sending 1080p to a phone showing a thumbnail-sized player is pure waste.

A server-side decider is at best two control periods behind the thing it controls, and it cannot see the two variables that matter most: how full the buffer is right now, and how many pixels the display actually has. Put the controller where the state is.

The manifest, meanwhile, is roughly 2 KB, static, and identical for every viewer, so it caches at the edge like any other object. That is exactly what you want on the path that decides how fast playback starts.

The one thing the server must get right

Every rung has to place its keyframes at the same presentation timestamps — the timeline positions at which frames are meant to be displayed — so that segment n of the 480p rendition and segment n of the 1080p rendition cover the same six seconds of the same content. Without that guarantee, a mid-stream quality switch produces a visible gap or a repeated frame.

This is why the transcoder forces IDR frames onto a fixed grid, instead of letting each encoder pick its own scene cuts. And it costs real bitrate, because a forced keyframe in the middle of a static shot is bits spent on nothing.

That cost is the price of switchability, and it turns segment duration into an economic decision. The arithmetic needs one distinction: a keyframe (the IDR frame of Deep dive 2 transcoding as a dag) encodes a whole picture from scratch, while an inter frame encodes only what changed since the previous one and is therefore roughly ten times cheaper in bits.

Shorter segments mean more forced keyframes per minute, so the bitrate penalty is just the ratio of average frame costs at two different GOP lengths. The block below computes that average both ways. Read the mean frame cost formula as: one frame in every G costs 10, the other G - 1 cost 1 each, so the mean is (10 + G - 1) / G.

assume  a keyframe costs about 10x an inter frame at the same quality,
        at 30 fps, with a forced keyframe at every segment boundary

mean frame cost with a GOP of G frames is (10 + (G - 1)) / G

6-second segments, G = 180
  (180 + 9) / 180                        =  1.050
2-second segments, G = 60
  (60 + 9) / 60                          =  1.150
bitrate penalty of 2 s against 6 s
  1.150 / 1.050                          =  1.095
applied to ch 02's egress bill, $/year
  800,000,000 x 0.095                    =  76,000,000

segment requests per rendition for a 12-minute video
  720 / 6                                =  120
  720 / 2                                =  360

Two-second segments cost $76 M a year in extra bits and triple the request count, to buy a switching latency nobody watching a recorded video can perceive. Ship 6 seconds for on-demand. Ship 2 seconds or less for live, where segment duration is a floor on end-to-end latency and the economics invert. Say which product you are sizing, because the same question has opposite answers.

12. Dedup and copyright, which are different problems

Two problems sound like one here, and conflating them is expensive.

Deduplication — dedup for short — means noticing that an uploaded file is byte-for-byte identical to one you already store, and keeping only one copy.

Copyright matching means noticing that an uploaded video contains the same content as a protected work even though not one byte agrees, because it was re-encoded, cropped, or re-recorded off a screen.

The first is a hash lookup and is nearly free. The second needs a different technique, a different latency budget, and a different attitude to being wrong.

Exact dedup: cheap, and three ways to get it wrong

Price the easy one first. The block below assumes 3% of uploads are exact re-uploads and asks what that saves in storage and in transcode boxes.

assume 3% of uploads are byte-identical re-uploads
  500,000 x 0.03                         =  15,000
storage avoided per day, TB
  15,000 x 828 / 1,000,000               =  12.4
per year, PB
  12.4 x 365 / 1,000                     =  4.5
transcode boxes avoided, of the 1,281-box fleet
  1,281 x 0.03                           =  38

Exact deduplication is cheap and worth doing, but it comes with three constraints that are easy to miss.

• Reference-count the stored bytes. Once two video_ids point at one stored object, deleting a video must decrement a counter rather than unlink the object, or the second uploader’s video disappears the moment the first uploader deletes theirs.
• Never let dedup change the observable timing of an upload. If a duplicate completes instantly, anyone can test whether a specific file already exists in the system simply by uploading a copy and watching the clock. That is a membership oracle — a way to ask “is this exact content in your system?” and get a reliable yes or no — and pointed at private videos it is a privacy breach. Run the full upload every time and deduplicate on the storage side only.
• This catches literal re-uploads and nothing else. A re-encode of the same content produces a completely different byte string, so exact hashing has zero recall — recall being the share of true matches you actually find — against the case people mean when they say “duplicate”.

Copyright matching, where the error budget is a headcount

That case needs perceptual fingerprints: compact summaries computed from what the audio and video look and sound like rather than from their bytes, designed so that two encodes of the same content produce nearly the same fingerprint. A pHash, or perceptual hash, is the video half of that — a short code computed from a frame’s coarse visual structure, which survives re-encoding.

That pipeline is separate from publication for structural reasons rather than organizational ones, and the arithmetic below says why.

Read the block in two halves. The top half sizes the index and the query rate, which turn out to be small. The bottom half sizes the human consequence of being wrong, which does not.

assume  100 M reference assets, mean 5 minutes, fingerprinted at
        32 B of audio hash + 8 B of video pHash per second

reference index, bytes
  100,000,000 x 300 x 40                 =  1,200,000,000,000
in TB
  1,200,000,000,000 / 1,000,000,000,000  =  1.2
query windows per day
  500,000 x 720                          =  360,000,000
per second
  360,000,000 / 100,000                  =  3,600

wrongful claims/day at a 0.1% false-positive rate
  500,000 x 0.001                        =  500
at 1%
  500,000 x 0.01                         =  5,000
appeals one reviewer handles per day                          60
reviewers needed at 1%
  5,000 / 60                             =  83

The 40 bytes per second in that block is the combined rate: 32 bytes of audio hash plus 8 bytes of video pHash for every second of reference material.

A 1.2 TB index answering 3,600 lookups a second is small by the standards of anything else in this chapter, so the retrieval mechanics are not the hard part.

The error budget is the hard part, and it is set by a headcount rather than by a model: the precision target is whatever the appeals team can absorb. Precision here is the share of raised claims that are correct — the complement of the false-positive rate from What actually breaks. At a 1% false-positive rate you generate 5,000 wrongful claims a day and need 83 reviewers; at 0.1% you generate 500 and need 9.

That inversion — the model’s threshold read off the staffing plan rather than off a validation curve — is the difference between a candidate who has shipped an enforcement system and one who has read about one. Three separations follow from it.

• Copyright matching must not gate publication. A reference scan takes minutes to hours; a publish takes 86 seconds. So publish first and claim later, gating publication only for accounts that already have a history of claims.
• It needs a different retry policy from the transcode DAG. A failed encode simply retries. A failed match must never be allowed to silently pass, because “we scanned and found no match” and “the scan did not run” have very different legal meanings.
• Its false positives are a product decision and its false negatives are a legal one. A false negative is infringing content the matcher missed. Those two errors have different owners inside the company, which is why the thresholds belong in configuration that either owner can move, not baked into the model.

13. Bottlenecks and scaling

Every limit the chapter derived, collected so you can answer “what breaks first, and at what number?” without re-deriving anything — the summary you would draw on the board in the last five minutes.

The one that surprises people is the origin NIC. You can be entirely correct about the CDN and still need five thousand machines just to feed it, because the long tail is a first-access miss and no amount of edge memory fixes a first access. And note how that number is built — mean offered load, then a peak factor, then a utilization target, then a box count. Skip the middle two steps and you under-buy the fleet by 2.5x.

14. Failure modes

Nine ways this system actually fails in production, each with the trace an engineer would see, the signal that detects it, and the mechanism that prevents it. The pattern worth internalizing: almost every guard here is a design decision made in advance, not a runbook step taken during the incident.

15. Alternatives rejected

Seven designs a reasonable engineer would reach for instead, each with what is honestly attractive about it and the number that rules it out. Several of them are not wrong so much as right for a different product, and saying which is what separates a judgement from a rule.

A single POST with the whole file. The attraction is real: one endpoint, no upload ledger, and no resume logic to get wrong. It is rejected because it succeeds only 10% of the time on a 5 Mbps link that drops once every 500 seconds, and because retrying from zero means transferring an expected 3.9 times the file size — 9 failed attempts each dying, on average, 32.3% of the way through, which is where a truncated exponential puts the mean rather than at half. Chunking is not an optimization here, it is the difference between working and not working.

GPU or fixed-function hardware encoding for the main ladder. This is $5.2 M to $10.4 M a year cheaper, depending on whether both sides are counted as owned fleets or the GPUs are rented by the hour, and it has far lower wall clock. Both figures are priced on an L4-class encode part at $1.00/GPU-hour, because the H100 and A100 rates used elsewhere in this repo buy parts with no video encoder at all.

It is rejected because hardware encoders need roughly 15% more bits for the same perceptual quality, and 15% of an $800 M egress bill is $120 M — 11.5x to 23x the compute saving on either accounting, and 8.9x even if the encoders were free. It is correct for live streaming, where realtime is a hard deadline and no slow preset exists.

Keeping the full ladder in hot storage for every video. This buys real simplicity: no just-in-time fleet, no promotion logic, and uniform playback latency for every video in the catalogue. It is rejected at $114 M a year against $13 M, because 490 million videos average 0.4 views a day each. Just-in-time transcoding costs $47 M of that back — a JIT result is one rung of one video and is read about once before it expires, so there is no reuse divisor to spend — and the net saving is $54 M. It is only $54 M if the just-in-time path uses a fast preset; reuse the archival one and the compute exceeds the whole storage saving by 70%.

Server-side bitrate selection. Central control is genuinely appealing: one consistent policy, changeable without shipping a client, easier to reason about. It is rejected because the deciding state — buffer occupancy, instantaneous throughput, display size — lives on the client and moves faster than a 150 ms round trip plus telemetry aggregation can track. The server would be two control periods behind and blind to the two most important variables.

Two-second segments for video-on-demand. Shorter segments adapt faster, start faster, and behave better on a collapsing link. They are rejected on bits: three times as many keyframes is a 9.5% bitrate penalty, or $76 M a year, bought in exchange for switching latency that nobody watching a recorded video can perceive. They are right for live, where segment duration sets a floor on glass-to-glass latency — the delay from the camera lens to the viewer’s screen, which cannot be shorter than the time it takes to finish writing one segment.

One database row per media segment. It would be queryable, and a natural home for per-segment metadata. It is rejected at 300 million rows a day of pure naming, all of it information fully determined by duration and rung. Address segments by convention and generate the manifest arithmetically.

Perceptual dedup used as a storage mechanism. It has the one virtue exact hashing lacks: it actually catches re-encodes, which is what people mean when they say “duplicate”. It is rejected because two perceptually identical files may have different rights holders, different edits at the margins, and different owners, so collapsing them into one stored object is a legal problem rather than a storage saving. The perceptual pipeline exists, but its output is a claim, not a pointer.

16. Interviewer pushback

This design attracts seven questions, each answered here at interview length. The italic line under each names what the interviewer is actually testing, which is usually not the same as what they literally asked.

“Why 8 MB chunks?” Testing: whether the number was derived or remembered. From the failure rate of the link. On a 5 Mbps uplink dropping once every 500 seconds, a 720 MB file takes 1,152 seconds and expects 2.3 drops, so a single request finishes 10% of the time. Chunking trades two costs: about 150 ms of fixed per-chunk work for framing and the durable offset commit, against retry waste that grows with chunk size. Total time is a 1/S term plus an S term, so the optimum is where they are equal, and the closed form is S* = rate x sqrt(2h/p) — 0.625 MB/s times the square root of 150, which is 7.66, so 8. The curve is flat from 4 to 16 MB, so the precision does not matter, but the sensitivity does: quadruple the drop rate and the chunk halves. At 8 MB I expect 2.3 retries per upload and 1.3% of bytes re-sent, against 291% for the single-request design — and I would flag that the familiar 451% comes from assuming a failed attempt loses half the file, which is right for a chunk and wrong for a 1,152-second transfer, where the conditional mean is 32.3%.

“Walk me through the transcode fleet.” Testing: whether you can size compute from first principles. The ladder is five rungs and its total pixel count is 1.80x the top rung alone, because pixels roughly halve at each step down. At the archival preset a core does 1080p at 0.35x realtime, so 2.86 core-seconds per second of 1080p, times 1.80 for the ladder is 5.15, plus 0.10 for a single shared decode is 5.25 core-seconds per second of source. A 12-minute video is 3,780 core-seconds, or 1.05 core-hours. At 500,000 uploads a day that is 1.89 billion core-seconds of offered load; a 32-core box at 80% utilization supplies 2.21 million core-seconds a day of capacity, so 854 boxes exactly meet the mean. That is demand, not a fleet — a tier sized at the mean can hold a backlog level but never drain it — so I provision 1.5x, which is 1,281 boxes, and that is the number I would cost. I would not multiply by a peak factor on top, because transcoding sits behind a queue and a queue turns a spike into publish latency instead of machines; the 1.5x is there to buy the drain rate, with two priority lanes so a large channel is not stuck behind a backlog. Wall clock comes from the DAG: 24 work units of 30 seconds times five rungs is 120 independent tasks, and the longest is 30 seconds of 1080p at 2.86x, so 86 seconds instead of 63 minutes.

“Why not use GPUs? They are much faster at video.” Testing: whether you optimize the right bill. Because the transcode bill is not the bill. A GPU doing the whole ladder at 12x realtime is 60 GPU-seconds per video.

Two traps here, and I would name both unprompted. First the price: hardware encode is a T4 / L4 / L40S-class part at about $1.00 a GPU-hour, not the $2.00 or $2.50 you would quote for a training accelerator, because those parts have a hardware decoder and no hardware encoder — quoting their rate prices a capability this job never uses. Second the accounting: put both sides on the same footing or the answer swings by 2x. Counted as owned fleets at the same 80% utilization it is 435 GPUs at $3.8 M against 854 CPU boxes at $9.0 M, a $5.2 M saving. Counted as the 1,281-box CPU fleet I actually provision against GPU hours rented as consumed, it is $13.5 M against $3.0 M, a $10.4 M saving.

But fixed-function encoders need about 15% more bits for the same quality, and this system’s egress is $800 M a year, so that 15% is $120 M. The penalty is 11.5 to 23 times the saving depending on which accounting you picked — and the honest note is that at a training accelerator’s $2.00 the same two accountings give 16.3 and 88, so pricing the part correctly halves the range without touching the verdict. It cannot touch the verdict, because the saving cannot exceed the whole CPU bill and $120 M is still 8.9 times even that.

It is the read amplification doing it: at 306 bytes read per byte written, anything that trades output size for input cost loses by roughly that factor. I would use hardware encoding for live, where realtime is a hard deadline and the slow preset is not available anyway.

“You are storing 414 TB a day of renditions. What do you do about it?” Testing: whether you know the view distribution. Tier on it. Under a Zipf fit over 500 million videos the top 10 million take 80.5% of views and the other 490 million average 0.4 views a day each. Keeping the full ladder hot everywhere is 414 PB at $0.023 per GB-month, which is $114 M a year. Instead the head keeps everything hot and the tail keeps only 1080p and 360p in a cold class at a fifth the price. That is $13 M a year, saving $101 M.

I would be explicit that those are not the two most-played rungs — they carry 41% of playback seconds against 57% for 720p plus 1080p — because the criterion is derivability, not popularity. Everything below the top rung can be generated from it and nothing can be generated from below, so you keep the top rung because it is the only irreplaceable one, plus one cheap floor.

The missing rungs are generated on demand, and I would be careful about the divisor there, because it is where this gets quoted wrong. A JIT result is one rung of one video, not a video, so the reuse is 0.398 x rung share x 7 days, which is 0.95 for 720p and below that for everything else. Every one is under one, so there is effectively no reuse: the fleet is 4,510 boxes and $47 M, not the 1,616 and $17 M you get by dividing by the per-video 2.79. Net $54 M.

The detail I would insist on is the preset. JIT output is read about once before its cache expires, so it gets the fastest preset that still looks acceptable. Reuse the archival preset and the JIT fleet is 16,291 boxes and $171 M, which is 70% more than the entire storage saving. The preset should be a function of the expected read count.

“What cache hit ratio do you assume at the CDN, and why?” Testing: whether hit ratio is computed or wished for. I do not assume one. The head is 805 million views a day over 10 million videos across 50 PoPs, so 1.6 views per video per PoP per day, and a hit ratio near one. The tail is 195 million views over 490 million videos across 50 PoPs, which is 0.00796 views per video per PoP per day; even with a seven-day TTL one cached copy serves 0.0557 requests, so essentially every tail request is a miss. Blended that is 0.805.

Origin therefore ships 21.45 PB a day, or 1,986 Gbps. That is a daily mean of offered load, not a fleet: at a 2x prime-time peak and NICs planned at 80%, it is 4,965 machines doing nothing but feeding the CDN. I would not quote the 1,986 as a machine count, because dividing a mean by a NIC’s rated capacity is how you under-buy a tier by 2.5x.

The important part is that this is a first-access miss, not a capacity miss, so a bigger edge cache buys nothing. The fix is fewer independent caches — but say what a cache holds before quoting a number. Four regional shields with a 30-day TTL get 2.99 requests per cached video, which reads as a 66.6% tail hit ratio and $52 M a year saved. That is the optimistic bound.

A cached object is one segment of one rung, and per (video, rung) the same 2.99 becomes 1.02 for 720p and below one for every other rung, so the real tail hit ratio is about half a percent. The algebra is 0.398 x TTL / shields x rung share, so you need TTL / shields above 7.4 days to clear one request per copy at all, and this design sits at 7.5. That is the finding: the shield’s tail economics are a knife edge in TTL / shields, and the lever is fewer shields or a much longer TTL, not more of them.

“Explain adaptive bitrate. What does the server do?” Testing: whether you know where the control loop lives. Almost nothing. The server publishes a manifest — about 2 KB, static, identical for everyone, cacheable at the edge — listing the available rungs and their segment URLs. The client measures its own throughput and buffer occupancy and picks the next segment’s rung, on a per-segment cadence of six seconds.

It has to be the client. The deciding state is buffer level and display size, both of which live there and move on a 100 ms scale, while a server-side decider is 150 ms of round trip plus telemetry batching away — at least two control periods behind the thing it controls, and blind to the screen size.

The one thing the server absolutely must get right is keyframe alignment: every rung places IDR frames at the same presentation timestamps, or a mid-stream switch produces a visible gap. That constraint is why the transcoder forces keyframes onto a fixed grid instead of onto scene cuts.

Forcing them costs bitrate, which is what makes segment duration an economic choice rather than a taste one. Six seconds for on-demand. Two seconds costs 9.5% more bits, which is $76 M a year, to buy adaptation speed a recorded video does not need.

“Two people upload the same file. What happens?” Testing: whether you see the second-order problems. The bytes are stored once and both videos point at them, reference-counted so a delete decrements rather than unlinks. At an assumed 3% duplicate rate that saves 12.4 TB a day, 4.5 PB a year, and about 38 boxes off the 1,281-box transcode fleet.

Two cautions come with it. The upload must not complete faster because it is a duplicate — if it does, anyone can test whether a specific private file exists by uploading a copy, which is a membership oracle. Run the full upload every time and deduplicate behind it. And exact hashing only catches literal re-uploads, since a re-encode is a different byte string, so recall against what people actually mean by “duplicate” is zero.

That second case is the perceptual fingerprint pipeline, and I keep it separate on purpose. Its latency budget is hours rather than 86 seconds, so it cannot gate publication, and its output is a claim with legal consequences rather than a storage pointer.

Its threshold is set by appeals capacity, not by a validation curve. One percent false positives at 500,000 uploads a day is 5,000 claims, which at 60 appeals per reviewer per day is 83 people.

The assumption ledger

Every design is a set of assumptions with a diagram attached, and the diagram is only correct relative to them. Everything this chapter has assumed is collected here, so that you can state the design’s foundations in twenty seconds and say what replaces the design when each one fails.

Sort each assumption into one of three bins. State it means you are free to pick a number and being wrong costs you a re-derivation, nothing more. Ask it means the answer changes a policy or a threshold and is worth an interviewer’s time. Load-bearing means that if the assumption is wrong the design is not suboptimal, it is invalid — a box appears or disappears, rather than the count inside a box changing. The one-line test, from ch 03, is to move the assumption an order of magnitude in each direction and ask whether the set of boxes changes or only the number of machines inside them.

The sentence that makes this visible to an interviewer: “This design rests on four things. One, egress is three hundred times ingest — relax that and it stops being a delivery-network problem. Two, views are Zipf-distributed, which is what makes tiering and the shield tier pay. Three, hardware encoders cost about 15% more bits, which is the only number that could flip the software-encode decision and the one I would measure first. Four, publication is allowed to be asynchronous, which is what lets me size transcoding at 1.5x the mean instead of at peak.”

Cheat sheet

Every line below is derived somewhere above; this table is the recall test, not the explanation.

Related: the egress estimate extended here is 02 — Back-Of-The-Envelope; 05 — Consistent Hashing partitions the metadata and explains why a viral video is a hot key; ml/04 — Video Search owns retrieval over this corpus; ml/06 — Video Recommendation owns what appears in the feed.