18 — Design Nearby Friends

“Show me which of my friends are close by, live.”

What this chapter teaches

This problem looks like a map search where the pins move. It is not.

It is a presence system — the machinery that tracks who is online right now — with a coordinate bolted on. That reframing deletes almost every piece of storage machinery you would otherwise reach for.

By the end you will be able to:

• Price durability against freshness in one ratio, and say the number out loud.
• Decide whether to push updates to clients or let them pull, using a rule that has nothing to do with geography.
• Explain why the geospatial index this problem seems to demand is the wrong tool here.
• Say exactly how long it takes a user to become invisible, and why that number is the same number as the freshness budget.

Nothing here assumes you have read another chapter. Anything borrowed from one is restated in a sentence first.

What goes in and what comes out

Fix both flows before touching any mechanism: the input on top, the output underneath, each with one concrete example record.

in    a stream of location fixes, one per publishing user every ~30 s
      example:  {user_id: 42, lat: 40.7580, lng: -73.9855, accuracy: 12, ts: ...}

out   for each viewer with a map open, a stream of dots for the friends who
      share with them, each carrying a distance and an age
      example:  [{friend_id: 91, lat, lng, distance_m: 1840, age_s: 12}, ...]

Two words from that block, defined now because everything below leans on them.

A fix is one position reading from a device — one GPS sample, with a timestamp.

age_s is how many seconds ago that reading was taken. It looks like a debugging field. It is not: Deep dive 4 ttl staleness and adaptive intervals shows it is the single most important field in the response, because a dot on a map is only as useful as it is recent.

Which problem this actually is

This looks like chapter 17 with moving points. It is not. It is chapter 12’s presence system with a payload.

Chapter 17 builds a read-heavy index over restaurants that never move. Chapter 12 keeps tens of millions of phones connected and tracks which ones are still alive. This problem inherits its shape from the second, not the first.

The tell is in the numbers rather than the words. Three differences:

• A proximity service is read-heavy over a corpus that barely changes. This is write-heavy: every phone reports a new position every 30 seconds.
• A restaurant’s address is true for a decade. A location fix is worthless within a minute of being written.
• A proximity service picks recipients by radius. This one picks them by social graph — your friend list decides who sees you.

Nothing about the spatial index is the hard part here. Deep dive 3 the geospatial index you probably do not need shows you may not need one at all.

The four ways candidates lose this

1. They write every location fix to a durable store. That accumulates a million working sets a year for no benefit (Back of the envelope and why the database is the wrong answer).
2. They fan out to everyone in a radius instead of to the friend list (Deep dive 1 push pull and the ratio that inverts).
3. They optimize how accurate a dot is, when the interesting number is how stale it is (Deep dive 4 ttl staleness and adaptive intervals).
4. They treat “go invisible” as a switch in the client rather than something the server enforces (Deep dive 5 privacy and why the ttl is the mechanism).

What this chapter borrows and does not re-derive

Three results come from other chapters. You do not need to have read them — here is what each word means and where the derivation lives.

• Geohash and k-ring — a geohash is a short string naming a rectangle on the map, produced by repeatedly halving the world; a k-ring is the set of grid cells within k neighbour-steps of a starting cell. Both, plus every cell size quoted below, come from Deep dive 2 geohash derived.
• The connection tier — the fleet of machines that hold 20 million open sockets. Sized in Deep dive 1 the connection tier and why 3 boxes is 100; this chapter treats it as already paid for.
• The push/pull cost model — the rule for deciding whether a producer’s update is copied to every consumer up front or fetched on demand. Derived in The break even is posting rate not follower count and inverted in Deep dive 1 push pull and the ratio that inverts.

1. Framing: what decision, and what breaks

Two decisions carry the design, and each goes specifically wrong when answered the obvious way. The second one is what interviews are actually about.

Decision 1 — where does a location live?

If the answer is “a database”, the design is already wrong. Back of the envelope and why the database is the wrong answer prices exactly how wrong, in terabytes.

Decision 2 — who gets told, and when?

The recipient set is a friend list, not a disc. A disc is the set of everyone within some radius of a point, which is how chapter 17 frames every query.

That difference inverts the whole geospatial framing. You are not searching a map for who happens to be nearby. You are pushing to a named list of people who already know your identity, and each of them separately checks whether you are close to them.

Searching is a lookup over an unknown, unbounded set. Pushing is a loop over a list you already hold. Those need completely different machinery, and Deep dive 3 the geospatial index you probably do not need is where the difference gets priced.

What breaks if you get either one wrong

• Durable writes at the update rate accumulate a million working sets a year for zero value (Back of the envelope and why the database is the wrong answer).
• Naive fanout — sending every update to every friend — multiplies an already-large write rate by the friend count (Deep dive 1 push pull and the ratio that inverts).
• No expiry on a stored position means a dot that says a friend is two blocks away when they left an hour ago, which is worse than showing nothing, because a user acts on it (Deep dive 4 ttl staleness and adaptive intervals).
• Privacy enforced in the client means a modified client sees everyone (Deep dive 5 privacy and why the ttl is the mechanism).

2. Requirements

The functional list is ordinary. The five non-functional constraints are not: between them they delete the entire storage layer you would normally build.

Functional

• A user publishes location while sharing is on and the app is running.
• A user sees friends who share with them, with distance, on a map.
• “Nearby” highlights friends inside a radius — 5 km is the default.
• Sharing is per-friend, revocable, and instantly effective.
• Invisible mode: stop publishing, and disappear from everyone’s map.

Non-functional — these decide the design

Three terms in the table need defining first.

• TTL (time to live) — an expiry stamped on a stored value. After that many seconds the store deletes it on its own, without anyone asking.
• Write-ahead log — the durable append-only file a database writes before it acknowledges a change, so a crash can be replayed from it. It is what makes a write survive a power cut. It is also the thing this design deliberately does not have.
• Quorum — the rule that a write counts as done only once enough replicas have acknowledged it.

Each row pairs a cause on the left with its mechanical consequence on the right — something this design either builds or refuses to build.

NAT in the third row is network address translation, the box between a phone and the internet that rewrites addresses; it forgets an idle connection after a few minutes, which is why a client must send something periodically whether or not it has news.

“Losing a location is fine, showing a stale one is not” is the sentence the whole design follows from. It is the opposite of every storage chapter in this book, and it is why none of that machinery appears here.

3. Back of the envelope, and why the database is the wrong answer

The update rate derived here settles the chapter’s central question — whether locations are stored durably — with a single ratio.

3.1 The population and the update rate

The population numbers come from Back of the envelope so the two chapters cannot disagree. Two terms first: DAU is daily active users, the count of distinct people who open the app in a day. A concurrent connection is a phone holding an open socket to the service at a given instant — far fewer than DAU, because most of those people are not using the app right now.

The block below walks from users down to updates per second. Each line either states an assumption or does one multiplication on the line above it.

DAU                            500,000,000
concurrent connections         20,000,000
share with sharing enabled     50 %
concurrent publishers          20,000,000 x 0.5     =  10,000,000
update interval                30 s
location updates/s             10,000,000 / 30      =  333,333
peak, 2.5x average             333,333 x 2.5        =  833,333

The last two lines are the ones to memorize. Ten million phones each sending one fix every 30 seconds is 10,000,000 / 30 = 333,333 fixes per second. Real traffic is not flat across the day, so the busy hour is taken at 2.5x the daily average, giving 833,333 fixes per second.

3.2 Which rate gets spent where

Two rates just appeared, and the chapter has to declare which one it spends on what. This is the single easiest place to look rigorous and still be 2.5x wrong.

Three terms, and the distinction between them is the whole point:

• Offered load — the work that arrives per second. 333,333/s is the average offered load, spread over a day. 833,333/s is the peak offered load, the same figure during the busiest hour.
• Capacity — what a machine or a fleet can absorb per second. This is a property of the hardware, not of the traffic.
• Utilization — what fraction of its capacity you actually plan to use. A component run at 100% busy has no headroom, so any small burst queues, and the queue grows without bound. Real designs target something like 60%.

Neither 333,333 nor 833,333 is a capacity. Neither becomes one until a utilization target is written next to it.

The rule for the rest of this page, stated once so no derivation below has to argue it again:

Every fleet, core count and device count is sized on the 833,333/s peak. Every bytes-per-day and cost-per-year figure uses the 333,333/s average.

That split is not arbitrary. Hardware has to survive the worst second of the day, so it is bought against the peak. A storage bill accumulates over a year, so it is computed from the average.

3.3 What one location record costs

Before any total, price one record. The block below is a field-by-field byte count; the number that matters is the 28 at the bottom.

user_id      8 B
lat          4 B    int32 microdegrees, ~11 cm resolution
lng          4 B
recorded_at  8 B    server receive time, ms
accuracy     2 B    metres
flags        2 B    moving, battery-saver, precision level
                 -----
                  28 B

Microdegrees means the coordinate is stored as an integer number of millionths of a degree rather than as a floating-point number. One degree of latitude is about 111,320 m, so one microdegree is 111,320 / 1,000,000 = 0.111 m, roughly 11 cm. A 4-byte signed integer holds up to about 2.1 billion, and the largest coordinate you need is 180 degrees = 180,000,000 microdegrees, so 4 bytes is comfortably enough.

3.4 The ratio that settles it

Two numbers decide whether locations get a durable store. The working set is the data that has to be live in memory at any instant — here, one current position per publishing user.

live working set   10,000,000 x 28              =  280,000,000  B  =  280 MB

if every update is durably stored (average rate, for a per-year figure)
  bytes/s          333,333 x 28                 =  9,333,324    B/s
  bytes/day        9,333,324 x 86,400           =  806,399,193,600  B  =  806 GB/day
  bytes/year       806,399,193,600 x 365         =  294,335,705,664,000  B  =  294 TB
  at 3 replicas    294 x 3                      =  883    TB/year

Line by line: ten million live positions at 28 bytes each is 280 MB, which fits in the RAM of one ordinary machine. If instead you append every fix to disk, you write 333,333 records a second, which is 9.3 MB/s, which is 806 GB every day, which is 294 TB after a year — and 883 TB once you keep three copies for durability.

Now divide the second by the first.

ratio    294,335,705,664,000 B/year / 280,000,000 B   =  1,051,199  per year

Mind the units on that ratio. It is bytes accumulated per year divided by bytes live now, so what comes out is a rate — a per-year figure — not a dimensionless multiple.

Say it as “a year of history is a million working sets”, not “the history is a million times bigger than the working set”. The second sentence has no time unit in it and is only true for the particular window you happened to pick; pick a month and the number changes. The argument survives the correction unharmed, because the window that matters is the one a storage bill is written for, and that window is a year.

The entire live location of ten million people is 280 MB. A year of stored history is 294 TB — a million working sets — to preserve data that is wrong 30 seconds after it is written. That is the finding, and it is the sentence to say out loud before drawing anything.

3.5 The device-count argument, and why it does not survive

There is a second argument against durable writes, made from how many disks you would need rather than how many bytes. You will see it quoted often. It does not hold up, and knowing why is worth more than the argument itself.

Here is the version usually given. It takes a figure of 100,000,000 / 8,192 = 12,207 random 8 KB page writes per device per second, then divides the offered load by it:

the version usually quoted
  page writes per device      100,000,000 / 8,192   =  12,207
  devices                     333,333 / 12,207      =  27.3  ->  27
  with 3 replicas             27 x 3                =  81

Both halves of that are wrong.

The 12,207 is derived from a queue-depth-1 number — one request in flight at a time — which The two rows that will burn you flags explicitly as a latency measurement being misread as a device capacity. A real drive is fed many requests at once and goes far faster.

And the load figure is the average, divided at 100% utilization, which §3.2 just ruled out on both counts.

Redo it against a real device and against the peak:

random 4 KB IOPS one NVMe supplies at depth (ch 02)      500,000
an 8 KB page is two of those, so 8 KB writes/s -- CAPACITY
  500,000 / 2                                         =  250,000
peak offered load, from above                            833,333
device utilization we would design to                        0.6
usable capacity per device        250,000 x 0.6       =  150,000
devices for the write path        833,333 / 150,000   =  5.6  ->  6
with 3 replicas                   6 x 3               =  18

Three terms from that block. IOPS is input/output operations per second. An NVMe drive is a modern solid-state disk attached directly to the PCI bus. “At depth” means many requests are in flight at once, which is how such a device reaches its rated throughput instead of its one-at-a-time throughput.

Eighteen devices, not eighty-one. And even eighteen is a worst case, because the calculation assumes one random page write hits the disk for every fix.

Real storage engines do not work that way. An LSM store — a log-structured merge tree, the design behind Cassandra and RocksDB — buffers writes in memory and flushes them out as large sorted files, so many logical writes become one sequential disk write. Check what that costs in bandwidth rather than in operations:

logical bytes/s at peak     833,333 x 28        =  23,333,324  B/s
write amplification         10x
physical bytes/s            23,333,324 x 10     =  233,333,240 B/s  =  233 MB/s
one NVMe sequential                                1,000        MB/s
fraction of one device      233 / 1,000         =  23 %

Write amplification is the ratio of bytes the storage engine actually writes to disk against the bytes you handed it, inflated by the later merge passes that rewrite the same data. Ten times is a normal figure for an LSM store.

So the peak fits in 23% of one device’s sequential bandwidth. At the average rate it is 93 MB/s, under a tenth of one device — but this chapter’s rule is to size on the peak, so 23% is the number to quote.

Say it plainly: the device leg of this argument collapses, and it was always the weakest one.

Two legs survive:

1. The ratio. 294 TB a year against a 280 MB working set — a million working sets a year — to keep data that is wrong 30 seconds after it is written.
2. The coupling, and this is the one to lead with. A durable write puts a replicated commit on the critical path of a fix that will be superseded before anyone reads it. You pay latency and gain a new thing that can fail, in exchange for preserving a value with a 30-second shelf life.

3.6 Bandwidth is a non-issue, and say so

Network is the one resource nobody needs to worry about here, which is worth stating explicitly so an interviewer knows you checked. Convert the byte rate to bits (8 bits per byte) and compare against a network card:

ingest bits/s, at the average
  9,333,324 x 8                                 =  74,666,592     =  74.7 Mbps
at the 2.5x peak   74.7 x 2.5                                     =  186.7 Mbps

A NIC is the network card on a machine, and a normal one moves 1 Gbps = 1,000 Mbps. Peak ingest of 186.7 Mbps is 186.7 / 1,000 = 19% of a single NIC — for the entire firehose of ten million phones.

The bytes were never the problem. The durability was.

4. API sketch

WS below is a WebSocket: a connection that stays open and lets both sides send at any time, rather than one request getting exactly one response. POST and GET are ordinary HTTP calls.

Read the first two lines together — they are one socket carrying traffic in both directions. Everything after them is a control-plane call that happens rarely.

WS   /v1/location            client -> {lat, lng, accuracy, ts, moving}
                             server -> {friend_id, lat, lng, distance_m, age_s}[]

POST /v1/sharing             {friend_id, mode: precise | city | off}
POST /v1/visibility          {mode: on | ghost | off}
GET  /v1/friends/nearby?radius_m=5000
                             -> {friends: [{id, distance_m, age_s, precision}]}
GET  /v1/sharing/audit       -> {viewer_id, last_seen_at}[]

Three commitments are baked into that shape, and each one removes a class of client bug.

age_s is returned with every dot. The client cannot draw a stale position as a fresh one, because it is holding the staleness in its hand. If the field were absent, every client would have to guess, and every client would guess differently.

precision is returned. A city-level dot cannot be rendered as a street-level pin, because the client is told which one it received.

One socket carries both directions. Publishing and receiving share the same connection. Opening a second socket would double a fleet already sized at 20 million connections, and buy nothing.

5. Data model

5.1 The three pieces of state

Three stores, and the whole design is visible in their annotations. Look at the middle word on each first line: only one of the three says “durable”.

location_live        in memory, sharded by user_id, TTL 60 s
  user_id  ->  {lat, lng, recorded_at, accuracy, flags}

sharing_edges        durable, sharded by owner_id
  owner_id, viewer_id, mode, created_at        the authorization set

subscriptions        in memory, sharded by publisher_id, TTL 120 s
  publisher_id  ->  {viewer_id: gateway_id}    who is watching right now

Sharded by means the rows are split across machines by hashing that column, so every machine owns a disjoint slice of the keys and no key lives in two places.

What each one is for:

• location_live — one slot per publishing user, holding their current position. This is the only thing that moves at 333,333/s, and it is the only thing with no durability. Written by ingest, read at fanout.
• sharing_edges — the authorization set: who is allowed to see whom, and at what precision. It changes a few times per user per year, so it is a small, boring, heavily cached relational table.
• subscriptions — who currently has a map open on whom, and which gateway machine to send their dots to. Rebuilt from scratch every time someone opens the app, so it does not need durability either.

5.2 Shard location_live on user_id, not on cell

The obvious alternative is to give each shard a region of the map, so a shard owns everyone currently standing inside its cells. That is how a proximity service does it, and here it is a mistake.

The reason: users move, and cells do not. When a driver leaves one cell and enters the next, their record has to be handed from one machine to another. Price how often that happens, using the precision-6 cell height of 610 m from Bits per character and where the cell sizes come from and a driving speed of 30 m/s (about 108 km/h):

cell crossings for a driver    610 / 30 m/s     =  20.3 s
share of publishers driving    20 %
publishers driving             10,000,000 x 0.2 =  2,000,000
shard migrations/s             2,000,000 / 20.3 =  98,522

Read that as: a driver crosses a cell boundary every 20.3 seconds. Two million publishers are driving at any moment. So two million migrations happen every 20.3 seconds, which is 98,522 per second.

Sharding on cell turns one in five publishers into a cross-shard move every twenty seconds, for 98,522 migrations per second of pure overhead. None of that work produces an answer for anyone. It exists only to keep the shard map consistent with the physical world.

Sharding on user_id makes the same update a single in-place write to a fixed owner. A user’s home shard never changes no matter where they physically go, so there is nothing to migrate.

Use the ch 05 consistent-hashing ring on user_id — a scheme that maps both keys and machines onto a circle, so that adding a machine moves only a 1/n slice of the keys instead of reshuffling everything.

6. High-level architecture

Connected together, the pieces make one argument, and it lives in a single edge: fanout goes only to people currently looking.

6.1 The diagram

Seven boxes. Trace the arrows left to right: a fix comes in from a phone, gets authorized, gets stored, gets published, and comes back out through the gateway. The two shaded boxes are the only ones that own data — everything else is stateless plumbing.

flowchart LR
    M["Mobile clients<br/>20 M sockets"] --> GW["Gateway tier<br/>ch 12 section 7"]
    GW --> IN["Location ingest<br/>authorize, rate limit, TTL"]
    IN --> LS["location_live<br/>sharded by user_id, in RAM"]
    IN --> PS["Pub/sub by publisher_id"]
    PS --> GW
    SUB["Subscription registry<br/>who is watching whom"] --> PS
    GW --> SUB
    SE[("sharing_edges<br/>durable")] --> IN

    style LS fill:#1d3557,color:#fff
    style SE fill:#1d3557,color:#fff

6.2 What each box is

Each of these is referred to by name later, so read them once now.

• Mobile clients 20 M sockets — the phones. Each holds one open WebSocket that carries fixes up and dots down.
• Gateway tier ch 12 section 7 — the fleet of machines that terminate those sockets. Inherited wholesale from Deep dive 1 the connection tier and why 3 boxes is 100 rather than designed here: 100 boxes at 200 k connections each. That count comes from blast radius — how many users one machine failure takes down — not from how much RAM a socket needs.
• Location ingest — authorizes the fix, applies the rate limit, and writes it with a TTL. Fully specified in Deep dive 6 the ingest path and the queue of depth one.
• location_live sharded by user_id, in RAM — the single in-memory slot per user from Data model.
• Pub/sub by publisher_id — publish/subscribe is a message bus where senders publish to a named topic and interested parties subscribe to that topic, with neither side knowing the other exists. Here the topic name is the publisher’s user id, so “subscribing to Alice” literally means subscribing to topic alice_user_id.
• Subscription registry who is watching whom — the in-memory table recording which viewers currently have a map open on which publishers, and which gateway machine each viewer is attached to.
• sharing_edges durable — the one durable store, holding who is allowed to see whom.

6.3 What the colours mean

The colours follow ch 01’s key. The interesting column is the third one, where three of the four rows say “Nothing”.

Two blue boxes and nothing else coloured is the chapter’s thesis rendered as a picture.

sharing_edges is the one durable store. location_live is the one volatile store — and per Data model it is the only copy of a location that exists anywhere, which makes it authoritative rather than a cache, however much the TTL makes it look like one. A cache has something to fall back to. This does not.

The other three boxes are uncoloured because none of them owns a value. Ingest is stateless. The pub/sub bus forwards and forgets. The subscription registry holds routing state that the next map-open rebuilds from scratch, so losing it costs one round trip and nothing more.

6.4 The diagram as one sentence

A location fix enters through the socket it will leave through, is authorized against a durable edge set, lands in a TTL’d memory cell, and is republished only to viewers who are currently looking.

That last clause is the argument the whole architecture exists to make, and Deep dive 1 push pull and the ratio that inverts prices it.

7. Deep dive 1: push, pull, and the ratio that inverts

The standard push-versus-pull rule from the news feed problem points the opposite way here, and the fix that follows turns out to save exactly the amount the rule said push would cost.

7.1 The rule, and why the friend count is irrelevant to it

Two words first. Push (or fan out on write) means: when a producer writes something, immediately copy it to every consumer who cares. Pull (or fan out on read) means: store the write once, and let each consumer fetch it when they look.

The break even is posting rate not follower count derives the general result. Take one producer with F consumers, a producer write rate of w, and a consumer read rate of r:

• Push does F copies per write, so it costs F x w per producer.
• Pull does F fetches per read, so it costs F x r per producer.

Compare them by dividing: (F x w) / (F x r) = w / r. The F appears on both sides and cancels.

So the follower count does not decide push versus pull. The producer’s write rate against the consumer’s read rate does. That result is counter-intuitive enough that it is worth being able to re-derive in two lines, because the instinct in an interview is always to reach for the fan-out number.

The mechanism is identical here. Only the two rates change, and they change by two orders of magnitude.

7.2 The two rates, and the direction they point

Compute w and r per sharing user per day. The first line is the one to slow down on: it converts concurrent publishers into seconds spent publishing per user per day.

Ten million of five hundred million users are publishing at any instant. So an average user is publishing 2% of the time, which is 2% of 86,400 seconds in a day:

sharing seconds/day    10,000,000 / 500,000,000 x 86,400   =  1,728
updates/day    w       1,728 / 30                          =  57.6
map opens/day  r       3
w / r                  57.6 / 3                            =  19.2

So an average sharing user publishes for 1,728 seconds a day — about 29 minutes — and at one fix every 30 seconds that is 57.6 writes. They open the map 3 times.

A location publisher writes 19.2 times for every read. A news-feed author writes once for every 25 reads. Same rule, opposite answer. Side by side:

7.3 Why the answer is neither push nor pull

The rule says pull, and pull here is unusually cheap. The viewer’s friend set is small and bounded, and one lookup per friend from an in-memory hash map is 100 ns (ch 02).

But pure pull means the client has to poll — ask again and again — and a map that must feel live means polling every few seconds, over a socket you are already holding open. That is push with extra steps and worse latency.

The resolution is the one ch 12 reaches for presence, arrived at from the other side: push, but only along edges someone is actually looking at. Call it subscribe-on-view. Opening the map subscribes you to your friends’ topics; closing it lets the subscription lapse.

7.4 Pricing naive push against subscribe-on-view

First, how many of a viewer’s friends are even publishing at a given moment. Mean friend count is 200, and a given friend is publishing only if they are both connected and sharing:

P(friend publishing)   20,000,000 / 500,000,000 x 0.5   =  0.02
live friends on a map  200 x 0.02                       =  4

Four live friends per open map. That “4” is used in both halves of what follows.

Now the two schemes. The top half is naive push: every fix goes to every live friend. The bottom half is subscribe-on-view: a fix goes only to friends with a map open on you.

naive push to all live friends
                       333,333 x 4                      =  1,333,332  sends/s

subscribe on view
  concurrent viewers   500,000,000 x 3 x 30 / 86,400    =  520,833
  subscriptions        520,833 x 4                      =  2,083,332
  watchers/publisher   2,083,332 / 10,000,000           =  0.2083
  sends/s              333,333 x 0.2083                 =  69,444
  reduction            1,333,332 / 69,444               =  19.2

Walk the concurrent viewers line, since it is the only non-obvious one. It assumes a 30-second session: each of 500 million users opens the map 3 times a day for 30 seconds, which is 90 seconds of viewing out of 86,400, so at any instant 500,000,000 x 90 / 86,400 = 520,833 people are looking.

Each of those viewers watches 4 live friends, giving 2,083,332 live subscriptions. Spread across 10 million publishers, the average publisher has 0.2083 people watching them — less than one. That is the whole trick: most publishers are being watched by nobody, and naive push sends to all four of their live friends anyway.

The saving is 19.2x — exactly the write:read ratio from §7.2, and that is not a coincidence.

Here is why the two numbers are the same. Naive push pays publishers x F_live. Subscribe-on-view pays viewers x F_live. Divide, and F_live cancels, leaving publishers / viewers — and publishers-to-viewers is the write-to-read ratio of the population, just counted per-instant instead of per-day.

Subscribe-on-view converts a write-heavy fanout into a read-heavy one, and the exchange rate is the very number that made push wrong.

7.5 What the fanout actually costs

Egress is the bytes leaving the service. After the 19.2x reduction, at 28 bytes per record and 8 bits per byte:

push bits/s            69,444 x 28 x 8                  =  15,555,456   =  15.6 Mbps

at the 2.5x peak
  sends/s              69,444 x 2.5                     =  173,610
  push bits/s          173,610 x 28 x 8                 =  38,888,640   =  38.9 Mbps

Fifteen megabits per second at the average, 39 at peak, to keep half a million live maps updated. The fanout was never expensive once it was pointed at people who were looking.

8. Deep dive 2: how nearby is “nearby”, and what the map actually shows

How many friends will the feature actually have to show? This is the estimate most candidates never compute, and the answer is small enough that it changes the product rather than the system.

8.1 Expected friends within 5 km

Friendship is geographically clustered, but not tightly. Model it in two steps.

Step one: how many of your friends are in your city at all. Assume 30% are in the same metro area.

Step two: given that they are in your metro, what is the chance they are within 5 km of you. Model the metro as a 20 km disc with friends spread evenly across it. Then the chance is just the ratio of the two areas:

5 km disc area    pi x 5^2    =  78.54  km^2
20 km disc area   pi x 20^2   =  1,256.6 km^2
ratio             78.54 / 1,256.6  =  0.0625     (the pi cancels: 25/400)

Multiply the two steps together, then by the 4 live friends from Deep dive 1 push pull and the ratio that inverts:

P(friend within 5 km)   0.30 x 5 x 5 / (20 x 20)     =  0.01875
live and within 5 km    4 x 0.01875                  =  0.075

Expected number of friends both live and within five kilometres: 0.075.

That is an average, so it does not directly tell you how often the list is empty. Treat the friends as independent and the count as Poisson — the standard model for “rare independent events, known average” — and the chance of seeing zero is e^-0.075 = 0.928.

So 93% of sessions show an empty “nearby” list: thirteen out of fourteen.

8.2 Three consequences, all of them design decisions

None of these is a complaint about the estimate. Each is something you change because of it.

• The default view cannot be “nearby”. It must be “all friends sharing with you, sorted by distance” — 4 dots, mostly far away — with nearby as a highlight. A feature that is empty 93% of the time gets deleted.
• The read path is trivial and the write path is everything. 0.075 relevant results per query means no ranking, no pagination, no candidate generation. Every engineering minute belongs upstream.
• “Recently nearby” is what makes it work. Widen the time window rather than the radius: a friend who was within 5 km in the last hour is the useful signal, and it is a different data structure — a short per-user ring buffer, a fixed-size array that overwrites its oldest entry, holding cell ids rather than a live dot.

8.3 Sizing the “recently nearby” buffer

The buffer stores a cell id — the precision-6 geohash from ch 17 — rather than a coordinate, because “which 610-metre box were they in” is all this feature needs.

One entry per fix, one hour of history, for every publisher:

retention              3,600 s
entries per user       3,600 / 30                    =  120
bytes per entry        8 B cell id + 4 B timestamp   =  12
bytes per user         120 x 12                      =  1,440
all publishers         10,000,000 x 1,440            =  14,400,000,000  B  =  14.4 GB

An hour is 3,600 seconds, one fix every 30 seconds gives 120 entries, at 12 bytes each that is 1,440 bytes per user, and ten million users make 14.4 GB.

Fourteen gigabytes buys an hour of “recently nearby” for everybody. That is the feature that makes the product non-empty, and it is still an in-memory structure with a TTL — no new storage tier appears.

9. Deep dive 3: the geospatial index you probably do not need

Here is the chapter’s central result: of the two ways to compute the answer set, the one everybody reaches for is both slower and a privacy hazard.

A geospatial index is any structure that lets you ask “what is near this point” without scanning everything — the grids and trees derived in ch 17. Here is why this chapter mostly does not want one.

9.1 Two ways to compute the same answer set

A viewer’s answer set is friends INTERSECT nearby — the people who are both your friends and close to you. Set intersection does not care which side you start from, so there are two orders to do it in, and they cost wildly different amounts.

The asymmetry that decides it: the friend set is bounded at 200. The nearby set is not bounded at all — it is however many people happen to be standing in a 5 km circle, which in Manhattan is a lot.

friend-first    200 hash lookups x 100 ns                    =  0.00002 s  =  20 us
                then 200 exact distance tests, ch 17 section 11

cell-first      k-ring lookup, ch 17 section 10.2, returns
                every publisher in the disc, then intersect with 200 friends

Friend-first walks your 200 friends, looks each one up in location_live, and measures the distance. 200 lookups at 100 ns each is 20,000 ns, which is 20 microseconds.

Cell-first asks the spatial index for everyone in the disc, then throws away the ones who are not your friends. The cost of that depends entirely on how many people are in the disc.

9.2 How many strangers is “everyone in the disc”

This needs a publisher density — publishers per square kilometre — and it is worth deriving rather than borrowing. Ch 17’s 50/km^2 is a density of businesses, a completely different population that happens to land in a similar range.

Build it from this chapter’s own figures, in the densest place the product plausibly runs:

product penetration    500,000,000 DAU / ~5 B internet users  =  0.10
Manhattan DAU          1,600,000 residents x 0.10             =  160,000
concurrent share       20,000,000 / 500,000,000               =  0.04
sharing share                                                    0.50
concurrent publishers  160,000 x 0.04 x 0.5                   =  3,200
Manhattan area (ch 17)                                           59.1  km^2
publisher density      3,200 / 59.1                           =  54.1  per km^2

The chain: one in ten internet users has the app, so 1.6 million Manhattan residents give 160,000 DAU. Of those, 4% are connected at any instant and half of those share, leaving 3,200 concurrent publishers spread over 59.1 km^2 — about 54 per km^2.

Call it 50/km^2. That it matches chapter 17’s business density is a coincidence of magnitude, not a reuse of the figure.

Multiply by the area of a 5 km disc, pi x 5^2 = 78.54 km^2:

strangers in the disc  78.54 x 50                             =  3,927

Roughly four thousand strangers fetched, distance-tested and discarded, to find at most four friends.

9.3 The verdict, and the rule behind it

Compare the two directly: 3,927 / 200 = 19.6, so cell-first touches about twenty times as many records.

Friend-first is 20 microseconds and returns exactly the right set. Cell-first touches 3,927 candidates where friend-first touches 200 — twenty times the work to reach the same answer — and it pulls the positions of thousands of strangers into a service that has no right to them.

The second half of that matters as much as the first. Even if the extra work were free, cell-first makes a component that never needed stranger data into one that handles it constantly.

The rule generalizes and is worth stating as one:

Use a spatial index when the candidate set is unbounded. Use the relationship when it is bounded. A proximity service has no relationship to filter on, so it must index space. Nearby friends has a 200-element filter sitting right there.

Two places the index does come back:

• Friend-of-friend discovery, or “people near you” — an unbounded candidate set, so the k-ring from The k ring and the radius it actually covers applies unchanged. Note that this is a different product with a different privacy posture, and saying so is worth a mark.
• Very large follower graphs. At F = 5,000 the friend-first scan is 5,000 x 100 ns = 500 us, still fine; at F = 100,000 it is 10 ms and you index instead. The crossover is set by F, not by geography.

10. Deep dive 4: TTL, staleness, and adaptive intervals

A location stops being true the instant it is taken. What it has instead is an error that grows with time and with speed. That turns the TTL into a distance budget.

10.1 The TTL in metres

First, how old a dot can get before it is deleted.

Declare a dot dead after two missed updates. If updates come every 30 s, a dot is deleted somewhere between 30 and 60 seconds after the last fix, depending on where in the cycle the user went quiet — so its age at expiry is uniform in [30, 60] s. That is the same [h, 2h] window Deep dive 6 presence where the heartbeat interval comes from derives for heartbeats, where h is the interval and you allow exactly one miss before declaring death.

Take the worst case, 60 s, and multiply by speed to get distance. Speeds in metres per second: 1.4 m/s is a walk, 5 m/s a bicycle, 30 m/s a car on a motorway.

walking   1.4 x 60      =  84     m
cycling   5 x 60        =  300    m
driving   30 x 60       =  1,800  m

At the moment a dot expires it is 84 m wrong for a pedestrian and 1.8 km wrong for a driver, from the same 60-second TTL. One interval cannot serve both. Eighty-four metres is the wrong side of a street; 1.8 km is the wrong side of a city.

10.2 Why you cannot just update faster

Invert the calculation. Pick an error budget and solve for the interval it demands:

error budget            50 m
walking interval        50 / 1.4      =  35.7  s
driving interval        50 / 30       =  1.7   s

A pedestrian can be 36 seconds stale and stay inside 50 m. A driver has 1.7 seconds.

Price that 1.7 s interval against the rate from Back of the envelope and why the database is the wrong answer, keeping the 20% driving share from Data model:

drivers                 10,000,000 x 0.2         =  2,000,000
driver updates/s        2,000,000 / 1.7          =  1,176,471
everyone else           8,000,000 / 30           =  266,667
total                   1,176,471 + 266,667      =  1,443,138
vs the 30 s baseline    1,443,138 / 333,333      =  4.33

Two million drivers each sending every 1.7 s is 1.18 million fixes per second on their own — more than three times the entire current system — and the other eight million users add 266,667 on top.

Bounding a driver’s error at 50 m costs 4.3x the entire ingest rate, and drivers are only one fifth of the population. You do not buy that.

10.3 Send velocity instead, and let the client predict

Add the user’s velocity to each fix and let the client fill in the gaps itself:

velocity fields         vx, vy as int16 cm/s     =  4 B
record                  28 + 4                   =  32 B

Four extra bytes on a 28-byte record is a 14% payload increase, and it buys the client the ability to dead-reckon: advance the last known position along the last known velocity as time passes, instead of leaving the dot frozen where it was last seen.

Dead reckoning is exact on a straight road and wrong the moment the road turns. So pair it with an event-driven send: transmit immediately when heading changes by more than 30 degrees, and treat the timer as a floor rather than as the schedule.

How often does that fire? Estimate turn frequency in two environments and blend them, assuming 80% of driving is urban:

urban   turn every 400 m at 10 m/s      400 / 10     =  40   s between events
highway turn every 5,000 m at 30 m/s    5,000 / 30   =  167  s between events
blended 0.8 / 40 + 0.2 / 167                          =  0.0212  events/s/user
30 s timer                              1 / 30       =  0.0333
reduction                               0.0333 / 0.0212  =  1.57

The blend is done in rates — events per second — not in intervals, because rates are what add. Turning every 40 s is 0.025 events/s, turning every 167 s is 0.006 events/s, and 80/20 of those is 0.0212. Against the fixed timer’s 0.0333 events/s, that is a 1.57x reduction for moving users.

10.4 The other end: users who are not moving

GPS jitter is the small random wobble between successive satellite fixes of a phone that is sitting still. It makes a parked phone report a brand-new position every 30 seconds, forever, all of it noise.

Fix it with a displacement gate: suppress the send unless the device has actually moved past a threshold distance. Fall back to a bare keepalive — a packet with no payload, sent only to prove the connection is alive — every 180 s, which is the interval ch 12 derives as enough to keep a NAT entry from being forgotten.

Assume 70% of publishers are stationary at any instant:

stationary share        70 %
stationary ops/s        7,000,000 / 180          =  38,889
moving ops/s            3,000,000 / 30           =  100,000
total ops/s             38,889 + 100,000         =  138,889
reduction               333,333 / 138,889        =  2.40
bytes/s                 38,889 x 12 + 100,000 x 28   =  3,266,668
byte reduction          9,333,324 / 3,266,668    =  2.86

Seven million stationary users on a 180 s keepalive contribute 38,889 ops/s instead of 233,333. Three million moving users still send every 30 s. Total drops from 333,333 to 138,889, a 2.40x cut in operations.

The byte cut is larger — 2.86x — because a stationary user’s keepalive costs only 12 bytes, not the full 28. There is no coordinate in it.

10.5 Do not multiply the two savings

Both reductions are measured against the same 333,333/s baseline, so they do not compose by multiplication.

1.57 x 2.40 = 3.77 is not a number this design produces. Quoting it — or bolting an invented extra factor onto the 2.40 to reach some rounder claim — is the standard way a chain of savings gets double-counted, and an interviewer who has done the arithmetic will catch it.

Compose them the only way that is meaningful: apply each rule to the population it actually governs. Movers send on events. Everyone else sends a keepalive.

moving users, event-driven   3,000,000 x 0.0212   =  63,600
stationary, 180 s keepalive  7,000,000 / 180      =  38,889
total ops/s                  63,600 + 38,889      =  102,489
reduction                    333,333 / 102,489    =  3.25

Three and a quarter times fewer updates for two client-side rules, and they cost the server nothing — both tests run on the phone, before anything is sent.

The general shape to carry away: the update rate should be a function of the world, not of a clock.

11. Deep dive 5: privacy, and why the TTL is the mechanism

Four properties, and each has to be enforced somewhere specific. None of them needs a component that is not already in the design — and the freshness budget doubles as the strongest of them.

11.1 Authorize at fanout, not at render

The sharing_edges check runs in the ingest path, when the server decides who to publish a fix to.

The consequence is the whole point: a viewer’s client never receives a coordinate it is not entitled to. There is nothing on the device to un-hide. A modified client, a patched app, a proxy sniffing the socket — none of them gains anything, because the bytes were never sent.

That is the difference between a privacy feature and a privacy setting. A setting asks the client to behave. A feature makes misbehaving impossible.

11.2 Going invisible in bounded time

Invisible mode does three things, in order:

1. Stop publishing.
2. Drop the subscription registry entries, so nothing is routed to viewers.
3. Delete the location_live cell.

Now assume step 3 fails — the shard is unreachable, the delete is lost. Every dot already drawn on a viewer’s screen still disappears within 60 s, because each dot carries age_s and the client refuses to render anything past the TTL.

worst-case time to disappear   the TTL      =  60 s

The TTL that exists for freshness is also the privacy backstop, and that is the strongest argument for having one at all. A system with no TTL has no bounded time-to-invisible — only a promise that a delete went through.

11.3 Reduce precision on the server, by truncating

“City-level” sharing means the server replaces the coordinate with the centre of a coarse cell before it is published. The client never sees the precise value.

The table maps each sharing mode to a geohash precision from Bits per character and where the cell sizes come from. Read the last column: it is what the viewer can actually conclude about you.

Truncate, do not add noise. This is the one place where the obvious alternative is actively broken.

Noise is averageable. Add a random offset to each fix and a viewer who collects 100 fixes of a stationary user can average them; the random offsets cancel and the true position emerges. Watching for an hour defeats it entirely.

Truncation is idempotent — applying it twice gives the same answer as applying it once. The cell centre is the same value every single time, so a hundred observations tell a viewer exactly what one observation told them.

That distinction is the answer to “why not just add jitter”, and shipping the jitter version is a mistake real products have made.

11.4 Make observation observable

GET /v1/sharing/audit returns who has seen your location and when.

It costs one row per (viewer, publisher) pair per day, and it is the only mechanism that lets a user verify the other three properties rather than trust them.

12. Deep dive 6: the ingest path, and the queue of depth one

Three things happen to every one of the 333,333 fixes per second — 833,333 at peak, and the peak is what the tier is sized on. Each of the three has an obvious default, and each obvious default is wrong here, for a reason specific to data with a short shelf life.

12.1 Authorization: move the check off the hot path

The naive placement is one sharing_edges lookup per fix. That is 333,333 reads/s against a table whose rows change a few times per user per year. You would be re-asking the same question 57 times a day for every user.

Move the check to subscription creation instead, and cache the answer inside the subscription entry:

map opens/s              500,000,000 x 3 / 86,400   =  17,361
subscriptions created    17,361 x 4                 =  69,444
reduction                333,333 / 69,444           =  4.80

Half a billion users opening the map 3 times a day is 17,361 opens per second, and each open creates 4 subscriptions (one per live friend), so 69,444 authorization checks per second instead of 333,333.

A 4.8x cut, and the check now runs against a cached row rather than on the hot path.

Notice that 69,444 is the same number as the send rate in Deep dive 1 push pull and the ratio that inverts. That is not a coincidence: both quantities are total live subscriptions / session length, so authorization runs exactly once per delivered stream, which is the correct frequency.

One catch. If authorization is cached, revoking access must not wait for that cache to expire. So revocation deletes the matching subscription entries directly rather than letting the 120 s TTL lapse, and the exposure window becomes one propagation hop — the time for the delete to reach the shard, milliseconds — instead of 120 seconds.

12.2 Rate limiting: the 30-second interval is only a client-side promise

Nothing forces a client to honour the 30 s interval. A modified or simply buggy client can publish at 100 Hz. Price 1% of the population doing that:

abusive publishers       10,000,000 x 0.01          =  100,000
their update rate        100,000 x 100              =  10,000,000
vs the whole system      10,000,000 / 333,333       =  30

A hundred thousand clients at 100 updates per second is 10 million updates per second, against a system designed for 333,333.

One percent of clients misbehaving is 30x the entire designed load. Which means the rate limiter is not a nicety; without it, a bad app release is an outage.

Use a per-user token bucket — a counter refilled at a fixed rate, where each request spends one token, so a steady rate passes through and a burst is capped at the bucket size (Token bucket). One update per 5 s with a burst of 5 caps the abuse.

It costs nothing, because the counter lives in the same shard as the user’s location. No extra store, no extra round trip.

12.3 Backpressure: the only correct queue depth is one

Backpressure is what a service does when work arrives faster than it can process it. The normal answer is to buffer, absorb the burst, and catch up.

Do not do that here.

Fixes are not events. They are successive estimates of a single value, so an older fix carries no information that a newer one lacks. Buffering them preserves exactly the data you no longer want.

Price a modest buffer:

queue depth              10 fixes
age of the head          10 x 30                    =  300  s
vs the TTL               300 / 60                   =  5

Ten queued fixes at 30 s apart means the head-of-line item — the oldest one, the one processed next — was recorded 300 seconds ago. The TTL is 60 seconds. So that fix is five TTLs past its own expiry date.

A ten-deep queue delivers a fix five TTLs stale. The system would be carefully processing data it is contractually obliged to discard.

The correct structure is a single slot per user that a newer fix overwrites. That makes backpressure a no-op: under load you lose intermediate positions and keep the latest one, which is exactly the degradation the product wants.

12.4 CPU, and what the tier is actually sized by

The last number settles what the ingest tier costs. Add up the per-fix work, then apply the average and the peak:

per fix   parse 2 us + token bucket 0.1 us + slot write 0.2 us   =  2.3  us
core-seconds/s demanded, at the average offered load
          333,333 x 0.0000023                                    =  0.77
the same at the 833,333/s peak
          833,333 x 0.0000023                                    =  1.92
cores of CAPACITY at 60% utilization
          1.92 / 0.6                                             =  3.2

At the peak, 833,333 fixes a second at 2.3 microseconds each demands 1.92 core-seconds of work per second. A core you plan to run at 60% supplies 0.6 core-seconds per second, so you need 1.92 / 0.6 = 3.2 cores.

A little over three cores for the entire application-level ingest of ten million publishers.

You will often see this quoted as “0.77, so under one core”. That version uses the average load and assumes a core running 100% busy — it is a demand figure wearing a fleet’s clothes, exactly the mistake §3.2 rules out. Three cores is the same conclusion counted honestly, and it is still nothing.

The real point: the ingest tier is not sized by the location work at all. It is sized by the sockets in front of it — the 100 boxes at 200 k connections each from Deep dive 1 the connection tier and why 3 boxes is 100, which was already paid for.

13. Bottlenecks and scaling

Read the middle column first — it is the size of the problem. The right column is what you do about it, and for two of the six rows the answer is “nothing, by construction”.

Two footnotes on that table.

The 200 bytes per entry in the memory row is the 28-byte record plus the per-key bookkeeping any hash map adds — pointers, hash slots, allocator rounding. Seven times overhead sounds bad and does not matter, because 10,000,000 x 200 B = 2.0 GB still fits in one machine.

The “dense venue” row is worth pausing on. A stadium with 50,000 publishers in one cell is the classic hot-spot question, and here the answer is that the question does not apply: nothing in this design is keyed by cell, so a stadium is 50,000 users hashed uniformly across every shard. That is the payoff for the shard-key decision in Data model, collected here.

The one genuinely large number in this system is 20 million sockets, and it is inherited rather than created. Everything this chapter adds — 2 GB of state, 187 Mbps in and 39 Mbps out at peak, about three cores of ingest — is small.

14. Failure modes

When the product’s output is a dot on a map, the failure class that matters is the system misleading a user while appearing perfectly healthy.

Note what is missing from the table: there is no data-loss row and no consistency row. Every failure here is about a user being shown something wrong, not about the system losing something. That is what a design with no durable hot path looks like.

Jittered backoff in the third row means each client waits a random amount before retrying, so a fleet that all disconnected together does not all reconnect together.

15. Alternatives rejected

When an interviewer proposes something more conventional, this is the table to have ready. Every row names the specific number that kills the alternative, so the answer is a figure rather than a preference.

16. Interviewer pushback

These six questions separate a candidate who pattern-matched “location, therefore geohash” from one who priced the problem. The answer after each is written the way you would say it out loud — every one of them leads with a number.

“Why not just store locations in Cassandra and read the latest?”

Because the value is dead in 30 seconds and the cost is permanent. The live working set is 280 MB; a year of history at the same update rate is 294 TB before replication, which is a million working sets a year.

I would deliberately not lead with a device count, and I would say why. The familiar version of that argument divides by 12,207 writes per device, which is a queue-depth-1 latency figure misread as a device ceiling, and then divides the average load by it at full utilization. Done properly — peak load, a real 500 k-IOPS device, 60% utilization — it is six devices, eighteen with replication, and under one if the store batches at all. The objection evaporates.

The two arguments that survive are the ratio and the coupling: a replicated commit on the critical path of a value that will be overwritten before anyone reads it.

If a product genuinely needs history — a timeline feature — I would sample it at one fix per five minutes into a separate cold store. That is 30 seconds over 300, so a tenth of the volume, and it is a decision made explicitly rather than by accident.

“News feed says fan out on write. Why not here?”

Same derivation, opposite inputs. The friend count cancels on both sides, so the comparison is producer rate against consumer rate. A feed author posts 0.2 times a day against 5 reads, so push wins by 25x. A location publisher emits 57.6 fixes a day against 3 map opens, so push loses by 19.2x. What I do instead is push only along edges someone is currently watching, and the saving is exactly 19.2x — the same ratio, because the reduction factor is publishers over concurrent viewers.

“Where is the geohash?”

Mostly absent, deliberately. The answer set is friends intersected with nearby, and the friend set is bounded at 200, so I scan 200 in-memory entries in 20 microseconds and distance-test them exactly. A cell index would fetch every publisher in a 5 km disc — thousands of strangers — to reach the same answer, and it would put stranger positions in a service that should never hold them. The index comes back for “people near you”, which is an unbounded candidate set and a different product.

“How long until a user is truly invisible?”

Sixty seconds worst case, and it is not a separate mechanism. Invisible mode stops publishing, drops the subscription entries and deletes the live cell; even if that last step fails, the TTL expires the dot on every viewer’s map because the client renders age_s and refuses anything past the TTL. That is the strongest reason to have a TTL at all — a design without one has no bounded time-to-invisible, only a promise.

“Your product is empty 93% of the time. Is the design wrong?”

The design is right and the framing was wrong. Expected live friends within 5 km is 0.075, so “nearby” cannot be the default view. The default is all sharing friends sorted by distance — four dots — and “nearby” is a highlight. The feature that makes it non-empty is “recently nearby” over an hour, which is 14.4 GB of per-user cell ring buffers. That estimate is exactly the kind that should change the product, and I would rather surface it in the interview than after launch.

“What if a user has 5,000 friends?”

The friend-first scan is 5,000 lookups at 100 ns, so 500 microseconds — still fine. It breaks around 100,000, at 10 ms, and that is where I would flip to a cell index and intersect the other way. The crossover is set by the friend count, not by anything geographic, which is a nice check that the rule in Deep dive 3 the geospatial index you probably do not need is really about bounded versus unbounded sets.

The assumption ledger

Every design is a set of assumptions with a diagram attached, and the diagram is only correct relative to them. The ledger below collects everything this chapter has leaned on, so you can state the design’s foundations in twenty seconds and say what replaces the design when each one fails.

Each assumption goes in one of three bins:

• State it — you are free to pick a value. Being wrong costs a re-derivation and nothing more.
• Ask it — the answer moves a policy or a threshold, so it is worth an interviewer’s time.
• Load-bearing — if this is wrong the design is not suboptimal, it is invalid. A box appears or disappears, rather than the machine count inside a box changing.

The test for which bin something belongs in, from ch 03: move the assumption an order of magnitude in each direction and ask whether the set of boxes changes, or only the number of machines inside them.

The last column is the one to read carefully. It is not “this gets worse” — it is the different system you would be designing instead.

The number that cannot sink the design

The one to mark explicitly as not load-bearing is the mean friend count of 200.

It is the number that looks most like it should matter. It appears in the fanout arithmetic, the nearby estimate and the index decision. And Deep dive 3 the geospatial index you probably do not need already proves it does not matter.

Two reasons.

First, headroom. At F = 5,000 the friend-first scan is 500 microseconds and every conclusion in the chapter is unchanged. The design only flips at F = 100,000, where the scan reaches 10 ms and you index space instead. From 200 to 100,000 is a factor of 500, so no plausible error in the estimate moves a box.

Second, and more fundamentally, F cancels on both sides of the push-versus-pull comparison in Deep dive 1 push pull and the ratio that inverts. It cannot affect the single largest decision in the chapter even in principle.

Knowing which number cannot sink the design is worth as much as knowing which can: it tells you not to spend the interview defending 200.

The sentence that makes this visible to an interviewer: “This design rests on four things. One, a location is worthless in a minute, which is what deletes the entire storage layer. Two, the recipients are a bounded friend list rather than a radius, which is what deletes the spatial index. Three, publishers vastly outnumber concurrent viewers, which is what makes subscribe-on-view a 19x saving rather than a complication. Four, privacy has to survive a modified client, which is what puts the authorization check at fanout. The friend count everything is counted with is not one of them — it has five hundred times the headroom it needs.”

Cheat sheet

Every line below is derived somewhere above. This table is the recall test, not the explanation — if a row does not make sense on its own, go back to the section that built it.

Related: 17 — Proximity Service derives every cell scheme cited here; 12 — Chat System owns the 20 M-socket tier and the heartbeat interval; 11 — News Feed owns the push/pull cost model this chapter inverts; 19 — Google Maps takes the same firehose and turns it into traffic.