The Rust middlemen that let Discord serve a viral message just once

Picture a Discord channel that suddenly goes viral: within a single second, tens of thousands of clients scroll to the same spot and ask for the same handful of messages. Because a distributed store keeps each range of data on one partition, every one of those reads stampedes toward the same node. That lopsided pile-up is the classic hot partition — one shard melting while the rest of the cluster yawns. Discord's write-up How Discord Stores Trillions of Messages frames this as the failure mode that kept biting them as they grew.

The interesting part of their answer is where they solved it. Rather than throwing a larger cluster at the problem, they inserted a fleet of small data services — implemented in Rust — directly in front of the datastore. These services track which queries are currently outstanding. When a burst of identical reads shows up, only the first one is forwarded; the rest are parked and attached to that same in-flight request. When the answer comes back, it fans out to every parked caller at once.

Seen from the database, a viral moment stops looking like a spike at all — it flattens into roughly one query regardless of how many fans piled on. The lesson generalizes far beyond chat apps: the cheapest request is the one you recognize as a duplicate and never send twice.