Why adding randomness to your retries makes systems more reliable

In the Amazon Builders' Library, engineer Marc Brooker dissects a failure pattern that quietly takes down healthy systems. When a service hiccups, every client that got an error retries. If they all use plain exponential backoff — wait 1s, then 2s, then 4s — they retry in sync, hammering the recovering server in coordinated waves. This is the thundering herd: the retries meant to help become a self-inflicted denial-of-service.

The fix is almost absurdly simple: add jitter, a bit of deliberate randomness to each client's wait time. Instead of everyone waiting exactly 4 seconds, each waits a random amount up to 4 seconds. The herd spreads out, peak load drops, and — counterintuitively — all the work finishes faster because the server is never slammed.

Brooker's broader lesson is that retries are load. A retry is a bet that the problem was transient, but during a real overload, retries pour fuel on the fire. The article argues for pairing backoff with jitter, capping total attempts, and treating retry storms as a first-class risk rather than a free safety net.