How a neural network figures out who to blame

A neural network starts out useless — its millions of internal knobs (called weights) are basically random, so its first guesses are garbage. The magic isn't in the guessing; it's in how it improves. The core trick is an algorithm called backpropagation, and it's really just an elegant way of assigning blame.

Here's the loop. In the forward pass, data flows through the layers and the network produces an output. A loss function then measures how wrong that output was — say, it predicted "cat" with 20% confidence when the answer was cat. Now the network needs to know: which weights caused this mistake, and in which direction should each one move? That's where the backward pass comes in.

Tap to zoom

Backpropagation uses the chain rule from calculus to send the error backwards through the network, layer by layer, computing how much each individual weight contributed to the mistake — its gradient. Weights that pushed hard toward the wrong answer get flagged for a bigger correction. Gradient descent then nudges every weight a tiny step in the blame-reducing direction, and the whole cycle repeats millions of times. Backprop finds the direction; gradient descent takes the step.