BeCuriouzLearn something new every day
Why AI runs on graphics cards, not regular processors
💡 Neural networks are mostly giant matrix multiplications, and GPUs win because their thousands of simple cores do that math in parallel instead of one step at a time like a CPU.
It sounds strange that the chips built to render video-game explosions are the same ones training ChatGPT. But under the hood, both jobs are the same math: multiplying enormous grids of numbers, called matrices, over and over. A neural network is essentially a stack of matrix multiplications, and so is figuring out the color of millions of pixels on screen.
The key difference is how processors are built. A CPU has a handful of very fast, very clever cores — great for running one complicated task at a time, like your operating system. A GPU has thousands of simpler cores that all do arithmetic simultaneously. That trait is called parallelism, and matrix math is embarrassingly parallel: each of the thousands of multiply-add operations is independent, so they can all happen at once instead of one after another.
This is why a single high-end AI GPU can be dramatically faster than a CPU for training, and why the companies building large models spend billions on them. A CPU racing through math one clever step at a time simply can't keep up with thousands of dumb cores grinding through the same additions all at once — for this particular kind of work, brute-force width beats cleverness.