The API trick that makes repeat calls cheaper — and it's mostly free

Every time you call an LLM, the model re-reads your entire prompt from scratch — the system instructions, the long document you pasted, the tool definitions, all of it. If you're building an app that sends the same big chunk of context on call after call, you're paying full price to process identical text over and over. Prompt caching is the fix: you mark the stable part of your prompt, and the API stores its processed form so later requests can reuse it instead of recomputing.

The economics are the surprising part. Writing to the cache costs a bit more than a normal token (you pay a one-time premium to store it), but reading from the cache costs a fraction of the normal price — often around a tenth. So caching pays off whenever the same context is reused within the cache's lifetime, which is why it shines for chatbots with long system prompts, document Q&A, and agents that carry big tool definitions across many turns.

The key rule is order: caching works on a prefix. The cached block and everything before it must stay byte-for-byte identical, while the part that changes — the user's new question — goes at the end. Put your stable, reusable content first and your variable content last, and the cache stays valid across calls.