Prompt Caching: When It Saves 90%
AI pricing guide · updated 2026-08-07
Prompt caching is the largest single discount available on most AI APIs, cached input generally costs about a tenth of the standard rate. It is also the one discount that can increase your bill, because writing to the cache costs more than not using it at all.
What caching actually does
When consecutive requests share an identical prefix (a long system prompt, a fixed set of instructions, a document you are asking many questions about) the provider can keep that prefix's computed state and reuse it instead of reprocessing it.
You are billed for the reuse at a heavily reduced rate. Every cached-input rate in our catalogue is exactly 10% of that model's standard input rate: a 90% discount on the cached portion.
Critically, caching applies to a *prefix*. The shared content must be identical and at the start of the request, which is why prompt structure matters: put the stable material first and the variable material last.
The part people miss: cache writes cost extra
Creating the cache entry is billed at a premium, commonly 1.25× the standard input rate. So the first request that populates the cache costs 25% more than it would have without caching.
That means caching is only a saving if the entry gets reused enough times to earn back the write premium. With a 1.25× write and a 0.1× read, one write plus one read costs 1.35× the input rate against 2× for two uncached requests, so it pays from the second hit onward, and pays enormously by the tenth.
The failure mode is caching something that is rarely reused: a prefix that changes per user, or traffic so sparse that entries expire before a second request arrives. Then you are simply paying 25% more.
Cache lifetime decides whether sparse traffic benefits
Cache entries expire. Minimum lifetimes are typically measured in minutes (some providers guarantee around thirty) which is ample for a busy application and useless for one receiving a handful of requests an hour.
So the question is not just "do my requests share a prefix" but "do they share it within the cache window". A support tool handling steady traffic benefits enormously; a nightly batch job that touches the same prompt once per run does not.
Where caching pays most
Long stable system prompts. If you are sending several thousand tokens of instructions on every request, caching them is close to free money.
Document question-answering. Cache the document once, then ask many questions against it; this is the canonical case, and the saving scales with document length.
Multi-turn conversations, where the transcript prefix grows but its earlier portion stays identical. Each turn can reuse everything before the newest exchange.
It pays least on short prompts, per-user personalised prefixes, and anything where the variable content sits near the start.
Frequently asked questions
How much does prompt caching save?
Cached input typically costs 10% of the standard input rate, a 90% discount on the cached portion of a request. Every cached-input rate in our catalogue follows that pattern. The saving applies only to the shared prefix, so the benefit scales with how much of your request is stable and reused.
Can prompt caching make my bill higher?
Yes. Writing a cache entry is usually billed at about 1.25× the standard input rate, so a prefix that is written but rarely reused costs 25% more than not caching at all. Caching pays from roughly the second hit onward, the risk is per-user prefixes or traffic too sparse for entries to be reused before they expire.
What can be cached?
An identical prefix at the start of the request. That means putting stable content (system instructions, reference documents, few-shot examples) first, and variable content last. Changing anything in the cached region invalidates it, so ordering your prompt deliberately is what makes caching work.
How long does a prompt cache last?
Minutes, typically, with some providers guaranteeing a minimum of around thirty. That is plenty for steady traffic and not enough for sparse workloads, so cache lifetime effectively decides whether low-volume applications can benefit at all.
See it in the data
Related guides
Terms used in this guide
Published by Tokenando. Last updated 2026-08-07. Figures in this guide are computed from our own pricing index and dated where they can move; see the methodology and corrections policy.