One developer reconciled 119,866 of their own Claude API calls this spring and found $2,530 in pure overpayment, about 17% of the bill, with the waste concentrated in a single month. Nothing had changed in their code. What changed was a default they were never told about: the prompt cache’s lifetime quietly dropped from one hour to five minutes around March 6, 2026, and every session pause longer than five minutes started re-uploading context at the expensive write rate instead of reading it back cheap. The finding sits in GitHub issue #46829, which was closed “not planned.”
That episode is the whole argument for treating prompt caching as an operations discipline rather than a checkbox. Caching is the single largest lever on an AI bill. It is also the one that fails silently, and the three major providers implement it three incompatible ways, so “we turned on caching” tells you almost nothing about what you are actually paying.
The cache read is the cheapest token you will ever buy
Every frontier model charges you for the input tokens it has to process. Caching lets it skip the processing on a prefix it has already seen and bill you a fraction of the rate for the part it recognizes.
The discounts are steep enough to reorder your entire cost model. On Anthropic’s platform, cache reads are billed at 0.1x the base input price, a 90% discount, and on Claude Fable 5.1 and Mythos 5.1 the cache read drops to 0.025x, which is 97.5% off. Google passes on roughly a 90% discount on implicit cache hits for Gemini 2.5 and newer, and its explicit cache reads on the Gemini 3 Pro tier run 75% off. OpenAI’s automatic prompt caching takes 50% off cached input, and on the current flagship the effect is larger: GPT-6 Astra bills cached input at $1 per million against a $10 standard rate, the same 90% break, a number I unpacked in the Astra cost-per-task breakdown.
For any workload with a large stable prefix, a long system prompt, a fixed tool schema, a retrieved document set reused across turns, the cache read is where most of your money either stays or leaves. An agent loop that reprocesses a 40,000 token context on every step pays full freight forty times. The same loop with a warm cache pays the write once and reads for a tenth or less after that. This is why cache hit rate, not token count and not model choice, is usually the biggest line you can move.
Three vendors, three caches that behave nothing alike
The trap is assuming caching works the same everywhere. It does not, and the differences decide what you have to build.
| Anthropic (Claude) | OpenAI | Google (Gemini) | |
|---|---|---|---|
| How it turns on | Explicit: you place cache_control breakpoints |
Automatic on prompts over 1,024 tokens | Implicit automatic, plus optional explicit caches |
| Cache read price | 0.1x base (0.025x on Fable/Mythos 5.1) | 50% off (about 0.1x on Astra) | ~0.1x implicit; 75% off explicit on 3 Pro |
| You control lifetime? | Yes: 5-minute default, 1-hour option | No: a short rolling window, roughly 5 to 10 minutes | Implicit no; explicit yes, with a storage meter |
| Minimum to cache | 512 to 4,096 tokens by model | 1,024 tokens | 32,768 tokens for explicit caches |
| Extra cost to hold it | Write premium: 1.25x (5m) or 2x (1h) | None | Explicit storage: about $1/M per hour (Flash), $4.50/M (Pro) |
Anthropic hands you the controls and the responsibility. You mark up to four breakpoints per request, the system does a backward lookback across at most 20 positions to find a matching prefix, and you choose the time-to-live. A cached block is refreshed for free every time it is used, so a busy prefix rarely pays the write premium twice. But the lifetime is measured from the start of the request, not the end, so if a response streams for four minutes on a five-minute cache, your next call has about one minute to land before the entry is gone.
OpenAI removes the controls entirely. Caching is on by default, it needs an exact prefix match, it activates at 1,024 tokens and grows in 128-token increments, and you cannot pin the lifetime. You get a prompt_cache_key to help route repeat traffic to the same cache and a Prompt Caching Dashboard plus Cache Diagnostics to see hit rates. Simplicity in exchange for less say over when the discount fires.
Gemini splits the difference. Implicit caching is automatic and free to manage. Explicit caching, per Google’s context caching docs, guarantees the discount but adds a storage meter you pay by the hour and a 32,768-token floor, so it only pays off when you query the same large context more than three or four times inside the window.
If you route across models, and the case for keeping a second model qualified is strong, none of your caching assumptions port. A prompt architected to hit Anthropic’s explicit breakpoints is not the same prompt that keeps OpenAI’s automatic prefix stable.
The default that moved without a memo
Back to March. For the 33 days from February 1 to March 5, the developer in issue #46829 shows a clean one-hour TTL as the default across two machines and two accounts. Around March 6 and 7 the five-minute token type reappeared, and by March 8 it was 83% of cache-creation tokens. Their February waste ran 1.1%. Their March waste ran 25.9%.
The mechanism is the write-to-read ratio. A cache write costs 1.25x the base rate; a cache read costs 0.1x. That makes the write 12.5 times more expensive than the read on the older Claude models in the analysis. Every time a five-minute cache expired during a normal pause, the next turn billed a fresh write rather than a cheap read. On a model where the read is 0.025x, that penalty is 50 times, not 12.5, so the newer and cheaper the cache read, the more a lifetime miss actually costs you in proportion. Subscription users in the same thread report hitting their five-hour quota for the first time that month, because cache creation counts against quota at full rate while reads barely register.
I have watched the enterprise version of this movie. Running IT operations at a large telecom, I once spent a week reconciling a cloud bill that jumped with no change in our deployment. The cause was a provider quietly altering how a metered service rounded usage. Nobody sent a notice; the number in the invoice was the only evidence. The lesson transfers directly. A billing default you did not set and cannot see is still your bill, and the only defense is reading your own telemetry against what you expected to pay.
Four ways a cache that is “on” quietly stops firing
Caching is usually enabled once and never watched again, so it degrades as the code around it changes. The failures are boring and they are everywhere.
A moving value at the front of the prompt kills everything behind it. A “Current time: 14:32:07” line at the head of a system prompt, or a session ID, or a per-user token in the cacheable region means the prefix hash differs on every call, so the cache writes fresh and never reads. Practitioners debugging zero hit rates trace most cases to exactly this: a dynamic element buried in what should be a stable prefix.
The prefix falls under the minimum. Shared instructions and tool definitions that add up to 800 tokens will not cache on a model with a 1,024-token floor. The counterintuitive fix is sometimes to make the prompt longer, padding the stable region with genuinely useful reference material until it clears the threshold.
The lifetime is shorter than your think time. This is the March story generalized. If your users pause to read, or a tool call runs long, or a human sits in the loop, a five-minute window expires between turns and you pay writes forever. The reuse pattern, not the price sheet, tells you which TTL to buy.
Reordering breaks the match. Move the tool schema below the retrieved documents, inject a field the model returns into the next turn’s prefix, and the hash changes. On append-only formats, and Fable 5.1 now enforces append-only history where editing an earlier turn before a thinking block returns an error, this is easy to trip without noticing.
What actually controls the bill
Turning caching on is step zero. The control comes from four habits, and they are ordinary operations work, not prompt magic.
Instrument the hit rate as a first-class metric. Every provider returns the cached token count in the response, cache_read_input_tokens on Anthropic, cached_tokens on OpenAI. Wire that into your observability and alert on a drop, so a regression pages someone instead of surfacing in next month’s invoice. A hit rate under 30% on a workload with a fixed system prompt is not a tuning problem, it is a structural one, and it means a dynamic value is sitting in your cacheable region.
Architect the prompt prefix-first. Put everything stable at the front in a fixed order: system instructions, tool schema, long reference material, retrieved context. Push everything variable to the back: the user’s turn, the timestamp, the per-request identifiers. The cache can only help with what stays identical, so the design goal is to maximize the identical prefix.
Match the lifetime to the reuse density, deliberately. If a prefix is reused several times a minute, the short default is right and the free refresh-on-use keeps it warm. If reuse is spaced minutes apart, as it is in most human-in-the-loop and long agent runs, pay for the longer window or the explicit cache and treat the write premium as the cost of not re-uploading. Multi-agent systems feel this hardest, because the failures in agent workflows live in the handoffs where context gets rebuilt.
Reconcile the invoice to an expected cache-read ratio. This is the FinOps move and the one the March episode vindicates. Decide what fraction of your input tokens should be reads, compare it to what the bill shows, and investigate the gap. That reconciliation is how a default change gets caught in a day instead of a quarter, and it is the same discipline that separates a controlled spend from the end of tokenmaxxing many teams are now living through.
Caching sits alongside routing and effort tuning as the levers that actually decide where the money in the model stack goes. It is the strongest of the three and the least watched, because the discount is invisible when it works and invisible when it stops. The teams keeping their bills flat this year are not the ones who found a cheaper model. They are the ones reading their own cache-read numbers every week and noticing, fast, when a number they did not touch decides to move.