Prompt caching is a prefix problem

$37,901.73. That is what one developer reported paying for a month of a coding agent on Claude Opus 4.6 through Amazon Bedrock. The answers cost $698. The repeated, uncached prompt in front of them cost $35,600. Nothing was looping, nothing was compromised, and nobody had misread a price list. One prompt caching detail that no layer verified turned 79% of the input tokens into 96% of the input bill.

One account holder's reported month. Output, the number everyone watches, was $698 of $37,901.73. Figures as reported on Hacker News, 2026-04-28.

Three things to keep, every price verified against provider pages on 2026-09-07.

• On current Claude models and the GPT-5.6 line, cached input costs about a tenth of uncached input. The discount is not where providers differ.
• The minimum prompt length that qualifies does differ, and sharply: 512 tokens on Claude Opus 5, 1,024 on Sonnet 5, and 4,096 on Haiku 4.5. Those floors are not ordered by price.
• So the cheapest model is not the lowest rate per token. It is the one whose floor your prompt clears. On identical traffic the cheap model can cost 4.7 times the mid-tier one, and the flagship can beat both.

How prompt caching actually bills

You are buying a byte-identical prefix that gets reused inside a short window. That is the mechanism, and the pricing follows it.

Claude's rates are the clearest version. A cache read costs 0.1 times base input; the write that populates the cache costs 1.25 times base for a five-minute entry or 2 times for an hour. Claude Fable 5.1 and Mythos 5.1 read at 0.025 times instead. Anthropic states the break-even plainly on its pricing page: because a hit is a tenth of the price, one read pays for a five-minute write, and two reads pay for an hour write.

OpenAI splits the same way now: on GPT-5.6 and later writes cost 1.25 times uncached input and reads 0.1 times. Older OpenAI models charge nothing to write and discount less on read, tracking generation rather than tier. gpt-4o cached input is 50% off, gpt-4.1 and o3 are 75% off, and gpt-5-pro, o1-pro and o3-pro list no cached rate at all.

Two details then decide more of your bill than model choice does. Lifetime is measured from the start of the request that writes or reads the entry, not from when it finishes, so a response that streams for four minutes eats four minutes of a five-minute window. And refresh is free on use, so a busy prefix stays warm while one used every hour dies between calls. The same reuse cuts time to first token, and neither vendor publishes a number worth quoting, so neither will I.

Two more cache economies, briefly

Gemini meters caching like storage. The pricing page charges $0.50 to $4.50 per million tokens per hour depending on model, so an idle cache keeps billing whether or not anything reads it. Its 3.1 Pro also keeps a context-size price cliff that Claude 4.6+ removed: $2.00 per million input tokens at or below 200k, $4.00 above.

DeepSeek runs the widest spread I found. On deepseek-v4-flash a cache hit is $0.007 per million against $0.22 on a miss, off-peak, and peak hours double both since 2026-08-16. It is also the strictest about matching: its caching guide matches whole prefix units rather than the longest shared prefix, so its own worked example misses twice on the same document before the third hits.

The minimum is a floor, not a formality

Before any number: this compares money, not models. Paying for Claude Haiku 4.5 with no cache against paying for Claude Sonnet 5 with one, on the same prompt. In that comparison the cheaper line is also the more capable one, which is the point. It is not an argument that small models are expensive, and not a recommendation to buy up a tier for quality.

Every provider refuses to cache a prompt below a minimum length, and refuses silently. From the caching docs: "Any requests to cache fewer than this number of tokens will be processed without caching, and no error is returned."

Take a support agent: a system prompt, five tool definitions, three few-shot examples, one customer message per request. The reusable part measures 2,048 tokens, and you serve 200 requests an hour.

Claude Haiku 4.5 at $1 per million input tokens, uncached, because 2,048 is under its 4,096 floor:

200 x 2,048 tokens x $1 / 1,000,000 = $0.4096

Claude Sonnet 5 at $2 per million, with one breakpoint after the examples, so the prefix is written once and read 199 times:

write: 2,048 x $2.50 / 1,000,000 = $0.00512
reads: 199 x 2,048 x $0.20 / 1,000,000 = $0.08151
total = $0.0866

The mid-tier model with a cache costs one fifth of the budget model without one, a 4.7x gap. Same prompt, same 200 requests, and the only difference is a prefix sitting between two floors.

Same prompt, same 200 requests, three models. Haiku cannot cache below 4,096 tokens, Sonnet not until 1,024. Derived from Claude caching minimums and per-model prices, verified 2026-09-07.

Three boundaries keep this honest. The 4.7x is measured at 200 reuses and converges toward 5x; at 50 it is 4.1x. It flips at three reuses, since Sonnet pays the write premium while Haiku pays only rate. And the band is real: at 4,096 tokens Haiku starts caching, its reads at $0.10 per million undercut Sonnet's $0.20, and the small model is small again.

Below 1,024 tokens the ordering inverts once more. At a 512-token prefix Opus 5 clears its floor and Sonnet does not clear its 1,024, so 200 requests cost $0.0541 on Opus against $0.1024 on Haiku: the flagship at roughly half the budget tier's price, on a threshold rather than a capability.

This is a Claude-shaped inversion. OpenAI holds one minimum across the GPT-5.6 line, so floors do not differ by tier there. Its version of the trap is the write charge: gpt-5.4 paid nothing to populate a cache and gpt-5.6-sol pays 1.25x, so a breakpoint left on volatile content goes from free to expensive on migration day.

Measure the prefix you actually send

The character count of your prompt file is not the number that matters. A 2,048-token prefix clears Sonnet's floor and sits at half of Haiku's, so measure against the model you actually deploy:

curl https://api.anthropic.com/v1/messages/count_tokens \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-sonnet-5",
    "tools": [ ... your tool definitions, in your real order ... ],
    "system": [ ... your system blocks ... ],
    "messages": [{ "role": "user", "content": "hi" }]
  }'

The token counting endpoint returns {"input_tokens": N}. It is free to call, rate-limited by tier rather than tokens, and it counts under the tokenizer of the model you name, which matters because Claude 4.7 and later produce roughly 30% more tokens for the same text. It does not touch the cache, so measure the way you intend to send: same tools, same order.

Then read the response fields, because the obvious one is a trap. On Claude, input_tokens counts only what sits after your last cache breakpoint.

total input = cache_read_input_tokens + cache_creation_input_tokens + input_tokens

A request with 100,000 tokens of cached prefix and a 50-token user message reports input_tokens: 50. Anyone computing spend from that field alone is off by three orders of magnitude. OpenAI's equivalents are usage.input_tokens_details.cached_tokens and cache_write_tokens, and on models before GPT-5.6 the cached count is rounded down to a multiple of 128.

Then price a request:

def input_cost(usage, price_per_million, read_mult=0.1, write_mult=1.25):
    reads = usage["cache_read_input_tokens"]
    writes = usage["cache_creation_input_tokens"]
    ordinary = usage["input_tokens"]          # tokens after the last breakpoint
    weighted = ordinary + reads * read_mult + writes * write_mult
    return weighted * price_per_million / 1_000_000

Check it against Anthropic's own example: 10,000 uncached tokens and 40,000 reads on Opus 5 gives $0.05 plus $0.02, which is what this returns.

Over a JSONL log of usage objects, the realized hit rate is two commands:

jq -s 'map(.usage.cache_read_input_tokens) | add' usage.jsonl
jq -s 'map(.usage.cache_read_input_tokens
          + .usage.cache_creation_input_tokens
          + .usage.input_tokens) | add' usage.jsonl

Divide the first by the second. Manage cost per successful request, and treat hit rate as the explanation when it moves.

The discount is not a coupon. It is a function of how much of your prompt you keep identical. Derived from the 0.1x cache read rate on a $2 base input, verified 2026-09-07.

Why prompt cache misses happen

A miss has a cause, and there are only a handful. Each is a property of your code, not the provider's luck, and the stack failure behind that bill has the same shape as AI code review is not independent: four layers in a row, each assuming the one below it handled caching.

The breakpoint is on content that changes. Anthropic's own example: stable blocks, then a block carrying a timestamp, breakpoint on that last block. Every request writes a new entry, the lookback finds nothing, and you pay a write forever and a read never. Automatic caching lands in the same hole, because it picks the last cacheable block.

Tool definitions moved. The prefix order is tools, system, messages, and changing tools invalidates all three levels. Hold order and serialization fixed; a gateway that re-serializes tool JSON yields a fresh prefix hash on every call. Tool overhead is billable anyway, since the tool-use system prompt adds 286 tokens on Opus 5, 354 on Sonnet 5, and 496 on Haiku 4.5.

The lookback ran out. The system walks back at most 20 positions from your breakpoint, and writes happen only at breakpoints, so a turn that appends more than 20 blocks misses. Place a second breakpoint before you need it.

The window closed. A long streamed response eats the five minutes, so take the hour tier for bursty traffic you can amortize at 2x the write.

The model changed under the conversation. Caches are per model, so a router failing over mid-thread or an A/B test pinning variants resets the prefix for everything after it. That is "route to the cheaper model" fighting "cache aggressively", and it is the same mechanism as the price inversion above.

Everything was cold at once. An entry is readable only after the first response begins, so twenty parallel identical requests pay twenty writes. Above roughly 15 requests per minute OpenAI can also route you to a machine holding nothing, and prompt_cache_key biases placement without guaranteeing a hit.

The history was edited. Compaction, summarization and truncation change the prefix, so fewer tokens with a worse hit rate can cost more. Compare totals, not token counts.

On the Claude API you can stop guessing. Cache diagnostics, beta header cache-diagnosis-2026-04-07, takes the previous response's id and returns cache_miss_reason as model_changed, system_changed, tools_changed or messages_changed, sized by cache_missed_input_tokens. It reports only the earliest divergence, and its fingerprint is hashes, never your prompt text. It is missing on Bedrock, Google Cloud, Microsoft Foundry, and Claude Platform on AWS, which is the stack that bill came from.

A migration day is a cache-cold day

Moving to a new model moves you out of a warm cache. Caches are keyed per model, so the window after a cutover re-pays writes, and the floor may not be the floor you measured against.

On Sonnet 5 with a 4,096-token prefix, a write costs 12.5 times what the same request pays as a hit: $0.01024 against $0.00082. Each cold request adds $0.00942, so the first 200 after a cutover cost about $1.88 extra. That is the whole transient, and it is small.

The part that matters is the re-pricing. gpt-5.4 is $2.50 per million input tokens and gpt-5.6-sol is $4.00, so identical traffic moves 60% up before you touch a prompt. Sonnet 4.6 at $3 to Sonnet 5 at $2 moves the other way. And the new model's minimum is an untested variable: a prefix that cleared 1,024 can fail to clear 2,048.

So the migration checklist gains a line: after any model change, re-measure the prefix against the new floor and compare realized cost per request against your baseline. AI model fatigue is a business rhythm argued for gating every switch on your own evals. This arithmetic belongs in that gate, since a switch that wins on quality and loses 60% on input price is still a decision, and a worse one.

The five-minute audit

1. Measure your reusable prefix with count_tokens against the model you deploy, with your real tools in your real order.
2. Compare it to the minimum for that model and platform. Below the floor, extend the prefix with material that helps, or stop paying for a breakpoint that cannot fire.
3. Put the breakpoint on the last block identical across the requests you want to share a cache. Not the last block.
4. Log cache_read_input_tokens, cache_creation_input_tokens, input_tokens and the model name per request. You cannot reconstruct them later.
5. Compute cost per successful request weekly, and treat hit rate as the explanation when it moves.
6. Put a hard spend cap on the account and an alert above it. An alert is not a kill switch, and that distinction is most of the $37,901.73 story.
7. Re-run step 1 after every migration, and after any change to how you serialize tools.
8. Check your retention posture. Zero Data Retention organizations on OpenAI default to in_memory cache lifetime instead of 24h, and Claude splits caches by workspace.

Where these prices stop being true

Every figure above was verified against provider documentation on 2026-09-07. Treat that date as the expiration, not the footnote. Rates move underneath this arithmetic without warning: DeepSeek re-priced on 2026-08-16, Sonnet 5's scheduled increase to $3 per million on 2026-09-01 was cancelled so the introductory $2 became standard, and Gemini's 3.x Flash prices are already published to double on 2027-01-01. Re-read the Claude pricing page and the OpenAI pricing page before you act on a number here, and re-check the minimum table too, since that is the row people forget.

Two limits. Nothing here is a measured hit rate from live traffic; every figure is a published price multiplied by a token count, and the formulas are printed so you can check them. The $37,901.73 breakdown is one account holder's console report, not an audit, and their implied rates run about 10% above first-party list pricing, consistent with Bedrock's regional endpoint premium. Quality, latency, and rate-limit headroom are out of scope, as is the flat-plan versus API-metering argument.

The plumbing this arithmetic plugs into, per-request cost tracking, structured output, retries and backoff, is chapter 4 of AI Engineering. If the numbers say your path is hardware you already own, Run AI Locally covers it, and /tools/llm-fit will tell you whether a given model fits the machine in front of you.