One Timestamp Turned a $422 Bot Bill Into $1,863

Take a hypothetical small online store whose support bot answers 1,200 questions a day. One line in its prompt decides whether its bill shrinks or grows. At illustrative API prices, its model bill without prompt caching comes to about $1,539 a month. With prompt caching switched on correctly, the same traffic costs about $422. With caching switched on and one innocent-looking line added to the system prompt, it costs $1,863, more than having no cache at all.

Same bot, same questions, same model. The only thing that changes is how the prompt is built. This article follows that one scenario end to end, with every number shown, so you can see exactly where the savings come from and exactly how they disappear.

The setup: one support bot, 1,200 questions a day

Meet Harbor & Pine, a hypothetical online furniture store. Its support bot sends the model the same block of context on every request, plus whatever the customer just typed.

Here is what goes into each request:

• System instructions: about 1,500 tokens. Tone, escalation rules, what the bot must never promise.
• Catalogue and policies: about 10,500 tokens. Product dimensions, delivery zones, the returns policy, warranty terms.
• The variable part: the customer’s question plus a short conversation history, averaging 500 tokens.
• The answer: about 350 output tokens.

So every request carries a 12,000-token static block that never changes, and a 500-token tail that always does. That ratio, 24 parts identical to 1 part new, is the whole reason this example works. Hold onto it.

For pricing, we’ll use a hypothetical mid-tier model at round numbers: $3 per million input tokens and $15 per million output tokens. These are illustrative, not any specific model’s price. For the caching multipliers, we’ll use the structure Anthropic publishes: a cache write costs 1.25x the base input price for a five-minute cache, and a cache read costs 0.1x. Anthropic’s newest flagship models bill cache reads even lower, at 0.05x or less, which only makes the savings below bigger.

What every request costs before caching

Without caching, the model reprocesses all 12,500 input tokens from scratch every time.

• Input: 12,500 tokens × $3 per million = $0.0375
• Output: 350 tokens × $15 per million = $0.00525
• Total: $0.04275 per request

Four cents sounds like nothing. Multiply by 1,200 requests a day and you get $51.30 a day, or $1,539 over a 30-day month.

Notice where the money goes. Input is $0.0375 of the $0.04275, about 88% of every request. And almost all of that input is the static block the model has already read a thousand times today. That’s the waste prompt caching targets.

How prompt caching actually changes the bill

When a model reads your prompt, it computes internal attention states for every token before it writes a single word of the answer. Prompt caching tells the provider to keep those computed states for the beginning of your prompt, so the next request that starts with the exact same tokens can skip that work. If you want the deeper picture of why that stored state is so valuable to serving costs, the coverage of how DeepSeek compressed its KV cache down to 890 bytes shows how seriously labs take this memory.

Three rules govern everything that follows.

It’s a prefix match, not a similarity match. The cache hits only if your new prompt begins with exactly the same tokens as the cached one, in the same order. One changed character near the top breaks the match for everything after it.

The first request pays to write. On providers with explicit caching, storing the prefix costs a premium over normal input. That’s the 1.25x in our example.

The cache expires. On Anthropic’s default cache, the stored prefix lives for five minutes, and each hit resets that clock at no extra cost. Go quiet for longer than that and the next request writes the cache again from scratch.

Not every provider works the same way, and the details keep moving. OpenAI caches long prompts automatically on supported models. Its older models charge nothing extra to write the cache, while its newest models (GPT-5.6 and later) bill writes at 1.25x and keep a prefix for 30 minutes after its last use. Google applies implicit caching by default on Gemini 2.5 and newer models, and offers explicit context caching, billed partly as an hourly storage fee, on some of them. The arithmetic below uses the write-premium model because it’s the one where mistakes actually cost money, but the prefix logic applies everywhere.

Request two pays for request one

Run the Harbor & Pine request through all three states: no cache, a cache write, and a cache read.

Part of the request	No cache	Cache write (5 min)	Cache read
Static prefix, 12,000 tokens	$0.0360	$0.0450	$0.0036
Variable input, 500 tokens	$0.0015	$0.0015	$0.0015
Output, 350 tokens	$0.00525	$0.00525	$0.00525
Total per request	$0.04275	$0.05175	$0.01035

A cache read costs less than a quarter of an uncached request. The input portion alone falls from $0.0375 to $0.0051, an 86% cut.

The write costs $0.009 more than an uncached request. Each read saves $0.0324. So the break-even is brutally simple: if the same prefix gets used even twice inside the cache window, caching wins. One read repays the write premium more than three times over.

That ratio holds for any prefix size under these multipliers. You pay an extra 0.25x once and save 0.9x on every hit. Everything that follows in this article comes down to one question: does a second request arrive before the cache expires?

A full month with the cache warm

Harbor & Pine’s traffic arrives mostly between 9am and 9pm. That’s 1,200 requests over 12 hours, an average of one every 36 seconds. With a five-minute window that resets on every hit, the cache stays warm most of the day.

It doesn’t stay warm all day. Traffic is bursty. The first question each morning finds a cold cache. There are lulls mid-afternoon and late evening where five minutes pass without a customer. Assume 40 cold starts a day, each paying the write price, and 1,160 requests that hit the cache.

• Writes: 40 × $0.05175 = $2.07
• Reads: 1,160 × $0.01035 = $12.01
• Daily total: $14.08, against $51.30 uncached

Over 30 days, that’s about $422 against $1,539. A 73% cut on the whole bill, output included, from a change that touches no business logic.

Is the 40-cold-starts assumption doing too much work? Stress-test it. Say traffic is much lumpier and 200 of the 1,200 daily requests land on a cold cache. That’s 200 × $0.05175 = $10.35 in writes plus 1,000 × $0.01035 = $10.35 in reads, so $20.70 a day, or $621 a month. Still a 60% saving. When the prefix is large and traffic is steady, the result barely cares how pessimistic you get.

Caching also changes speed. Because the cached portion doesn’t need reprocessing, time to the first token of the answer drops, and the effect grows with prefix length. For a chat interface where a customer is watching a typing indicator, that matters as much as the invoice.

The timestamp that turned caching into a surcharge

Now the expensive version. A developer notices the bot sometimes gets confused about delivery cut-offs, so they add one helpful line at the very top of the system prompt:

Current date and time: 2026-10-15 14:03:22

Perfectly reasonable. It also changes the first tokens of the prompt on every single request. The prefix never matches. The cache never hits.

Worse, on a provider with explicit cache writes, the code still marks the prefix for caching, so every request pays the write premium and gets nothing back:

• 1,200 writes × $0.05175 = $62.10 a day
• Monthly: about $1,863

That’s $324 a month more than not caching at all, and $1,441 a month more than the working version. And nothing visibly breaks. The bot answers correctly, quickly enough, and the only symptom is an invoice that drifted upward.

The same trap shows up in quieter forms:

• A customer’s name or account ID inserted into the system prompt.
• Retrieved documents placed before the static instructions instead of after them.
• A JSON object whose keys get serialized in a different order each time.
• Tool definitions that are regenerated or reordered between requests. On Anthropic, tools sit ahead of the system prompt in the cache order, so changing them invalidates everything downstream.

The fix is always the same: static content first, variable content last. Move the timestamp down into the user turn, right next to the customer’s question, and the 12,000-token prefix becomes identical again. The bot still knows the time. The cache still works.

On caching with no write premium, such as Google’s implicit caching or OpenAI’s older models, the timestamp bug doesn’t add a surcharge. It simply cancels the discount. That’s the “does nothing” case, and it’s arguably harder to spot, because the bill looks exactly like it did before you thought you’d optimized it.

The quiet internal tool where the cache keeps going cold

Harbor & Pine also has a second, internal tool: a policy assistant the warehouse team uses to check return rules. Same 12,000-token prefix, same request shape. But it gets 20 questions a day spread across an eight-hour shift, roughly one every 24 minutes.

With a five-minute cache, every request arrives after the previous cache has expired. Every request is a write.

• Uncached: 20 × $0.04275 = $0.855 a day
• Five-minute cache: 20 × $0.05175 = $1.035 a day

Caching makes this tool about 21% more expensive. Low, sparse traffic is the second classic way the feature backfires.

Anthropic offers a longer cache for exactly this pattern: a one-hour lifetime with writes at 2x the base input price. Rerun the numbers:

• One write: 12,000 × $6 per million = $0.072, plus $0.0015 variable input and $0.00525 output = $0.07875
• Nineteen reads, each arriving within the hour and refreshing the timer: 19 × $0.01035 = $0.197
• Daily total: about $0.275, against $0.855 uncached

A 68% saving, from the same tool that lost money on the short cache. The dollar amounts are small here, but the pattern scales: any workload where requests arrive minutes apart rather than seconds apart should be priced against the longer cache before you decide caching isn’t worth it.

The rule of thumb: match the cache lifetime to the typical gap between requests. If the gap is under five minutes, use the short cache. If it’s under an hour, the longer cache’s higher write price pays off as soon as at least two more requests follow the write inside that hour. If requests arrive hours apart, caching can’t help you and you should leave it off.

When the output is the bill, caching barely moves it

Harbor & Pine’s marketing team runs a third workload: a product-description writer. It sends a 2,000-token brand guide and asks for a 4,000-token description.

• Input: 2,000 × $3 per million = $0.006
• Output: 4,000 × $15 per million = $0.060

Even a perfect cache, hitting every time with a 90% discount on the prefix, saves about $0.0054 per request. That’s 8% of a $0.066 request. Caching only ever discounts input. When output tokens are the bill, it’s a rounding error.

For output-heavy jobs, the levers that actually matter are elsewhere: a cheaper model for the drafting pass, tighter length limits, or splitting the job so the expensive model only does the hard part. The framework for matching frontier models to specific jobs is a better use of an afternoon than tuning a cache on this workload. And if a smaller model can handle the task, a cheaper small model such as Claude Haiku 5.5, which Anthropic says costs around 75% less to run than Haiku 4.5 on average, will do far more for output-heavy work than any cache.

There’s also a floor. Prompts below a minimum length can’t be cached at all, and the minimum varies by model: on Anthropic’s current lineup it ranges from 512 to 4,096 tokens, and OpenAI’s newest models need 1,024. Below the floor, the request simply runs uncached, with no error. A bot with a 400-token system prompt gets nothing from caching no matter how busy it is.

Where the cache breakpoint goes, in code

On Anthropic’s API, you mark the end of the cacheable prefix with a cache_control field. Everything up to and including that block gets cached. For Harbor & Pine, the structure looks like this:

{
  "model": "your-model-id",
  "system": [
    {
      "type": "text",
      "text": "You are the support assistant for Harbor & Pine..."
    },
    {
      "type": "text",
      "text": "<catalogue and policies, ~10,500 tokens>",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [
    {
      "role": "user",
      "content": "Current date and time: 2026-10-15 14:03\n\nCan I return a sofa after 40 days?"
    }
  ]
}

For the one-hour cache, the same field becomes { "type": "ephemeral", "ttl": "1h" }.

Notice where the timestamp went: into the user message, after the breakpoint. The full syntax and current model support are in Anthropic’s prompt caching documentation and, for OpenAI’s version, OpenAI’s prompt caching guide.

Anthropic allows up to four breakpoints per request, which matters for long conversations. Harbor & Pine’s three-turn chats don’t need it, but an agent that runs for an hour and accumulates 80,000 tokens of history does. Moving a breakpoint forward to the end of the latest turn means each new step only pays full price for what was just added. That’s a large part of why long agentic coding sessions, like the ones in this realistic SaaS MVP build with Claude Code, are affordable at all.

How to prove the cache is actually working

Never assume. The API response tells you what happened, and it’s the only place the timestamp bug shows up before the invoice does.

On Anthropic, the usage block reports cache_creation_input_tokens and cache_read_input_tokens separately from regular input. On OpenAI, look for cached_tokens inside the input token details of the usage block (newer models also report cache_write_tokens).

Run this test after any change to the prompt:

1. Send the same request twice, a few seconds apart.
2. On the first, expect cache-creation tokens close to your prefix size.
3. On the second, expect cache-read tokens close to the same number.
4. If the second shows zero reads, something above your breakpoint is changing. Diff the two raw prompts token by token until you find it.

Then log the ratio of read tokens to total input tokens in production. For Harbor & Pine’s support bot, a healthy ratio sits near 90%. If it slides toward zero after a deploy, someone has put something dynamic at the top of the prompt again.

The test to run before you switch it on

Every scenario in this article reduces to four checks, in order:

1. Is the static prefix long? Above the provider minimum, and ideally thousands of tokens. A short system prompt can’t be cached meaningfully.
2. Is input most of the bill? If output tokens dominate, caching saves single-digit percentages at best.
3. Will a second request arrive before the cache expires? Seconds apart: short cache. Minutes apart: long cache. Hours apart: no cache.
4. Is the prefix truly identical every time? No timestamps, names, IDs, retrieved snippets or reshuffled tools above the breakpoint.

Four yeses, and you’re looking at Harbor & Pine’s 73%. One no, and you’re looking at either nothing or a surcharge. Run the double-request check from the previous section today, because the most common prompt caching setup in production is one that someone switched on, never verified, and is quietly paying the write premium for on every call.