Jonathon's AI Wiki

Prompt Caching for Agencies — When It Saves You Money

9 min read

Source: ai-research/prompt-caching-anthropic-docs-2026-04-27.md, ai-research/prompt-caching-anthropic-blog-announcement-2026-04-27.md, ai-research/prompt-caching-anthropic-pricing-2026-04-27.md, ai-research/prompt-caching-anthropic-cookbook-notebook-2026-04-27.md, raw/reddit-1uf8gd5.md, raw/reddit-1v434n3.md

Agencies pay the cached-prefix tax over and over: the same brand voice doc, the same skill bundle, the same CLAUDE.md, the same do-not-say list, fed to Claude on every prompt. Prompt caching lets you pay full price once, then read the same prefix back for ten cents on the dollar. If you are running real client work — Smile Springs Family Dental’s blog calendar, fifty FLUQs scoring runs, a multi-agent pipeline — caching is not an optimization. It is the difference between an API bill that scales linearly with output and one that scales with cleverness.

Key Takeaways

  • Cache writes cost 1.25x base input (5-minute TTL) or 2x (1-hour). Cache reads cost 0.1x — a 90% discount on every repeated token.
  • A 5-min cache pays for itself after one hit. A 1-hour cache pays for itself after two hits.
  • Sonnet 4.6 cache reads are 3.00 base. Opus 4.7 cache reads are 5.00 base.
  • Minimum cacheable prefix is 2,048 tokens for Sonnet 4.6 and 4,096 for Opus 4.7. Below that, caching is silently skipped.
  • Anthropic’s cookbook shows a 187K-token prefix going from 4.89s baseline to 1.48s on a hit — 3.3x faster.
  • Caching is destroyed by anything that mutates the cached prefix: timestamps, dynamic ordering, mid-prefix file inserts.

How Prompt Caching Works

You mark part of a prompt with cache_control: {"type": "ephemeral"}. The first request processes the prefix, charges you 1.25x the base input rate to write it to cache, and serves the response. The second identical request — within five minutes — reads the prefix from cache at 0.1x base input. The output is identical to a non-cached call. The savings are entirely on the input side.

Two TTLs: 5 minutes (default, 1.25x write) and 1 hour (beta, 2x write). Every cache hit refreshes the TTL, so a busy session keeps the cache warm essentially for free. Up to four explicit breakpoints per request, or use automatic caching — one cache_control field at the top level — and Claude moves the breakpoint forward as conversations grow.

A cache miss is the expensive default. Hits require the prefix to be byte-identical to a previously cached entry. Change one token, even an invisible one, and you eat a fresh cache write plus the full input rate for everything after. The system looks back up to 20 blocks from your breakpoint to find earlier hits, but past 20 blocks the entry is gone unless you add another breakpoint.

The Math for Agencies

Smile Springs Family Dental scenario: WEO Marketly runs a content calendar workflow on Sonnet 4.6. The system prompt — brand voice doc, do-not-say list, dental SEO skill bundle, persona profile — is 10,000 tokens. The team runs 50 prompts in a one-hour planning session.

Without caching, prefix cost only: 50 × 10,000 × 1.50**

With 5-min caching (refreshed by hits):

  • 1 cache write: 10,000 × 0.0375
  • 49 cache reads: 49 × 10,000 × 0.147
  • Total: $0.1845

That is 87.7% off the prefix cost — 264 / week on prefix alone, before output and per-prompt input. On Opus 4.7 the same scenario saves $4.41 per session — caching matters more the more expensive the model.

In the Wild — Sonnet 4.6 on Every Instagram DM (97% measured cache hit)

[Reddit signal — r/ClaudeAI 2026-06-25] Source: raw/reddit-1uf8gd5.md (307 score / 77 comments, OP timhartmann7). A real production deployment that validates the math above. A 7-location sushi chain takes ~90% of its orders through Instagram DMs; the operator put a Claude Sonnet 4.6 agent on every DM via the Meta API. The full menu — ingredients, calories, allergens, delivery zones, hours, prep times, and current promos for all 7 locations — must reach the model on every single message, because each reply needs the whole menu in front of it. At full input price that re-processes the entire block every time someone types “hi” — a non-starter at real volume. On ~97% of messages the static block reads from cache at a tenth of input price (~90% off); the only full-price tokens left are the customer’s message and the reply, both tiny next to the menu dataset. The operator’s framing: that is the whole gap between “too expensive to run per message” and “the owner forgot there’s an LLM in the loop at all.” The technique is exactly the stable-prefix discipline this article prescribes — menu and rules in one stable prefix, only the live conversation changes — which is what holds the hit rate near 97%.

Measuring the Hit Rate Across a Fleet (community report, 2026-07-23)

[Reddit signal — r/hermesagent 2026-07-23] Source: raw/reddit-1v434n3.md — a single low-engagement community post (20 score, 4 comments, OP u/Countlesshrs, “MODELS” flair) that links out to a third-party blog (ashu.io/blog/cache-stack-time-slash-llm-api-bill/). None of it is independently verified, and the tooling claims are the poster’s own experience running five agents. Treat as a lead to test, not a finding.

The sections above optimize one prompt against one vendor. This report extends the same lever to a fleet across several providers, where the operator’s problem is not “is my prefix stable” but “I cannot see my hit rate at all.” Three claims worth testing:

  • Hit rate is the number to instrument, and small gains are non-linear. The poster’s framing — cached tokens billed roughly 10–20× cheaper depending on provider, so a modest hit-rate gain nearly halves fresh-token spend — is arithmetically consistent with the discount structure: moving from ~88% to ~93% hits cuts the uncached remainder from 12% to 7%, roughly 40% off the expensive slice, even though the hit rate only moved five points. ^[inferred — the arithmetic is this wiki’s; the poster states the outcome, not the derivation]
  • A gateway gives you the per-request numbers your subscriptions hide. He runs the Bifrost gateway (official Docker image) in front of the fleet to log cached_read_tokens, cost, provider, and an attempt trail per request, with a web UI. The specific motivation is that subscription products like the Z.ai Coding Plan and OpenCode Go return no usage logs at all — a gateway is the only place to recover them. This is the fleet-scale version of the cache_read_input_tokens check in Try It below.
  • Load balancing silently fragments your cache. On OpenRouter, requests spread across backends land on different cache entries; injecting a fixed session_id reportedly took his hit rate from ~88% to ~93%. He also pins the cheapest provider with a preset so OpenRouter cannot quietly route to a pricier backend. The general principle generalizes beyond OpenRouter: anything that moves your request to a different machine invalidates the prefix just as surely as editing it does.

Two further claims are outside this article’s Anthropic-centric scope but are recorded as they may matter to multi-provider agencies: he reports time-of-day pricing on Chinese providers (DeepSeek roughly doubling during Beijing peak, Z.ai burning 2–3× quota at peak, MiMo discounting ~20% off-peak — with their off-peak falling in US afternoon/evening), and describes stacking subscription accounts through Bifrost fallback chains with cron-driven peak/off-peak provider switching. No Anthropic equivalent to time-of-day pricing is documented anywhere in this wiki, and none of these figures are corroborated.

When Caching Backfires

  • Timestamps inside the cached block. “Today is 2026-04-27” injected into the system prompt makes the prefix unique every day. Caching never hits.
  • Dynamic file uploads before the cached content. Cache is positional. If you put a per-request PDF before the static skill bundle, the bundle never caches.
  • Reordering tools or system blocks. Tool definitions sit at the top of the cache hierarchy — touch them and everything after invalidates.
  • Micro-edits to the prefix. Fixing a typo in the system prompt costs you the entire warm cache across every session running that prompt.
  • Prefix below the minimum. A 1,500-token system prompt on Sonnet 4.6 is too short to cache (2,048-token floor). No error, no warning — just silent full-price billing.
  • Routing through a load balancer. Every failure above is a change to what you send; this one is a change to where it lands. If a router or aggregator spreads identical requests across backends, each backend holds its own cache and your byte-identical prefix still misses. Pin the session and the provider. ^[community-reported, raw/reddit-1v434n3.md — single low-engagement source]

Agency Patterns That Win

  1. Stack the static stuff at the front. Brand voice, skill bundles, persona, examples — all before any per-request content. Put your cache_control on the last unchanging block.
  2. Use 1-hour TTL for batch runs. A blog generation job hitting Claude 30+ times in 90 minutes pays the 2x write once, then 0.1x for the rest. Cheaper than 18 separate 5-min writes.
  3. Move dynamic data into the user message, not the system prompt. The client brief, today’s date, the specific URL — all of those go after the cached prefix.
  4. Watch cache_read_input_tokens in the response usage. If it’s zero on call two, your cache is missing — diagnose before you burn through a billing cycle paying full input.

Try It

  1. Audit one production prompt. Count tokens in the static prefix (skill bundle, system, examples). If it’s over 2,048 on Sonnet or 4,096 on Opus, you have a caching candidate.
  2. Add cache_control: {"type": "ephemeral"} to the last static block. Run the prompt twice within 5 minutes. Diff cache_read_input_tokens between call one and call two — call two should be near-equal to your prefix length.
  3. For any workflow that runs >5 prompts in an hour, switch to ttl: "1h". The 2x write pays back after the second hit and you stop re-warming caches between sessions.

Graph View

Backlinks