Explainers · measured on 4.47M files
What is prompt caching? The 90%-off switch for what your AI re-reads.
Prompt caching lets a model like Claude re-read a stable chunk of your prompt (the standing instructions, the tool list, a long document) for roughly a tenth of the price, instead of paying to process it from scratch on every call. It routinely takes about 90% off repeated context. It is also widely misread: it speeds up re-reading text the model has already seen. It cannot speed up the slowest thing an AI agent does: finding the file in the first place. That work happens before a single token is ever cached.
In short: the takeaways
Prompt caching stores the processed form of a repeated prompt prefix. So the next request that starts with the same text skips re-processing it and pays a fraction of the price.
Cached reads bill at about 0.1×, roughly 90% off. But the first write costs about 1.25×, so you only win when the same prefix is reused (two reads break even on the 5-minute cache).
It is a prefix match: one changed byte voids the cache after it. A timestamp or a reordered tool list silently turns caching off.
It cannot speed up retrieval. Finding a file (the ripgrep grind, the directory walks) runs outside the model, before any token exists to cache.
The upstream fix is an index, not a bigger cache. Hand the agent one lookup instead: 85 μs to a filename, 16 ms to search every file's contents (a blink, on a 4.47-million-file machine).
The concept, plainly
Pay to read it once. Then pay a tenth to remember it.
Prompt caching is a way to reuse the work a model already did on a repeated piece of your prompt, so the next request that begins with the same text skips re-processing it and bills those tokens at a fraction of the price. Text is fed to a model as tokens, and before it can answer it has to "read" all of them: a real, paid computation. Caching saves the result of that read for a stable prefix and replays it on the next call.
First call: a cache write. The model processes your prefix and stores it. Those tokens cost about 1.25× the normal rate this once (2× if you ask for the hour-long cache).
Next call: a cache read. If the prefix is byte-for-byte identical, the model skips the work and reads it back at about 0.1×, roughly 90% off, and faster to first token.
Until it changes. The cache lives five minutes by default (or an hour), refreshed on every hit. Change one byte of the prefix and the next call pays full price and writes a fresh entry.
| The mechanic | The number |
|---|---|
| What it caches | the processed prefix: tool list, system prompt, stable context |
| Cache read price | ~0.1× base input, about 90% off |
| Cache write price | 1.25× (5-min) · 2× (1-hour), a one-time surcharge |
| Default lifetime | 5 minutes, refreshed on each hit; 1-hour option |
| Break-even | 2 reads (5-min) · 3 reads (1-hour) |
| Match type | exact prefix: one changed byte voids everything after it |
| Minimum cacheable prefix | ~1,024 to 4,096 tokens (model-dependent) |
| Breakpoints per request | up to 4 |
Numbers are the published mechanics of Anthropic's prompt caching (the same shape applies across major providers). The order the model reads your request is fixed (tools, then system prompt, then the conversation), so the stable material has to sit first for any of it to cache. For where those tokens end up, see what is the context window.
The centerpiece
The cached part is cheap. The fresh part is where the money goes.
Every call to the model is a stack: a reused cached prefix at the base, billed at about a tenth, and fresh tokens on top, billed at full price. An agent that shells out to grep piles a growing file-hunt dump into that fresh part every call. Give it an index and the fresh part collapses to one line. Caching discounts the blue base equally in both cases, and never touches the fresh tokens, which is exactly where retrieval lives.
Segment sizes are illustrative; the split is the point. The file-hunt share is measured: on a real machine about 58% of a Claude Code session's tokens went to finding files, not building. Ryzen 9 9950X3D · 64 GB DDR5 · NVMe · Windows 11 · 4.47M files.
Why it genuinely helps
Agents re-send everything, every turn.
The API is stateless: to keep a conversation going, an agent resends the entire prompt on every turn (the whole system prompt, the full tool list, any documents pinned into context). Claude Code works this way, and so do Cursor, Codex and the rest. Without caching, that fixed preamble is re-read at full price on turn 2, turn 3, turn 30. Cache it once and the whole stable base drops to about a tenth for the rest of the session. This is real, and worth doing.
The part nobody tells you
Three things prompt caching can't do.
Caching optimizes the one thing it touches (re-reading a stable prefix), and people quietly assume it fixes the rest of the bill. It doesn't. Here is the honest boundary.
It can't speed up retrieval. Finding the file (the directory walks, the content grind) happens before the model runs. No token exists yet to cache. Caching a fast lookup and caching a slow one save exactly the same nothing on the search itself. On a 4.47M-file drive we watched one content query take 93.8 seconds with ripgrep (a minute and a half of the agent grinding) against 16 milliseconds with an index, quicker than a blink; no cache setting narrows that gap, because it is upstream of the model entirely.
It can't discount the first look. A cache write costs about 1.25×, slightly more than not caching. You only win on the re-read. Content the agent sees once and forgets (which is most file-hunt output) never earns that surcharge back.
It can't cache what keeps changing. Directory listings and grep results differ every run. They are fresh, full-price tokens each call. And if they land before your stable context, they become the classic silent cache-buster that voids everything after them.
So the more your agent searches, the more fresh, uncacheable, cache-threatening tokens it makes: the exact opposite of what caching rewards. The fix isn't a better cache. It's not needing the search. That is a different problem. See where the tokens actually go in the token cost of file hunting and why one slow step dominates a run in the backbone of the LLM economy.
The upstream fix
Cache the prefix. Index the machine.
Interlinked keeps an always-warm index of every file on the machine (names and contents, across every repo and the roughly 90% of the disk that was never in git) and hands it to any agent over MCP. The slow retrieval step that caching can't touch collapses: one lookup returns the exact path or line, and the agent gets that one line back instead of a directory dump. The wall-clock time disappears, and the fresh, uncacheable junk that was crowding your window and threatening your cache mostly disappears with it. Caching and this are complementary: different axes of the same bill.
It stays current as you save (about a millisecond, under 30 ms worst case), sits at roughly 44 MB at rest, and keeps its index under 1% of the drive. One signed install wires it into 19 AI clients. How it plugs in: what is an MCP server.
CACHING MAKES RE-READING CHEAP. AN INDEX MAKES THE SEARCH DISAPPEAR. YOU WANT BOTH.
Questions people ask
Prompt caching, quick answers.
Do this today
Fix both halves of the bill.
Cache the stable prefix. Put your tool list and system prompt first, keep them byte-for-byte identical, and mark them for caching. That fixed base drops to about a tenth for the rest of the session.
Keep the volatile stuff last. Timestamps, per-request IDs and search results go after the cached prefix, never before it, or they void the cache for everything downstream.
Remove the search entirely. Give the agent an index instead of grep. One signed installer wires Interlinked into 19 AI clients: Claude Code, Cursor, Codex, Windsurf, Zed, Cline and the rest. No JSON editing.
Watch the fresh part shrink. The directory dumps that used to pile into every call (full-price, uncacheable, cache-threatening) become one line back. Same model, a much smaller bill.
Everything local is free forever, on all your devices, no card. Hosting starts at $5.99/mo. Download for Windows.