Explainers · latency · measured July 2026

Six seconds is nothing. Your agent does it 71 times to find one file.

Latency is the pause between a question and its answer. On a single tool call it's usually too small to feel. But an agent never makes one call. It makes a chain of them, each waiting on the last, and the waits don't average out. They stack. We watched a real one on a 4.47-million-file machine: Claude Code, 71 tool calls, 6 minutes and 57 seconds. To find one file.

Start here

Latency is the time between asking and getting an answer back.

For a human clicking around an editor, latency is a single wait you can shrug off. For an AI agent it's different in one decisive way: the agent pays it over and over, on every step of a task. Compounding latency is what happens when those waits chain: the agent's tool calls run one after another, each waiting on the last, so the delays add end to end instead of overlapping. Seconds per hop quietly become minutes per run.

TL;DR

01Latency is the wait between an agent asking for something (a file, a function, a fact) and getting it back.
02One tool call's latency is invisible. A task is dozens of dependent calls: the waits stack, they don't cancel.
03The slowest hop is almost always search: your agent shells out to ripgrep, which re-scans the disk every time.
04Measured: Claude Code spent 6m57s across 71 tool calls to find one file, and ~58% of its tokens on the hunt.
05One always-warm, whole-machine index answers each hop in microseconds (85 µs a file), so the chain never compounds.

The picture

One task. A chain of waits.

Here is the same task on two floors. Same chain of tool calls, same order: only the search layer under them changes. On the grinding floor each hop adds seconds; seventy-one of them total a coffee break. On the warm-index floor each hop answers at memory speed, and the total never leaves a blink.

This is the least glamorous layer in the stack and the one everything else stands on, the backbone of the LLM economy. To see where the wait hides inside a single one of those boxes, we break one call into its stages in the anatomy of a tool call.

Same per-hop wait · longer and longer chains

Multiplication, not addition.

The wait per hop barely changes. What changes is how many hops a task needs, and that number climbs fast once an agent is exploring an unfamiliar machine. A wait you'd never notice once becomes the whole run.

The chain	Wait per hop	Total: grinding floor	Total: warm index	What the wait feels like
1 hop	≈ 5.9 s	5.9 s	139 µs	a slow breath (vs a blink)
10 hops	≈ 5.9 s	~59 s	~1.4 ms	a lost train of thought (vs a blink)
40 hops	≈ 5.9 s	~3 m 55 s	~5.6 ms	check your phone, forget why (vs a blink)
71 hops · measured	≈ 5.9 s	6 m 57 s	~9.9 ms	a coffee run, to find one file (vs a blink)

Methodology: Ryzen 9 9950X3D · 64 GB DDR5 · NVMe · Windows 11 · 4,470,000 files. Per-hop grinding time is the measured run's own average (6 m 57 s ÷ 71 tool calls ≈ 5.9 s). The warm-index column is the same number of hops at the measured 139 µs median lookup across all 4.47M files; a single file lookup is 85 µs (best 2 µs). In the real run the agent didn't even need the hops: the chain collapsed to a single 16 ms call.

One real task, on the clock

What the compounding cost, in full.

6m 57s → 16ms

time to the file: grinding floor, then warm index

71 → 1

tool calls spent in the chain

~58% → ~0%

of session tokens burned on the hunt

That's roughly 7,200,000×, six minutes of an agent grinding versus a single blink. And latency isn't the only bill: every one of those 71 round trips also spends tokens the model has to read and then forget. The wait is felt by the human; the tokens are felt by the wallet. Both come from the same place: the search floor.

Why the waits don't cancel out

An agent can't fire its calls all at once. It waits, reads, decides, waits again.

01
Serial by nature. Each call's answer decides the next one. The agent can't parallelize a search it hasn't run yet, so the waits line up nose to tail instead of overlapping.
02
The slow hop repeats. Search isn't a one-time cost. Every time the agent needs a fact that isn't already in its context, it goes back to the disk, and pays the wait again.
03
Nothing stays warm. Without a persistent index, each query re-walks the filesystem from scratch. The 90% of your drive that was never in git gets re-scanned on every single hop.

Your agent shells out to ripgrep (a genuinely fast tool) on nearly every hop. But even fast, re-reading a 4.47-million-file machine per query is the spike in the chain: 93.8 seconds for a single content scan on this box, a query we answer in 16 ms. That's why one slow hop is so costly: see why one slow tool call ruins an agent run and the full ripgrep comparison.

Collapse the chain

Fix the floor, and every hop above it gets fast.

You can't make an agent's calls run in parallel, but you can make each one land instantly. One always-warm index of every file on the machine, names and contents, answers the hops that used to cost seconds in microseconds. Save a file and it's searchable about a millisecond later, so the index is never stale. The whole engine sits around 44 MB at rest (less than a browser tab), and one signed installer wires it into 19 AI clients at once.

85 µs

to find one file, 412,000× faster than the 35 seconds VS Code takes on the same drive

139 µs

median across all 4.47M files. Windows Search took 67 seconds for the same set

16 ms

for the content query that grinds ripgrep for 93.8 s: the spike, flattened

Everything local is free forever, on all your devices, no card. Hosting starts at $5.99/mo. Download for Windows →

Questions people actually ask

Latency, in plain terms.

What is latency in an AI agent?

Latency is the delay between the agent issuing a tool call (search a file, read code, run a command) and getting the result back. On one call it is usually too small to notice; the problem is that an agent makes a chain of them.

Why does latency compound instead of just adding up once?

Because an agent's tool calls are serial and dependent: each result decides the next call, so they can't run in parallel. The waits line up end to end across a whole run, and the slowest hop (search) repeats every time the agent needs a fact it doesn't already hold.

What is a good latency for an agent's file search?

Fast enough to disappear inside a chain: microseconds to low milliseconds. On a 4.47M-file machine, one file lookup returns in 85 µs and a content query that took ripgrep 93.8 seconds returns in 16 ms.

Does a faster model fix agent latency?

Not the part that compounds. A smarter model reasons better but still waits on the same tool calls; the floor under the model sets the ceiling on the run. Cutting the wait means fixing the tools, not the brain.

How do I reduce my AI agent's latency?

Attack the slowest, most-repeated hop: file and content search. Give every agent one always-fresh, whole-machine index instead of re-walking the disk on every query, and the chain stops compounding.

Stop paying the wait on every hop.

Give every agent on your machine one always-fresh index and the chain stops compounding: microseconds a hop, whether it's hop one or hop seventy-one.

DOWNLOAD FOR WINDOWS →

Free forever · No card · Hosting from $5.99/mo

Measured July 2026 · Ryzen 9 9950X3D · 4.47M files · Windows 11← All research