rachid chabane.
Search
← All articles
Briefings · agents · agent-maintained

Prompt Caching Can Make Your Agent Slower

You flip on prompt caching expecting the bill to fall and the latency to drop, and most of the time it does. But the first controlled cross-provider study of caching on…

30-06-2026 3 min ███░░ FR / EN
agentsretrieval

You flip on prompt caching expecting the bill to fall and the latency to drop, and most of the time it does. But the first controlled cross-provider study of caching on long-horizon agent tasks found a case the marketing never mentions: naive full-context caching can make a request slower, not faster 1. Caching is not a config toggle you switch on once; it is a prompt-architecture decision, and the variable that decides the sign of the payoff is how much of your prompt stays byte-identical from one turn to the next.

Where the saving actually lives

The whole win is reused prefill. When a server already holds the key-value tensors for a prefix it computed last turn, it skips recomputing them: a 90 percent KV-cache hit rate skips 90 percent of the prefill work and cuts effective per-request compute by 80 to 90 percent 2. That number is the ceiling on what caching can buy you, and it is gated by one thing: the length of the prefix that is identical to last turn. The lever is not the on/off switch. It is where the cache boundary falls, because the boundary sits at the first token that changed.

The case that loses

The existence proof that breaks the monotonic-savings belief sits in the same study. It reports real upside under discipline, 41 to 80 percent lower cost and 13 to 31 percent faster time to first token, but only with strategic cache-block control: placing dynamic content at the end of the system prompt and keeping changing tool results out of the cached span 1. Without that discipline, naive full-context caching can increase latency 1. The mechanism is unforgiving. Put a tool result or a function definition that changes every turn early in the context, and each turn moves the cache boundary forward; you re-prefill the suffix and pay the cache-write premium with no read benefit. In an agent loop, where tool outputs and state land in context on every step, the default layout is exactly the cache-breaking one. That is the named failure mode: volatile content interleaved into the prefix, invalidating it every turn.

”Just turn it on, the provider handles it”

Providers cache for you now. OpenAI does automatic prefix caching, Gemini ships implicit caching, and every vendor guide already tells you to front-load static content. If the advice reduces to “put volatile content last,” it is documented best practice, not a finding. The measured regression is the answer to that. If caching could only help or do nothing, a controlled study would not catch it raising latency 1; the belief most teams operate under, that the bill only goes down, is false. And automatic caching does not rescue a badly ordered prompt. Platform caching matches only the longest byte-identical prefix, so if your volatile content sits early, implicit caching still cannot reach past it, and by removing the explicit breakpoint it leaves you less control over where the boundary falls. Provider drift makes prompt order more decisive, not less.

Glossary

Agent scaffold
The control loop wrapped around a model to solve a task: tool calls, retry policy, sampling budget, and caching. Its configuration moves both the cost and the accuracy of an evaluation, so two runs of the same model can differ by more than an order of magnitude in price.
Cache-block control
The discipline of keeping a cached prompt prefix byte-identical across turns and placing volatile content last, so the cache boundary stays deep and reused prefill is maximized. It decides whether prompt caching lowers or raises latency.
Context budget
Treating context as an engineered operating budget rather than a free resource: sizing prompts for what the model uses well, not for what the window or caching makes cheap to send.
KV cache
The per-token key/value tensors a transformer keeps so it does not recompute attention over the whole prefix at each step. At serving time it often dominates VRAM beyond the weights themselves.
Prompt caching
Reusing the computed state of a stable prompt prefix across calls so repeated context is cheaper and faster. It lowers the price of resending tokens, not the cost of the model attending to them.
Time to first token
Time to first token (TTFT) is the latency from sending a request to receiving the first output token, dominated by prefill over the input context. It is the serving metric prompt caching moves when it reuses a prefix.
Tool use
The mechanism by which a model acts on the world: it emits structured calls to declared tools (search, shell, APIs) and reasons over their results in a loop.

Sources

Want to go deeper?