How long does an inference provider keep your prompt prefix in its cache? I sent the same ~4,000-token prefix to 35 OpenRouter backends, let it sit untouched, and then asked for it back. For the same open weights, the answer ranged from under 30 seconds to more than 24 hours — and the thing that decided it was not the model but the company serving it.

The surprising part is not that caches expire. It is that three providers serving byte-identical weights can disagree about your prefix's lifetime by four orders of magnitude, and that the model's own vendor is often not the one that holds it longest.

The probe is a single cycle I will call prime → verify → idle → probe: send a unique prefix, confirm ten seconds later that it cached, wait a fixed idle interval with no requests in between, then send it once more and read cached_tokens from the usage field. OpenRouter pins each cacheable prefix to one backend, so the second call lands on the same endpoint that primed it. What follows is 12,856 retention trials across three open-weight models — DeepSeek-V4-Flash, MiniMax-M2.5, and GLM-5.2 — and their 35 serving routes.1

On open weights you are not choosing a cache when you choose a model. You are choosing one when you choose a provider.

First, does the write even land?#

Before asking how long a cached prefix survives, I had to confirm it caches at all — and how quickly. A separate propagation sweep sends a fresh unique prefix and then re-probes it at fixed short waits up to 60 seconds after the prime. Each cell is the fraction of primes whose cached tokens were already visible at that wait, so a row that brightens left to right is an asynchronous write becoming readable, and a row that stays dark is a write that never surfaced.

Cache-write visibility heatmap for open-weight routes
Cache-write visibility: 34 of 35 open-weight routes surface the write within 2 seconds; deepseek-parasail never does inside 60 seconds.

34 of the 35 routes crossed 50% visibility within 2 seconds. Exactly one, deepseek-parasail, never surfaced the write inside the 60-second window. The practical reading is reassuring: back-to-back requests already land warm almost everywhere, so you do not need to insert an artificial pause before reusing a prefix. The exception matters more than the rule, though — a route that never shows the write gives you no cache benefit at all, whatever TTL it advertises.

Same weights, wildly different lifetimes#

This is the headline. I call the number effective TTL: the longest idle interval a route survives before verified retention first falls below half, scanned over a grid from 30 seconds to 24 hours. It is not an advertised or contractual figure — it is what the cache actually did under live load.

Effective prefix-cache TTL by serving provider, one panel per model
Effective TTL by provider; each panel is one model and green bars mark the vendor's own backend.

For the same open weights, serving infrastructure moved the effective TTL from essentially zero to the full 24-hour observation window. And the winner changes by model, so a single "best cache backend" is the wrong thing to look for.

Model Longest-retaining provider(s) Its TTL Vendor-native TTL
DeepSeek-V4-Flash deepseek 6h 6h
MiniMax-M2.5 minimax-deepinfra, minimax-friendli, minimax-siliconflow 1h 5m
GLM-5.2 glm-streamlake ≥24h 30m

The split is the whole story. For DeepSeek-V4-Flash the vendor's own deepseek backend leads at 6 hours. For MiniMax-M2.5 the vendor's minimax backend (5 minutes) is beaten twelvefold by minimax-siliconflow (1 hour). For GLM-5.2 the vendor's glm backend (30 minutes) is left far behind by glm-streamlake (more than 24 hours). The vendor logo wins once out of three.

The spread between the best and worst backend on identical weights is hours versus seconds.

So on open weights the serving provider, not the model, sets your cache TTL, and the vendor-native backend is not a safe default. Pick the route per model, and re-measure whenever you change routes.

How a cache dies: cliffs and ragged rows#

Each single TTL number collapses a decay curve. The full matrix is that curve, one column per idle interval — and it shows how a route loses its cache, which the one-number summary hides.

Retention probability by idle interval for every open-weight route
Verified retention probability per idle interval, rows sorted by effective TTL; gray cells had no verified sample.

Two row shapes are worth telling apart. A cliff — near-full retention up to some interval, then a collapse within a step or two — is the signature of timer-based eviction: the cache holds until a TTL expires. A ragged row — retention wobbling well below 100% with no clean boundary — points to load-driven capacity eviction, where your prefix competes with other tenants for cache space and can be dropped at any time. Compare your workload's idle-gap distribution against your route's row: if your typical gap sits left of the cliff, budget for warm hits; if it straddles the cliff or the row is ragged, a hit is a coin flip, and your cost and latency estimates should assume a cold full prefill.

The TTL you get depends on the hour#

Retention tracks backend load, so the same route can hold a prefix longer off-peak than at peak. Reading effective TTL one hour-of-day bucket at a time shows how much of the headline number is really an average over hours that behave differently.

Effective TTL by hour of day for open-weight routes
Effective TTL by prime-time hour (UTC), split by model; color encodes bucketed retention tiers so a tier change across the day stands out.

A route whose tier changes across the day has a load-bound cache. If your traffic is concentrated in particular hours, judge the route by its cells in those hours, not by the summary number.

How much of the prompt actually counts#

TTL is how long the cache survives. Depth is how much of the prompt it credits on a hit — a separate axis entirely. Depth here is the credited cached fraction, cached_tokens / prompt_tokens, measured only on retained hits. Fractions just below 1.0 usually reflect block-aligned caching, where only whole prefix blocks are credited; markedly lower fractions mean the route genuinely re-computes part of the prompt even when it reports a hit.

Credited cached fraction on retained hits, by model and route
Cache depth on retained hits: long-lived is not the same as deep — glm-streamlake retains longest yet credits a shallower fraction than many shorter-lived routes.

On a retained hit, most open-weight routes credit almost the whole prompt. But depth and TTL do not travel together: glm-streamlake retains the longest of any route here, and yet its credited fraction is shallower than routes that expire in minutes. When you compare routes on cost, multiply the advertised cache discount by the route's actual depth — a long-TTL route with shallow credit can save less per request than a shorter-lived one that credits the full prompt.

Does a warm hit actually feel faster?#

The payoff of a hit is skipping prefill, so a warm probe should return its first token faster than the cold prime. I measure time-to-first-token on the same streamed calls the retention probe already makes, which means it includes the OpenRouter gateway hop and any proxy buffering — so compare cold against warm within a route, not absolute milliseconds across routes.

First-token latency, cold prime versus warm probe, per route
Cold-prime versus warm-probe TTFT per route; the latency payoff exists only where the warm box sits clearly below the cold one.

The cached-token discount shows up on your bill either way. The latency payoff only shows up where the warm box sits clearly below the cold one. A route that credits tokens without dropping TTFT is saving you money, not time — and which of those you care about depends on whether your workload is cost-bound or interactive.

What this means if you're routing#

Turning the tiers into picks per model: the first column is the route I would choose on cache retention alone, the second lists others in the same long-retention tier (interchangeable on this axis), and the last names the routes that offer effectively no reuse window in this run.

Model First pick for cache TTL Other strong routes Weakest in this run
DeepSeek-V4-Flash deepseek (6h) none deepseek-parasail (<30s), deepseek-fireworks (30s), deepseek-gmicloud (30s)
MiniMax-M2.5 minimax-siliconflow (1h) minimax-deepinfra (1h), minimax-friendli (1h) none
GLM-5.2 glm-streamlake (≥24h) none glm-deepinfra (<30s), glm-friendli (<30s), glm-venice (<30s)

This is a cache-TTL read, not a universal routing recommendation — cost, quality, rate limits, and latency still matter and can outweigh it. But the retention spread on identical weights is large enough that cache-heavy workloads should treat provider choice as a first-order parameter, not a deployment afterthought.

Pick the provider like you're picking a cache#

On open weights the model is close to a commodity: the same weights are for sale from a dozen shops. The cache in front of them is not. Two routes to byte-identical weights differed here by four orders of magnitude in how long they held a prefix, and the vendor's own logo was no guarantee it would be the one holding it. If your workload leans on cached prefixes — an agent replaying a long system prompt, chat over a fixed context, RAG over a shared corpus — the provider is not a deployment detail. It is a cache decision, and it deserves a measurement rather than a guess.

A companion post runs the same probe against the big proprietary APIs — OpenAI, Gemini, Anthropic, and xAI — where the model and the serving stack change together and the windows are measured in minutes.

These numbers come from a probe I run through OpenRouter, snapshotted in early July 2026. Prefix caching is best-effort and load-dependent, so exact TTLs drift between runs and across times of day — read the tiers, not the decimals.

Footnotes

  1. The measured idle interval is verify → probe with no reads in between, so a miss means the prefix genuinely sat idle that long, not that a refresh confounded it. Retention is verified-only: it is computed only over trials whose prime provably cached by the +10s verify probe, so a route that never landed warm is excluded rather than scored as an eviction. Effective TTL is a first-crossing rule — the last idle interval before verified retention drops below 50%, scanning shortest-first, so a lone hit at a longer interval does not revive it. ≥ means window-limited: the cache survived the largest interval I observed for that route, and the true TTL may be longer. Retention tracks backend load, not just elapsed time, so exact numbers shift between runs; longer intervals are sometimes skipped after confirmed short-interval misses, which tilts long-interval estimates toward healthier cycles. And OpenRouter's prefix stickiness is not guaranteed across hour boundaries, one source of verify/probe noise. ↩