Tag
#kv-cache
Reports tagged kv-cache.
Reports
2 reportsAug 2026
08-04 Deep Dive The Cache Miss After the Gap Coding agents serve 95.7% of their input tokens from cache. The steps that miss cluster in one place — the first call after an idle gap — and that is where the serving work went. Jul 2026
07-29 Deep Dive The KV Cache Became Inventory — and Almost Nobody Charges Rent Yet Serving stacks now give the KV cache its own flash tier, its own router and its own rack. The bill a customer sees still discounts recomputation avoided, not memory held.