DeepSeek-V4.1-Flash cuts per-token global KV cache to 890 bytes, about a quarter of its predecessor
Synopsis
DeepSeek-AI released DeepSeek-V4.1-Flash, a multimodal mixture-of-experts model aimed at the KV cache cost of long-context agent workloads. The backbone has 552B parameters and supports contexts up to one million tokens, activating 16B parameters per token during decode and only 8B during prefill. Through cross-layer KV reuse in CSA2 and FP4 caching, its global KV cache held in HBM falls to 890 bytes per token, roughly a quarter of DeepSeek-V4-Flash; with SWA Bounded Replay, the persistent KV cache is about one eighth of that baseline, while overall performance is better.
Interpretation
For long-context agents the cost bottleneck shifts from computation to storing and moving the KV cache, and DeepSeek-V4.1-Flash cuts the global KV cache held in HBM to 890 bytes per token. DeepSeek-V4 combined a global attention branch with sliding-window attention; for sufficiently long sequences the global KV dominated the runtime cache footprint and was constrained by HBM capacity, while persistent KV caches were constrained by SSD and host memory capacity and by I/O bandwidth. Measured as global KV cache bytes per token, DeepSeek-V4.1-Flash is about a quarter of DeepSeek-V4-Flash and roughly 437 times smaller than DeepSeek-V1.
The compression stacks two paths, architecture and precision: cross-layer reuse shares global KV and indexer K, and the main KV cache is stored in FP4. DeepSeek-V4 used a hybrid of CSA and HCA, with each layer maintaining its own global KV and indexer K and an FP8 main KV cache. CSA2 statically assigns each layer one of three modes, Full, Reindex or Reuse; Reuse mode directly takes the preceding layer's global KV and Top-K indices, and the FP4 main KV cache, introduced through quantization-aware training, nearly halves storage versus FP8.
On the deployment side, sliding-window attention KV is no longer written to the persistent cache; instead it is approximately reconstructed by replaying only the most recent nwin tokens. DeepSeek-V4 required replaying L×nwin tokens for exact reconstruction, a cost the report calls prohibitive in production deployments. SWA Bounded Replay replays only the most recent nwin tokens, bringing the persistent KV cache to about one eighth of DeepSeek-V4-Flash; the report says this approximation causes only negligible performance degradation and simulates the same replay during post-training for train-aware adaptation.
Perspective
The demonstrated scope is DeepSeek-AI's comparison of DeepSeek-V4.1-Flash with its predecessor base models under the company's internal evaluation framework, with cache figures given as bytes per token and performance conclusions covering text and multimodal agent benchmarks. The intended readers are systems and model engineers concerned with long-context serving cost and KV cache management.
The report acknowledges a remaining gap to giant models on science-oriented agentic tasks that require expert-level domain knowledge, and a distinct overall multimodal gap versus giant closed-source systems. The KV states reconstructed by SWA Bounded Replay are by design not mathematically equivalent, though the report says the impact on response quality is negligible.
