NVFP4-MLA pushes both MLA KV-cache storage and attention math to FP4, matching BF16 accuracy while shrinking the cache 3.20-fold
Under long-context and agentic workloads, MLA's KV-cache capacity and attention computation dominate serving cost. NVFP4-MLA pushes both MLA KV-cache storage and both attention matrix multiplications to FP4, using selective residual compensation that adds an NVFP4 residual only to quantization-sensitive channels and two-dimensional quantization blocks that let the shared latent cache serve both reduction directions. On nine benchmarks for DeepSeek-R1 and Kimi-K3, average accuracy matches BF16 MLA while the cache shrinks 3.20 and 3.05 times relative to BF16.