Skip to main content
Search

# NVFP4-MLA

1 related articles

Subscribe to updates
  1. arXiv

    NVFP4-MLA pushes both MLA KV-cache storage and attention math to FP4, matching BF16 accuracy while shrinking the cache 3.20-fold

    Under long-context and agentic workloads, MLA's KV-cache capacity and attention computation dominate serving cost. NVFP4-MLA pushes both MLA KV-cache storage and both attention matrix multiplications to FP4, using selective residual compensation that adds an NVFP4 residual only to quantization-sensitive channels and two-dimensional quantization blocks that let the shared latent cache serve both reduction directions. On nine benchmarks for DeepSeek-R1 and Kimi-K3, average accuracy matches BF16 MLA while the cache shrinks 3.20 and 3.05 times relative to BF16.

You’re all caught up