Sparse attention moves long-context cost into the indexer: it takes 47% of a DeepSeek V3.2 prefill layer
In long-context inference, sparse attention lets each query attend to only a few keys, but the indexer that picks those keys still scores all N of them, so the cost moves rather than disappears. In a DeepSeek V3.2 prefill layer, the indexer and Top-k together take 47% of the time. A sweep over a production FP8 kernel yields eight design guidelines, chiefly keeping the product of query heads per group and query rows per step inside one Tensor Core tile and folding Top-k into the indexer budget.