Skip to main content
Search

# MassAlloc Attention

1 related articles

Subscribe to updates
  1. arXiv

    MALA lets attention allocate its own post-score compute from normalized contribution, cutting 128K training forward/backward latency to 1/2.2 and 1/3.0 of FullAttn

    The work introduces MassAlloc Attention (MALA), a fused attention primitive that keeps QK score discovery over every legal causal interaction and uses normalized online-softmax contribution to decide which tiles execute post-score computation; under exactly matched post-score work at 8K it reaches 0.0188% mean omitted mass versus 0.0182% for a per-instance reference-mass oracle, holds low output and gradient errors from 1K to 32K, reaches 89.67% associative-recall accuracy at 8K versus 89.97% for FullAttn, reduces training forward and backward latency by 2.2x and 3.0x and decoding latency by 1.6x at 128K with tensor parallelism on 8 GPUs, tracks FullAttn perplexity from 0.6B to 14B while cutting total training FLOPs by 23.

You’re all caught up