Skip to main content
Search

Asim Kadav

1 related articles

Subscribe to updates
  1. arXiv

    An equivalent reparameterization of the output head cuts Phi-4-mini W4 AW-MSE KL from 0.936 to 0.256 while keeping a 10.8% batch-one latency gain

    The work proposes softmax reparameterization: before quantization, subtract a scalar multiple of the vocabulary-row mean from every output-head row and pick the coefficient by validation KL separately for RTN, AW-MSE and full-Hessian GPTQ, preserving the full-precision softmax distribution and the trained decoder while improving low-bit output-head fidelity, for example lowering Phi-4-mini W4 AW-MSE KL from 0.936 to 0.256 and reducing batch-one generation latency by 10.8% relative to a BF16-head baseline in packed W4 deployment.

You’re all caught up