GDN recurrent states drop to 4 bits: loss-guided bit allocation plus a recent-write window keeps long reasoning from stalling
Synopsis
Linear-attention models compress history into a fixed-size recurrent state, and once that state is quantized to low bits, writeback errors enter every later recurrence, so long-reasoning scores collapse and outputs turn repetitive and overlong. GGA allocates head bit widths from the gradient of language-model loss with respect to the writeback state, while PW keeps recent pure writes as FP8 vector pairs and rolls them into the low-bit historical state by write age. Across seven evaluations on Qwen3.5-4B/9B, GGA@ raises the macro-average from STEPQuant@'s 31.44/37.42 to 50.92/67.26, and adding PW-S8 reaches 69.83/78.63.
Interpretation
Bits for the recurrent state are no longer spread by local distortion but by how much each head affects the final prediction loss. STEPQuant previously ranked risk from error persistence, candidate reconstruction distortion and readout weights; in the Qwen3.5-4B diagnostic bins it gave lower average bit width to the higher-impact bin. Holding the candidate quantizers and nominal budget fixed and replacing only the allocation lifted the seven-task macro-average on Qwen3.5-4B/9B from STEPQuant@'s 31.44/37.42 to GGA@'s 50.92/67.26.
Recent writes no longer enter the low-bit historical state immediately; they are held as separate branches in a fixed-length window. Stepwise full-state quantization folds a new write into the accumulated state right after the current readout, and it is then requantized along with the history. Adding PW-S8 on top of GGA@ raised the 4B/9B macro-average to 69.83/78.63; on one HMMT problem with the same prompt and seed, output went from 81,921 tokens cut off by the repetition detector after 831 consecutive "Wait." lines to 57,682 tokens ending naturally with answer 48.
At similar storage budgets, spending the extra capacity on maintaining recent writes beats raising the bit width of the accumulated state. Higher average bit width reduces distortion in the accumulated state, whereas the window spends capacity on delaying when recent writes are first quantized to low bits. Pairing @ with @+PW-S8, where the window configuration needs about 1.03 more bits/value, gave higher macro-averages and shorter task-equal mean outputs under both STEP and GGA allocation on both models.
Perspective
Evaluation covers Qwen3.5-4B and Qwen3.5-9B on seven tasks (AIME 2026, HMMT February 2026, GPQA Diamond, HumanEval+, MATH500, IFBench and LiveCodeBench v6) at the @, @ and @ nominal budgets; bit widths and FP16 pivots are calibrated offline on 32 WikiText-2 segments of 2048 tokens and then frozen, with no backpropagation at deployment.
Recurrence equivalence for PW is proved under ideal precision, but finite-precision rounding, quantization feedback and activation changes during free generation fall outside those assumptions, so elementwise errors are not guaranteed to shrink. The mechanisms behind overlong and repetitive generation are not fully characterized, systematic experiments on window length, queue precision and branch merging remain incomplete, dedicated inference kernels have not been developed, and coverage of model families and comparison methods is limited.
