Looped Transformers no longer wait a full loop per token: LoopFill merges suffix retrieval and shallow drafts into one tree, beating LoopAhead on seven benchmarks
Looped Transformers reuse the same layers repeatedly, so decoding each token requires several sequential passes and latency is high; earlier pipelined self-speculative methods drafted only along recurrent depth, one token at a time, and each token had to reach the drafting depth before the next could enter the pipeline. LoopFill combines multi-token continuations retrieved from suffix trees with the model's own shallow-loop predictions in a single draft tree, letting several future tokens enter the pipeline together while full-depth verification preserves the autoregressive output distribution. Across seven benchmarks under greedy decoding, its average speedup over autoregressive decoding exceeds the strongest baseline, LoopAhead, by 40% on Ouro-2.6B and 25% on Huginn-3.5B.