TaH2 allocates loop iterations per token and raises the AIME accuracy-compute slope by 53%
This work studies the test-time scaling of looped transformers through post-training, finds that existing looped models have steeper slopes yet underperform the non-looped baseline at matched compute, and proposes TaH2, which jointly post-trains the backbone and an iteration decider with lookahead depth supervision to label online which tokens benefit from further iterations, improving the AIME accuracy-compute slope from 1.79 to 2.74 (53%) and exceeding the baseline's peak accuracy by about 3.4 points at matched test-time compute.