BiToK-SD learns Top-K token selection via bilevel optimization, achieving the best average scores across three scales in mathematical-reasoning self-distillation
The work proposes BiToK-SD, which casts Top-K token selection in on-policy self-distillation as a bilevel optimization problem: the lower level turns discrete Top-K selection into differentiable soft weights via a threshold relaxation that adapts as the student policy evolves, while the upper level performs knowledge distillation only on the selected positions; trained for 200 steps on roughly 30K problem-solution pairs from the mathematical-reasoning subset of OpenThoughts with Qwen3-1.7B/4B/8B, it attains the highest average scores among all compared methods on AIME24, AIME25, and HMMT25 under Avg@64, exceeding the strongest baseline OPSD by 0.63, 0.58, and 0.21 average points respectively.