Distillation updates were going to style words: SD-OPD takes strength from the student's own shift and beats OPD by 7.76 to 10.37 points on six-task math average
Synopsis
On-policy distillation (OPD) uses the teacher–student log-probability difference as each token's update strength, but on math reasoning responses that difference at style-related positions averages 3.5 times its value at mathematical positions, so updates lean toward matching the teacher's expression. SD-OPD instead references the student's own pretrained base, sets update strength from the student–base log-probability change, keeps a coefficient only when its sign agrees with teacher feedback, and recomputes the student signal after each training round. Across 5 teacher–student configurations with Qwen3 students of 1.7B to 8B parameters, SD-OPD raises six-task mathematics average pass@1 (avg@16) over OPD by 7.76 to 10.37 percentage points.
Interpretation
The strength of a distillation update on math reasoning is no longer set by how far the teacher and student disagree, but by changes the student has already made relative to its own pretrained base, with the teacher deciding only direction. Earlier on-policy distillation used the teacher–student log-probability difference directly as a per-token update coefficient, and that difference mixes expression differences with reasoning gaps. On 8,192 DAPO-Math responses, the mean absolute teacher–student log-probability difference is 1.226 at style-related positions and 0.347 at mathematical positions, a 3.5-fold gap that persists with a larger teacher and on GSM8K.
The method takes centered student–base log-probability differences as per-token coefficients and gates them by the sign of teacher feedback. The student–base change is itself concentrated at style positions, with a style-to-mathematics amplitude ratio of 4.1, so it needs selection. Under the same sign gate, teacher–student magnitudes leave a style-to-mathematics ratio of 3.7, while student–base magnitudes lower it to 3.1 and cut style amplitude by 40.3%.
Across 5 teacher–student configurations, SD-OPD beats OPD on all six mathematics benchmarks and posts the highest six-task average in every configuration. The comparison includes the initial student, OPD and ExOPD under matched rollout and response budgets. Six-task averages improve by 7.76 to 10.37 points over OPD and by 3.45 to 14.89 over ExOPD; with Qwen3-30B-A3B-Instruct-2507 as teacher, the 8B student averages 76.11%, above the teacher's 73.82%.
Perspective
Evaluation covers three teacher checkpoints from two model families and Qwen3 students of 1.7B, 4B and 8B parameters on mathematical reasoning, using AIME24, AIME25, AIME26, HMMT25, AMC23 and MATH500 with shared rollout and response budgets across methods.
The token analysis uses fixed rule-based categories applied unchanged across models and datasets; the method requires the student's pretrained base and adds one base scoring pass per round; the authors list larger students and other domains as further work, and the length and entropy curves have not fully plateaued.
