Skip to main content
Back to timeline
arXivSource publication:

A model can teach itself to reason without an answer key: Stone moves agreement to easier stepping stones, gaining 13.2 and 22.7 points without thinking mode

Synopsis

On-policy self-distillation lets a language model be its own teacher, but that teacher still reads a ground-truth solution. Stone instead has the model write easier related problems for the ones it is unsure about, keeps only those it solves consistently, and shows the teacher their worked solutions with final answers removed; the student sees only the original problem. Across five math benchmarks, Qwen3-4B and Qwen3-8B reach the highest averages in thinking mode and gain 13.2 and 22.7 points without thinking mode, 2.5 and 2.7 times the gains of OPSD with ground-truth solutions.

Source-provided article image: Simplify, Agree, Distill: Label-Free Self-Distillation with Stepping Stones
arXiv

Interpretation

A language model can learn reasoning from itself without reading ground-truth answers, by moving the judgment of self-agreement from the hard problem to easier related problems it writes itself. Earlier on-policy self-distillation gave the teacher a ground-truth solution, while the label-free substitute, voting over attempts at the original problem, fails precisely on the hard problems worth learning from. Among the 9,218 training problems where Qwen3-4B is unsure, 28% have no answer shared by half its attempts, and on the rest the most common answer is wrong 31% of the time; the stepping stones written for these problems average 0.77 agreement over 8 attempts, against 0.54 on the problems themselves.

During training the teacher sees the stepping stones and their worked solutions with final answers removed, while the student sees only the original problem, so inference needs neither stones nor an extra model call. The teacher no longer reads a ground-truth solution but reads simpler problems the model wrote and passed through an agreement filter. Across five math benchmarks, Qwen3-4B and Qwen3-8B average 77.55 and 79.20 in thinking mode, above OPSD with ground-truth solutions at 76.20 and 77.97; without thinking mode they reach 54.17 and 66.29, 7.9 and 14.3 points above OPSD's 46.29 and 52.04.

The keep rule requires a stone to be answered consistently, more consistently than the original problem, and with an answer different from the model's most common answer to that problem, which confines the teacher's context to problems where the model is reliable. A stone sharing the original problem's answer is usually a restatement and would hand the teacher the model's often-wrong vote on that problem. Claude Opus 5 audited the 27,563 stones Qwen3-4B writes in thinking mode: 78% of kept stones are judged well-posed and correctly answered, against 27% of rejected ones, and kept stones are sound 76% of the time when the problem has no usable vote and 81% when its vote is wrong.

Perspective

The demonstrated setting is Qwen3-4B, Qwen3-8B and Gemma 4 E4B trained on the 29,434 math problems of the OPSD training set, evaluated on AIME24, AIME25, HMMT25, MATH500 and AMC23, with training updating only LoRA adapters.

Agreement on a stone is not correctness: the audit judges about one in five training stones ill-posed or wrongly answered, and that audit is itself model-based rather than human verification; generating and checking stones adds preprocessing time before distillation; whether the gains persist at larger scales remains open.