Skip to main content
Search

# Mathematical Reasoning

56 related articles

Subscribe to updates
  1. arXiv

    Long-sequence distillation stops handing the teacher the whole answer: ATI-OPD rewrites only the earliest divergent segment, and students' average accuracy on their own beats standard on-policy distillation

    On-policy distillation trains a student on its own responses using token-level teacher feedback, but as responses grow longer the student's prefix drifts from the teacher's own reasoning and later feedback weakens, bringing gradient spikes and extremely long responses. ATI-OPD has the teacher insert a short replacement at the earliest high-disagreement segment, hands generation back to the student, and excludes persistently divergent suffixes from the distillation loss. Across five mathematical reasoning benchmarks and two 1.5B teacher-student pairs, average student-only accuracy exceeds standard on-policy distillation.

  2. arXiv

    Diffusion language models learn when to rewrite: DiVR lifts a fine-tuned generator's five-benchmark average from 35.3% to 43.4%

    Masked diffusion language models can remask generated text, but deciding where to revise by confidence or self-judgment often leaves the error in place. DiVR instead learns an outcome verifier on the generator's own decodes and, at each block boundary, judges whether the partial solution can still reach a correct answer, resampling the whole block when the score is low. On the supervised fine-tuned LLaDA-8B-Base generator, the five-benchmark math and code average rises from 35.3% to 43.4% at the cost of greedy decoding.

  3. arXiv

    AI has its own proving style: reproving all 465 propositions of Euclid's Elements, AI runs match each other at 0.68 versus 0.37 with humans

    Three large language models independently reproved all 465 propositions of Euclid's Elements, and their citation choices differ systematically from Heath's human-proof baseline. Mean pairwise Jaccard similarity among three Claude Opus 4.6 runs is 0.68, nearly twice the 0.37 between AI and human proofs, so the difference is stable and reproducible. Its concrete signature is that about 40% of AI citations point to foundational elements such as definitions, common notions and postulates, against only 13% in human proofs.

  4. arXiv

    Masked diffusion models lock answers too early: HiLo-RD distills large-window trajectories, lifting GSM8K accuracy by 9.4 points

    Masked diffusion language models can fix answer fragments before the supporting reasoning is resolved when the response window is small, and those fragments then become context for later predictions. HiLo-RD uses the same model's large-window trajectory as a frozen teacher and aligns its hidden representations to cropped small-window views at committed positions. At the same small inference budget, GSM8K accuracy rises from 68.84% to 78.24% while mean readable length falls from 362.8 to 271.3 tokens, and the gains extend to code generation without code-domain training.

  5. arXiv

    Tool calls get worse with training: TACA supervises the reasoning-to-call transition and lifts six-benchmark accuracy by 4.61 points

    In multi-turn tool-use reinforcement learning, reasoning accuracy can rise early and then fall below the non-agentic baseline; TACA traces this to negative aggregate credit on tool-call tokens and instead supervises the step where reasoning turns into a tool call. Against MT-GRPO it raises mean accuracy over six reasoning benchmarks by 4.61 and 3.61 percentage points for Qwen2.5-1.5B and 3B, while tool execution success during training stays at 86.28%.

  6. arXiv

    Self-distillation's teacher also needs teaching: TeachBack's bidirectional preferences lift Qwen3 math averages by up to 4.3 points

    On-policy self-distillation lets one model act as a problem-only student and as a teacher given a reference solution, but the reference pushes the teacher toward short, answer-conditioned derivations that omit exploration and self-correction. TeachBack injects a trailing slice of the reference reasoning as optional working notes and optimizes preferences in both directions: teacher trajectories should win under the problem-only prompt, while student trajectories reshape future privileged teacher rollouts. Across Qwen3-1.7B, 4B and 8B, math averages rise by 4.3, 2.2 and 2.5 points over base, and Knight-and-Knaves averages by 6.1, 3.9 and 7.5 points.

  7. arXiv

    When every rollout fails, PICO turns all-wrong groups into trainable trajectories with teacher-constrained joint decoding

    Verifiable-reward reinforcement learning hits a learning cliff on problems beyond the model's reach: when a rollout group is entirely wrong, group-relative advantage collapses to zero and the update carries no signal. PICO triggers distribution-constrained joint decoding only on all-failure groups, intersecting the teacher's next-token candidates with the student's while a dynamic cutoff keeps the sampling distribution inside a finite divergence budget, yielding trajectories that pass verification and stay close to the current policy. On Qwen3-4B-Base and Qwen3-8B-Base, PICO averages 32.0 and 35.4 across AIME25, AIME26, HMMT26, MATH500, GPQA-Diamond and HLE, against 27.9 and 31.8 for GRPO.

  8. arXiv

    Diffusion language models can learn from autoregressive teachers: JustOPD lifts Dream-7B's math reasoning while keeping any-order decoding

    Autoregressive models generate strictly left to right, while diffusion language models denoise in arbitrary order, so the former's token-level supervision does not transfer directly. JustOPD forces the diffusion student to roll out left to right during training, letting a Qwen3-14B teacher supervise every intermediate state, while inference keeps any-order decoding. With Dream-7B as student and Qwen3-14B as teacher on GSM8K and MATH500, scores rise over the base model across several divergences.

  9. arXiv

    GRPO's group baseline can now tilt two ways: strategy-based subgroup offsets lift multi-sample coverage modestly but consistently

    GRPO scores every rollout of a prompt against one shared group-mean baseline, yet rollouts of the same prompt often follow different solution strategies with different reward distributions. This work defines the subgroup offset as a subgroup mean's deviation from the global mean and shifts the baseline along it by a single signed coefficient: the positive sign pulls credit toward a rollout's standing inside its subgroup, the negative sign pulls it toward the subgroup's overall quality. On Qwen3-8B-Base and Olmo3-7B-Base across four competition-level math benchmarks, both directions give modest but consistent gains over an anchor that shares the same strategy-conditioned sampling and differs only in the baseline, concentrated in pass@16, with the negative direction stronger.

  10. arXiv

    When students write wrong, teachers drift too: SPTA first teaches the teacher to continue from a student's half-finished reasoning

    On-policy distillation trains a student on trajectories the student itself generates, but student prefixes can carry reasoning errors or patterns unfamiliar to the teacher, and the teacher's token-level supervision becomes less reliable as prefixes grow longer. Before distillation begins, SPTA trains the teacher on student prefixes of random lengths, rewarding correct final answers, so the teacher learns to recover from student-induced states rather than imitate the student's reasoning; standard on-policy distillation then proceeds with the distillation objective unchanged. On mathematical reasoning benchmarks, students trained with SPTA average better than under standard on-policy distillation and better than under on-policy distillation with task-level RLVR-adapted teachers.