Skip to main content
Search

North South University

1 related articles

Subscribe to updates
  1. arXiv

    OASIS restricts self-distillation supervision to verified trajectories, widening its average gain over OPSD from 0.59 to 3.05 points across Qwen3-1.7B, 4B and 8B

    Using truncated-continuation probes, the work finds that in on-policy self-distillation (OPSD) the teacher's advantage over the student is concentrated on failed trajectories (77% versus 37% recovery at 1.7B) and shrinks with scale, and therefore proposes OASIS: keeping the OPSD objective unchanged, it applies supervision only along the shortest outcome-verified on-policy trajectory and replaces the written reference solution with a distinct same-problem self-generated rollout, typically the student's own unverified attempt, improving mean Avg@12 over OPSD by 0.59, 1.86 and 3.05 points on Qwen3-1.7B, 4B and 8B across AIME 2024, AIME 2025 and HMMT 2025 while requiring no written solutions.

You’re all caught up