Skip to main content
Search

Julianna Piskorz

1 related articles

Subscribe to updates
  1. arXiv

    Controlled distillation study finds on-policy rollouts are not universally better, with KL direction and learning rate dominating

    Independently varying rollout policy, token-level KL direction, and learning rate in strong-to-weak distillation across Llama 3 and Qwen2.5 model families and scientific, medical, and arithmetic reasoning tasks, the study finds no consistent advantage for on-policy rollouts in final accuracy, catastrophic forgetting, or parameter-update sparsity; forward KL is robust to rollout policy while reverse KL is substantially more sensitive and favors student-generated rollouts, learning rate governs forgetting and sparsity, and on-policy data helps only for generalisation to harder Countdown variants, an advantage that does not reliably persist after subsequent RLVR.

You’re all caught up