Skip to main content
Search

Bingxi Hou

1 related articles

Subscribe to updates
  1. arXiv

    Cross-tokenizer on-policy distillation: strict alignment already covers 85.57–96.98% of student tokens, a student-selected top-16 subset retains at least 96% of the gain, while adding span supervision on mismatch groups lowers accuracy

    Across three heterogeneous teacher–student pairs on mathematical reasoning and code generation, this study finds that strict alignment already covers 85.57–96.98% of student tokens despite static vocabulary Jaccard overlap of only 39.49–64.87%, that the shared vocabulary retains nearly all predictive probability mass at strict positions, that restricting reverse KL to a student-selected top-16 subset of the shared vocabulary preserves at least 96% of the full-shared-vocabulary improvement, and that adding span log-probability MSE on mismatch groups achieves complete supervision coverage yet lowers accuracy in all 18 positive-weight settings.

You’re all caught up