Skip to main content
Search

# Reverse KL Divergence

1 related articles

Subscribe to updates
  1. arXiv

    Tsinghua LeapLab replaces KL divergence with a binary directional reward, matching or beating on-policy distillation on math and code, and proposes C-MOPD for multi-teacher training

    This work revisits whether reverse KL divergence is necessary for on-policy distillation (OPD), proposes BinaryOPD, which keeps only the update direction as a binary reward and matches or slightly exceeds OPD across multiple student-teacher pairs on math and code, further finds that only the direction of a small subset of high-disagreement tokens is critical, and builds on this to propose C-MOPD, in which every sample is supervised by all teachers and which consistently outperforms MOPD on math and code benchmarks.

You’re all caught up