Skip to main content
Search

# ExpertAlign

1 related articles

Subscribe to updates
  1. arXiv

    MOPD-Router replaces domain-label hard routing with token-level routing, lifting multi-teacher distillation overall score by 5.88 points on unlabeled mixtures

    The work introduces MOPD-Router, a multi-teacher on-policy distillation framework that needs neither domain labels nor a separately trained routing model, selecting and weighting the full teacher pool at every token, and proposes ExpertAlign, a metric that scores each teacher by the positive cosine alignment between the specialization it acquired relative to the shared pre-RL base and the teaching direction it would apply to the current student; across unlabeled and domain-labeled training mixtures under strong-to-weak and same-size distillation, ExpertAlign achieves the strongest overall performance in all four settings, improving the overall score by 5.88 points (+12.3%) over Mean aggregation on unlabeled data and by 3.95 points (+7.

You’re all caught up