Selecting distillation positions by entropy shift lifts LLM judges 2–9 points over Dr. GRPO on subjective tasks
This work studies training LLM judges from natural language feedback, proposes using the per-position entropy shift between teacher and student to distinguish context sharpening from context spreading, and masks high-entropy-shift positions so that self-distilled judges outperform outcome-supervised RL (Dr. GRPO) by 2–9 percentage points on the evaluated subjective subcategories while remaining competitive on objective ones.