Swapping the student classifier for angular coordinates: ACD lifts homogeneous distillation by 2.51 points and heterogeneous pairs by 4.19
Knowledge distillation usually has students imitate a teacher's classifier outputs, but raw logits entangle transferable inter-class directional relations with parameterization artifacts such as feature magnitudes, weight norms and biases, producing a classifier geometry mismatch. Angular Conditional Distillation (ACD) rebuilds the student head as a normalized cosine readout that removes magnitudes and biases, then distills standardized conditional non-target relations, passing relative class relations without absolute confidence. On six homogeneous CIFAR-100 teacher-student pairs ACD improves on ordinary KD by 2.51 percentage points on average, and on five heterogeneous pairs it exceeds the strongest competing methods by 4.19 points on average.