Pruned MoE experts don't need retraining: GEAR² reuses already-activated experts per token and beats mean substitution on the nine-task zero-shot average at 50% pruning
Pruning experts from a mixture-of-experts language model saves GPU memory but discards what the removed experts contributed to the tokens routed to them. GEAR² leaves the main model untrained: it approximates each pruned expert's output by scaling an expert already activated for the same token, falls back to the mean output when a reliability gate judges that proxy unreliable, and adds a low-rank residual weighted by the missing routing mass. Across five MoE models from 16B to 106B parameters, 25% and 50% pruning, and nine zero-shot tasks, it gives the best pruned result in all 10 settings, and at 50% pruning it beats same-mask mean substitution by 0.96 to 3.44 points.