Equal rubric weights do not mean equal reward: DCC clips each criterion's score to a dual-center interval and lifts ranking accuracy
Synopsis
Rubric-based reinforcement learning usually adds up per-criterion judge scores with equal weights, yet criteria differ in how far their scores separate satisfied from unsatisfied responses, so equal weights do not give equal contributions. The work formalizes this as reward gain imbalance and derives an exact split of the reward difference between two candidates into a base satisfaction signal, an imbalance term and a residual; the imbalance term can amplify or attenuate the base signal, and misrankings concentrate on pairs it attenuates. The proposed Dual-Center Clipping aggregation (DCC) estimates two centers from the current candidate group alone and bounds each criterion's contribution; it beats the compared methods on the Exact metric across two datasets and two judge models.
Interpretation
In rubric-based rewards, each criterion's actual influence on the final reward depends both on the importance weight it is assigned and on its reward gain, the expected score gap between satisfied and unsatisfied responses. Prior practice summed criterion scores with equal weights, or tried to ease scoring heterogeneity by learning criterion weights, transforming score representations or redesigning aggregation, while how that heterogeneity affects candidate ranking remained unclear. Estimated from Mistral-7B scores on the PointRubric train split, the reward gain of raw scores has mean 21.9545 and variance 86.2175, so the gain is neither zero nor constant across criteria.
Removing reward gain imbalance outright does not improve ranking, even when calibration uses ground-truth labels. The intuition was that equalizing every criterion's gain would make contributions equal and bring rewards closer to the true order. After calibration, Exact falls from 75.30 to 70.26 on PointRubric and from 14.87 to 12.39 on RubricHub-6 for Mistral-7B raw scores; in the supplementary runs, Qwen3.6-27B raw scores on RubricHub-6 fall from 36.99 to 21.95.
The reward difference between any two candidates splits exactly into a base satisfaction signal, a reward gain imbalance term and a residual term, and the imbalance term can either widen or narrow the base signal. The decomposition turns the puzzle of unequal contributions under equal weights into testable terms and locates frequent misrankings on pairs where the imbalance term attenuates the base signal. For Mistral-7B on PointRubric, raw scores rank 99.00% of pairs correctly where the imbalance term is positive but only 86.44% of pairs where it is negative.
Rather than removing the imbalance, DCC clips each criterion's score to two centers fitted from the current candidate group, capping how much any single criterion can move the reward difference. Binary aggregation collapses scores into two levels, losing continuity and creating many ties, while probability aggregation saturates at a fixed point on the judge's score scale and saturates prematurely. DCC has the highest Exact among compared methods on both datasets and both judges: 1.92 and 1.24 percentage points above the best competitor under Mistral-7B, and 0.24 and 3.36 points under Qwen3.6-27B; as a GRPO reward for fine-tuning Mistral-7B, it raises the WritingBench overall score from a 43.32 baseline to 52.24 averaged over three seeds.
Perspective
The results rest on the PointRubric train split and on RubricHub-6 built from RubricHub, with Mistral-7B-Instruct-v0.3 and Qwen3.6-27B as judges and all aggregation methods operating on the same scores; GRPO fine-tuning trains for one epoch on the subset of RubricHub Writing prompts with 31 or 32 criteria and is evaluated with WritingBench's official settings.
Judge scores alone do not reveal which criteria a response truly satisfies, so no score-only aggregation method, DCC included, can guarantee a correct ranking for every candidate group; the precise score conditions under which DCC corrects a misranked pair are left for future work. Ablation shows removing the lower bound brings no measurable gain in ranking accuracy, but a new judge's or dataset's score distribution cannot be known in advance, so DCC keeps both bounds.
