Video-editing rewards mistake frozen clips for quality: an inter-frame dynamics input cuts VE-Bench motion-ladder capture from 97% to 0%
Synopsis
Instruction-guided video-editing reward models treat temporal consistency as quality and so favor candidates whose motion has been suppressed: on the VE-Bench motion ladder, the evaluated composite reward picked a motion-suppressed candidate in 97% of cases. A motion-aware reward model feeds per-frame RGB differences in as an explicit input and trains on motion-suppressed and fabricated-motion negatives, with a distillation anchor preserving alignment to human quality ratings; it cuts ladder capture to 0% and also detects segment freeze, burst freeze and flow attenuation that were absent from training.
Interpretation
Video-editing rewards have an exploitable failure mode: when an instruction changes appearance but leaves the action unspecified, the reward prefers candidates with suppressed motion over edits that keep the original movement. On VE-Bench, the evaluated composite reward ranked a motion-suppressed candidate above the original edit in 97% of cases; removing its temporal-consistency component substantially reduced that preference. Human raters clearly preferred the original edit in pairwise comparisons, even on the mild hold-2 suppression that the rewards struggle to resolve, so the reward ranking misses a distinction people resolve clearly.
Making inter-frame dynamics an input modality lets the reward see motion directly instead of recovering it from features that no longer carry it. A linear probe on the base reward's penultimate features separates original edits from frozen clips but performs near chance on hold-2 and hold-4, with confidence intervals covering chance. The dynamics stream is a parameter-free tanh-normalized per-frame RGB difference of the edited video, concatenated channel-wise with pixel tokens at video-token block 4 of a Qwen3-VL-32B backbone and fine-tuned with LoRA.
Preference training against motion-suppressed and fabricated-motion negatives, plus a distillation anchor, yields motion sensitivity and human quality alignment together. Ranking-only fine-tuning turns the reward into a specialized motion detector and loses the human quality calibration inherited from the base preference reward. The final configuration reaches 0% capture on the VE-Bench motion ladder and rejects per-frame affine-jitter fabricated motion in 100% of cases; removing the distillation anchor drops hold-2 detection from 100% to 87%, and removing fake-motion negatives drops rejection from 100% to 53%.
Under reward-guided fine-tuning, the motion-aware reward keeps generated motion near its initial level instead of trading motion for score as the composite does. On UniVideo's genuine-motion subset, the composite reduces motion, while the motion-aware reward shows a mean motion change close to zero. On CogVideoX-2b, the motion-aware reward's mean motion change also stays near the initial level, while both the VideoScore and RAFT-flow reward arms reduce motion substantially.
Perspective
The conclusions rest on the VE-Bench motion ladder, reward-guided fine-tuning on the UniVideo and CogVideoX-2b editing pipelines, and a Prolific human pairwise preference study; the motion-aware reward is trained on internal edits and evaluated on editing models disjoint from its training data.
Hold-2 candidates retain a high-score tail, so discrimination depends on within-source ranking rather than a single threshold; correlation with raw RAFT motion magnitude is weak, which the authors read as learning beyond motion magnitude without establishing statistical independence.
