Google Research introduces Diffusion Controller: a lightweight steering-damper network that beats LoRA on HPS-v2 win rates in gray-box settings, with a white-box version reaching a 90% win rate over baseline
Google Research engineers Chih-wei Hsu and Moonkyung Ryu present the Diffusion Controller framework, which reframes the diffusion denoising process as a smooth continuous control problem and uses a lightweight steering-damper network to dynamically correct the generation trajectory while the base model stays frozen; evaluated on a Stable Diffusion v1.4 backbone across SFT, RWL, and PPO regimes with the standardized Human Preference Score (HPS-v2), the framework is reported to outperform corresponding baselines in both white-box and gray-box settings, with the gray-box version beating LoRA on HPS-v2 win rates in the SFT and RWL tracks while manipulating significantly fewer internal model layers, and the white-box version achieving a 90% win rate over the baseline, all with a single inferenc