Skip to main content
Search

# Few-Step Diffusion Distillation

2 related articles

Subscribe to updates
  1. arXiv

    Few-step diffusion students lose directions, not variance: ReDiverse restores diversity at one intermediate block

    Distilling a diffusion model into a four-step student makes different random seeds produce nearly identical images and videos for the same prompt. By comparing teacher and student activations block by block, ReDiverse finds that the student keeps its total activation variance but concentrates it into far fewer directions, at one intermediate block during the first denoising step. ReDiverse-FT then uses LoRA at that block to regress the student's first-step activations onto the teacher's, while ReDiverse-TF perturbs activations along teacher principal directions; across four teacher-student pairs on SDXL, SD3, Wan 2.1 and MiniMax-H3, both restore diversity while largely preserving generation quality.

  2. arXiv

    VA-TDM swaps four-step supervision for a velocity-field adversarial game: 45.53% human preference against the many-step teacher on 33B MiniMax-H3

    Few-step diffusion distillation has long relied on estimating the student score, yet at intermediate steps that score is hard to track and the estimated distribution can differ from the one used to update the student. VA-TDM instead supervises velocity fields built from the student's own sampling process, uses a learned critic to compare student and teacher fields and turns their disagreement into generator feedback, unifying score distillation, Fisher matching and moment matching. On four-step SD3.5-Medium it raises GenEval from TDM's 0.61 to 0.67; on 33B MiniMax-H3 at the same 4-NFE budget it wins 61.33% of human comparisons against Fast-H3 and reaches a 45.53% preference rate against the 100-NFE teacher, versus 24.67% for Fast-H3.

You’re all caught up