Skip to main content
Search

# Diversity Collapse

2 related articles

Subscribe to updates
  1. arXiv

    Few-step diffusion students lose directions, not variance: ReDiverse restores diversity at one intermediate block

    Distilling a diffusion model into a four-step student makes different random seeds produce nearly identical images and videos for the same prompt. By comparing teacher and student activations block by block, ReDiverse finds that the student keeps its total activation variance but concentrates it into far fewer directions, at one intermediate block during the first denoising step. ReDiverse-FT then uses LoRA at that block to regress the student's first-step activations onto the teacher's, while ReDiverse-TF perturbs activations along teacher principal directions; across four teacher-student pairs on SDXL, SD3, Wan 2.1 and MiniMax-H3, both restore diversity while largely preserving generation quality.

  2. arXiv

    Modeling generative models as nodes in a directed weighted graph shows cross-model synthetic-data flow jointly sets the convergence threshold and diversity contraction of self-consuming training systems

    This work models multiple generative models that consume each other's synthetic data as nodes in a directed weighted graph, derives sufficient conditions for local asymptotic stability of the retraining dynamics via the Jacobian and a comparison matrix, and gives an upper bound on system diversity at the fixed point, showing that stability is jointly governed by local amplification factors and graph propagation while synthetic-data consumption induces a topology-dependent smoothing effect that contracts inter-model heterogeneity; experiments on an 8-mode Gaussian dataset and CIFAR-10 across several generative model classes and interaction topologies align with the theoretical predictions.

You’re all caught up