Skip to main content
Back to timeline
arXivSource publication:

A recoverable motion signal in the intermediate states of frozen video diffusion model Wan2.1 is decoded into 3D human motion generation

Synopsis

This work asks whether a frozen large-scale text-to-video diffusion model implicitly contains explicit 3D human motion knowledge, probes Wan2.1 and finds a recoverable motion signal present in its intermediate states across the entire denoising schedule, and introduces parasitic co-denoising together with its instantiation, the Parasitic Motion Decoder (PMD), an efficient flow-matching decoder that shares the host's noise schedule and reads its intermediate features through sigma-adaptive multi-layer fusion, leaving the host unmodified, leading dedicated motion generators on text-motion alignment at a small fraction of their trainable parameters while producing paired video and motion in a single pass.

Source-provided article image: Parasitic Co-Denoising: Unlocking 3D Human Motion Generation in a Frozen Video Diffusion Model
Figure 1 · arXiv

Interpretation

Introduces parasitic co-denoising, a paradigm in which motion is decoded from the host model along its denoising schedule rather than produced by an independent generator. Relative to the common practice of training a separate motion generation model, this paradigm reframes motion generation as decoding internal signals of a frozen video diffusion model, without training a separate motion model. The paradigm is supported by probing a frozen Wan2.1, which reveals a recoverable motion signal present in its intermediate states across the entire denoising schedule, not confined to the clean output.

Instantiates the paradigm as the Parasitic Motion Decoder (PMD), an efficient flow-matching decoder that shares the host's noise schedule and reads its intermediate features through sigma-adaptive multi-layer fusion, leaving the host unmodified. Compared with approaches that modify or retrain the host model, or that train dedicated generators on motion data, PMD reuses host features via a shared noise schedule and sigma-adaptive multi-layer fusion while keeping the host unchanged. The abstract states that PMD is a flow-matching decoder sharing the host's noise schedule, reading intermediate features through sigma-adaptive multi-layer fusion, and leaving the host unmodified.

PMD leads dedicated motion generators on text-motion alignment at a small fraction of their trainable parameters, while producing paired video and motion in a single pass. Relative to motion-only baselines, PMD produces video and motion together in a single pass; relative to dedicated motion generators, its coverage is drawn from the host rather than from motion data, giving higher parameter efficiency. The abstract reports that PMD leads dedicated motion generators on text-motion alignment at a small fraction of their trainable parameters and produces paired video and motion in a single pass, which motion-only baselines cannot match.

Perspective

This work targets researchers and practitioners who want to extract explicit 3D human motion from an existing large-scale text-to-video diffusion model, in settings where the host model stays frozen and only a lightweight decoder sharing its noise schedule is trained. Its value lies in achieving text-motion alignment at a small fraction of the trainable parameters of dedicated motion generators and in obtaining video and motion together in a single pass, removing the need to train a separate motion model.

The abstract does not give specific values for text-motion alignment, the absolute scale of PMD's trainable parameters, quantitative details of the probing experiments, or generalization validation beyond the host model. Readers interested in these quantitative points should consult the main text and experiments.