Long videos stop drifting: Reference Forcing anchors each new frame to the first, holding consistency over two hours
Synopsis
Autoregressive video diffusion models generate video chunk by chunk, but short-window distillation does not constrain long-range appearance, so subjects and backgrounds drift over time. Reference Forcing treats the initial frames as an appearance reference: a teacher built from segmentation and optical-flow models forces each region of a new frame to attend only to the same region of the initial frames, and this behavior is distilled into a separate anchor attention path in the student, which also guides correction of drifted values in the KV cache. On the LongLive base model it achieves the highest subject and background consistency at 30 and 60 seconds, and its advantage grows with video length.
Interpretation
Long video generation gains a route to consistency that does not sacrifice motion: each region of a new frame can refer only to the same region of the first chunk instead of competing with recent frames for one set of attention weights. Earlier approaches gave the model access to more generation history, for example keeping the first frames as an attention sink, compressing older frames, or retrieving history; but previously generated frames may already contain appearance errors, while stronger attention to the first frames freezes motion. On 60-second videos generated from MovieGenBench prompts with LongLive as the base, Reference Forcing reaches subject consistency 97.56 and background consistency 96.67, above LongLive's 97.33 and 96.49, while dynamic degree is 43.13 against LongLive's 42.19.
The implementation splits into two complementary corrections: one brings first-chunk appearance into the current frame, the other cleans appearance that has already drifted in the recent cache. The anchor path is a separate attention branch that attends only to the first chunk and adds no rotary positional encoding, so it matches by content rather than position; it is distilled from a teacher that segments the subject with Grounding DINO and SAM 2, matches the background with RAFT optical flow, and uses attention masks to restrict each region to the same region of the first chunk. The student adapts queries and keys with rank-16 LoRA and learns an anchor gate that sets how much each token uses the anchor path; no external models are needed at inference. Value correction matches tokens by mutual nearest neighbors in key similarity and interpolates recent-cache values toward their first-chunk counterparts by the gate, leaving the newest chunk untouched to avoid flicker.
Perspective
The demonstrated scope is 30-second, 60-second and 120-second videos from MovieGenBench prompts, on the LongLive-1.3B and Self-Forcing 1.3B base models, evaluated with six VBench-Long dimensions, DINOv2 long-range consistency and MiraBench tracking strength.
The reference uses only the first chunk, so content that first appears later has no reference, and updating the reference during generation is left for future work; the anchor path averages over the matched region of the first chunk, so imaging quality stays slightly below LongLive; the teacher's segmentation follows a single subject, so scenes with several similar subjects remain difficult.
