FlexVSR freezes the streaming super-resolution prior and trains a 0.274M-parameter adapter, so one model covers continuous scales
Synopsis
Real-time streaming video generation needs non-integer, variable magnification at deployment, yet fixed-scale generative video super-resolution either computes at its native scale and downsamples at redundant cost or retrains the diffusion transformer per scale and alters the shared prior. FlexVSR separates prior preparation from scale adaptation: the VAE, super-resolution DiT and upsampler stay frozen while a 0.274M-parameter latent adapter, ReLSA, builds a target-grid latent and then applies a scale-conditioned residual correction. Supervised at only two scales, it holds reconstruction and temporal consistency on unseen continuous scales and runs at lower latency than most compared streaming baselines.
Interpretation
Streaming generative video super-resolution can now serve continuous target scales from one frozen prior instead of preparing a model per magnification. Earlier fixed-scale streaming super-resolution either computed at its native scale and downsampled to the target, paying redundant DiT computation and pixel decoding, or naively resampled latents and lost noticeable reconstruction quality. On YouHQ40 and Inter4K-40, FlexVSR has the best LPIPS in all four dataset-scale settings; on YouHQ40 its PSNR is 0.46/0.82 dB above the strongest baseline LiteVSRN and 0.74/1.07 dB above FlashVSR-TN, with an average rank of 1.67 across six metric columns versus 2.83 for LiteVSRN.
Scale generalization reduces to a lightweight adaptation of the latent interface, leaving the backbone untouched. ReLSA is a 0.274M-parameter (273,584) latent adapter that separates target-grid construction from residual refinement: LIIF-style continuous querying aggregates local low-resolution features into a base latent, and a scale-conditioned residual branch combines aligned source features and grid geometry to predict an additive correction. Adaptation trains only ReLSA while the feature extractor, super-resolution DiT and Wan VAE stay frozen, supervised end-to-end through the frozen restorer in 7.3 H100 GPU-hours, against 17.4 H100 GPU-hours for Stage-1 prior preparation; inference runs frame-wise without future-frame lookahead and keeps the single-step, chunk-causal streaming protocol.
Supervision at a few scales extrapolates to magnifications never trained on, with no per-scale retraining at inference. Adaptation is supervised at only two scales, whereas fixed-scale models cannot directly cover intermediate or nearby factors. Across ten YouHQ40 scales, FlexVSR has the best tLPIPS at all ten and the best LPIPS from 1.25x to 1.50x, ranking second to Ultra FlashN at 1.20x; a separate ReLSA adapter trained on 21 scales reaches the best overall average rank of 1.16 across 17 scales, five of them unseen.
Perspective
The demonstrated setting is streaming generative video super-resolution on a Wan2.1-1.3B restoration prior, with the main model supervised at only two scales and evaluated on YouHQ40 and Inter4K-40 using full-frame RGB outputs against ground truth at matched resolution; because upstream text-to-video generations lack paired multi-scale ground truth, reference evaluation synthesizes low-resolution inputs by bicubic downsampling. The authors point to dynamic scale switching during live generation as a future direction.
Across model families, the same prior-preparation recipe does not guarantee an equally strong reference-scale prior: MiniMax-H3 uses a 24-channel VAE with stronger spatial compression, and VAE encoding and decoding alone reaches 30.68 dB versus 32.66 dB for Wan2.1, with lower end-to-end scores, though that comparison does not isolate the factors. The residual magnitude also needs precise calibration, since scaling it at test time lowers PSNR.
