Skip to main content
Search

# Pixel-Space Video Generation

2 related articles

Subscribe to updates
  1. arXiv

    Video diffusion drops the pretrained autoencoder: JVT reaches FVD 136.5 on UCF-101 with a plain pixel-space Transformer

    JVT moves video diffusion back into pixel space, denoising with a large-patch DiT-style Transformer and no pretrained autoencoder, cascaded stages or U-Net-style skip connections. Pixel-space video generation had mostly relied on U-shaped backbones or cascades, while latent DiTs dominate. On UCF-101 class-conditioned generation JVT reaches FVD 136.5 against 477.97 for the latent DiT baseline Latte, and joint RGB-disparity generation raises RTA@5 from 72.4 to 79.5 in the camera-conditioned task.

  2. arXiv

    PixVid moves video generation back to pixels: one token per frame, camera control lands on the next frame

    PixVid converts a pretrained latent video diffusion model into a frame-wise pixel-space generator: it keeps the diffusion transformer, replaces the VAE decoder with a convolutional detailer head, and finetunes from Wan2.1-14B. It reaches VBench totals of 85.09 at 960p and 84.78 at 480p, above Wan2.1's 84.05 and PixelUMM's 83.72, and keeps 98% of source motion on fast clips versus 83% for an identically finetuned latent model.

You’re all caught up