Video diffusion drops the pretrained autoencoder: JVT reaches FVD 136.5 on UCF-101 with a plain pixel-space Transformer
JVT moves video diffusion back into pixel space, denoising with a large-patch DiT-style Transformer and no pretrained autoencoder, cascaded stages or U-Net-style skip connections. Pixel-space video generation had mostly relied on U-shaped backbones or cascades, while latent DiTs dominate. On UCF-101 class-conditioned generation JVT reaches FVD 136.5 against 477.97 for the latent DiT baseline Latte, and joint RGB-disparity generation raises RTA@5 from 72.4 to 79.5 in the camera-conditioned task.