One video diffusion transformer now plays both dynamics: O2LaF learns actions from unlabeled video and transfers them zero-shot from humans to robots
O2LaF repurposes a single pretrained video diffusion transformer (video DiT) into both an inverse-dynamics encoder and a forward-dynamics generator, so the two share one video prior and learn transferable latent actions from video without action labels. Earlier inverse-dynamics models were largely trained from scratch, or pretrained video generators served only as the forward model. Across camera navigation and robotic manipulation, O2LaF sets a new state of the art in few-shot linear probing and supports zero-shot action transfer across visual contexts and human-to-robot embodiments.