Robot policies stop predicting the whole future frame: ResWAM learns only frame-to-frame residuals and reaches 81.1% on LIBERO-Plus
Synopsis
World-action models usually have the policy predict complete future observations, but little changes within an action chunk: on LIBERO, current and future frame features share an average cosine similarity of 0.96, so the future loss is largely satisfied by copying static background. ResWAM instead supervises only the residual from the current frame to the future frame in a frozen DINOv3 feature space, and a 0.5B dual-branch DiT jointly denoises actions and residuals within 2.3B total parameters. Without robot pretraining it reaches 81.1% average success on LIBERO-Plus and 93.8% on RoboTwin 2.0, at 2.7–4.0 and 1.0–1.4 TFLOPs of training and inference compute.
Interpretation
The supervision target of a world-action model shifts from complete future observations to the feature residual between the current and future frames, so the policy learns how the scene changes rather than what the future looks like. Earlier world-action models either reconstruct complete future frames with a video generation model or attach a separate world model that predicts complete future features, leaving the supervision dominated by nearly unchanged background. On LIBERO, current and future DINOv3 features share an average cosine similarity of 0.96, so the complete-future target is largely satisfied by copying current content; the residual target raises effective rank from 15.22 to 22.06 and stable rank from 3.08 to 5.48.
The residual branch and the action branch share one lightweight dual-branch DiT and are denoised together in latent space, with no pixel-level reconstruction and no separate world model. Video-generative world-action models fine-tune large video diffusion models with 3D video VAEs, while external-world-model designs query a separate pretrained world model first. ResWAM totals 2.3B parameters, including a 0.5B dual-branch DiT trained jointly with conditional flow matching; on all 50 bimanual RoboTwin 2.0 tasks, removing the residual branch drops average success from 93.8% to 89.9%, and predicting complete future features gives 90.9%.
Without robot pretraining, ResWAM reaches the highest average success among compared methods on three simulated benchmarks and on real FR3 arm tasks, at the lowest training and inference compute. Comparisons include VLAs and world-action models with robot pretraining, such as the strongest VLA baseline on LIBERO-Plus, VLA-JEPA at 79.5%, and the strongest world-action model baseline, LaWAM at 76.6%. 81.1% on LIBERO-Plus, 93.8% on RoboTwin 2.0, and 9.40% success with 13.41 score on RoboDojo; the real FR3 Open, Put and Stack tasks average 80.0% at 92.7 ms per-chunk latency.
Perspective
Validation covers all 10,030 perturbed episodes of LIBERO-Plus, all 50 bimanual RoboTwin 2.0 tasks, all 42 RoboDojo tasks over 2,100 episodes, and 30 trials per task on three tabletop tasks, Open, Put and Stack, with a Franka Research 3 arm; every result is obtained without robot pretraining.
The effective-rank and stable-rank advantage of the residual target is reported from window statistics on LIBERO and RoboTwin 2.0, and the real arm covers only three tabletop tasks; on Stack, ResWAM reaches 53.3% success, well below 90.0% on Open and 96.7% on Put.
