Skip to main content
Back to timeline
arXivSource publication:

Robot action experts stop learning only from demonstrations: HINT-WAM uses the demo's future frames as a teacher, lifting LIBERO-90 success from 78.88% to 85.18%

Synopsis

World action models couple action generation with future visual prediction, yet offline supervised fine-tuning trains the action expert only on noised demonstration actions and never on the intermediate states its own sampler produces. HINT-WAM lets the same model act as a teacher that additionally reads the demonstration's future video, distilling at the distribution level with an energy score on the student's own denoising trajectory while updating only the action expert. On the unseen LIBERO-90 tasks, success rises from 78.88% under supervised fine-tuning to 85.18%, above the 81.24% of full fine-tuning that also updates the video expert.

Source-provided article image: HINT-WAM: Hindsight-Informed Training for World Action Models
arXiv

Interpretation

Post-training for world action models gains a route that needs neither environment rewards nor simulator interaction: the action expert can be densely supervised on the states its own sampler actually visits. Offline supervised fine-tuning trains the action expert only on noised versions of demonstration actions and never on the intermediate states of its own sampler; on-policy reinforcement learning trains on the policy's own behavior but supplies only scalar rewards, making exploration and credit assignment costly under sparse success rewards. Across the 90 unseen LIBERO-90 tasks, HINT-WAM reaches 85.18% success, versus 78.88% for supervised fine-tuning and 81.24% for full fine-tuning; continuing behavior cloning for the same number of steps instead drops to 71.56%.

The teacher is the student's own weights plus a lightweight hindsight branch, and the student sees only inputs available at deployment. Earlier on-policy distillation for world action models requires an additional, stronger task-specific teacher, which makes post-training computationally expensive; HINT-WAM needs no separately trained teacher. The hindsight encoder and gated adapters total about 13M parameters and are used only by the teacher; the video expert stays frozen in every stage, and only the student is kept at deployment, so the policy never needs the future.

The distillation objective matches action distributions rather than regressing the teacher's velocity field, which keeps the student from collapsing onto the teacher's conditional mean. Directly regressing the teacher's velocity drives the student toward the teacher's conditional mean given what the student observes, and that mean may be an action correct under none of the modes. Under the same teacher, on-policy states, anchor and training budget, the energy score reaches 85.18% while velocity matching reaches 78.88%, essentially level with supervised fine-tuning.

Perspective

The demonstrated scope is the 90 unseen LIBERO-90 tasks and 25 hard RoboTwin 2.0 tasks, all in simulation, with 50 trials per task and shared initial states and sampler across methods. Here on-policy refers to the denoising states produced by the action sampler; observations still come from demonstrations, and no closed-loop interaction data is collected.

The on-policy states come from the sampler rather than closed-loop interaction, so compounding errors across control steps are not addressed; on RoboTwin 2.0 the released checkpoint was already trained on these demonstrations, leaving little headroom, and the difference from that checkpoint is not significant. Without replay, every method still forgets heavily on the four original suites, and HINT-WAM merely forgets least.