An agentic teacher patches VLA failures: simulation success rises to 87.8%, 90.6% and 30.0%
Synopsis
Pretrained vision-language-action (VLA) models fail when environmental noise or accumulated control error push them into states outside their training distribution, and DAgger-style correction needs repeated human monitoring and demonstration. Agentic-Teacher has coding agents analyze failed rollouts, localize causal frames, cluster failures by mode, and write closed-loop code policies that finish the remaining task from the failure state, then turns those successes into learnable demonstrations. In simulation, average success rises from 79.4% to 87.8% on LIBERO-10, from 80.4% to 90.6% on MetaWorld-10, and from 0% to 30.0% across three RoboCasa tasks.
Interpretation
Corrective data no longer depends on a human taking over each failure; coding agents generate it from the target VLA's own failed trajectories. DAgger and its HG-DAgger and ThriftyDAgger variants query an expert at learner-visited states, still requiring repeated human monitoring and intervention, which is costly to scale across tasks and environments. On LIBERO-10, MetaWorld-10 and three RoboCasa tasks, 50 base-policy rollouts per task are collected and the failed ones selected for repair; repaired data are merged with the initial successful rollouts for post-training, and the updated VLA is evaluated over 50 rollouts per task without agent or code-policy assistance.
The agent first localizes the causal frame behind a failure, groups similar failures into failure modes, and writes an executable, iteratively revised code policy for each group. The code policy takes over at the selected failure starting state and carries out the task stages that follow the diagnosed failure, rather than replaying the whole task. Programs are first tested on a small set of representative failures, and the harness returns task outcome, videos and action traces; failures trigger revision, successes expand to more starting states in that failure group, and successful programs and reusable skills enter an experience library for later repairs.
Successful code executions still pass through noise injection, idle-frame removal and a flow-matching loss filter before becoming VLA supervision. Program-generated trajectories can have narrow behavioral coverage, temporal redundancy or poor compatibility with the base policy, so task success alone does not make them good training data. In the LIBERO-10 trajectory-processing ablation, training on raw trajectories yields 0.0% success, noise injection raises it to 81.0%, idle-frame removal to 89.0%, and the flow-matching loss filter to 95.0%; the final step cuts trajectories from 120 to 110 while success still rises.
Perspective
All results come from simulation: LIBERO-10, MetaWorld-10 and three RoboCasa tasks, with 50 evaluations per task and seeds separate from repair-data collection. The intended readers are researchers and engineers working on robot policy learning and data engines; repair on physical robots requires resetting the scene and replaying the failed trajectory up to the causal frame, a cost that remains open.
Repair capability is limited by the coding agent's available skill-library coverage, and tasks with complex end-effector control or fine manipulation may need prolonged exploration; even a successful code-policy trajectory can contain behavior patterns that degrade VLA performance once learned. Joint multi-task training shares one hyperparameter configuration that may not be optimal for every task, and individual tasks drop: book-caddy and mug-microwave on LIBERO-10, and drawer-open-v3 on MetaWorld-10.
