Skip to main content
Back to timeline
arXivSource publication:

Long-horizon robot tasks stop riding on the VLA alone: RePlan-VLA adds progress monitoring and replanning for 62.5% average success on RoboCasa365

Synopsis

Long-horizon robot manipulation often fails as small errors accumulate, so RePlan-VLA moves long-horizon reasoning, progress tracking and error recovery out of the vision-language-action (VLA) policy into a VLA-agnostic outer framework, leaving the underlying policy frozen and requiring no extra fine-tuning. A planner vision-language model decomposes the long instruction into short subtasks, and a fine-tuned progress estimator judges completion and failure at the level of manipulation primitives and triggers replanning. On RoboCasa365 composite tasks it reaches 62.5% average success versus 40.6% for the strongest baseline Awomo-0.5I; on CALVIN ABCDharD it averages 4.10 consecutively completed tasks versus 3.58 for the strongest baseline XR-0.

Source-provided article image: RePlan-VLA: Progress-Aware Replanning for Long-Horizon Vision-Language-Action Models
arXiv

Interpretation

Long-horizon manipulation becomes a combination of a short-horizon policy and an outer execution monitor, so the underlying VLA only executes one short subtask instead of also carrying long-horizon reasoning, memory, progress tracking and error recovery. Earlier generalist VLAs and long-horizon methods treated the long task as a monolithic execution problem, without an explicit record of completed and failed subtasks, so missed prerequisites and assumed completions propagated into later manipulations. On 16 tasks each from the RoboCasa365 Composite-Seen and Composite-Unseen splits with five episodes per task, RePlan-VLA reaches 62.5% average success versus 40.6% for Awomo-0.5I; on unseen compositions RePlan-VLA reaches 56.3%, 21.3 points above the strongest unseen baseline.

During execution the system judges progress at the granularity of manipulation primitives, and when progress stalls or deviates it produces a natural-language diagnosis and rewrites the remaining plan while keeping the history of completed subtasks. The progress signal comes from continuous labels derived from spatial relationships among the gripper, target object and destination, rather than from elapsed time in demonstrations or a monotonically increasing task-level reward. The progress estimator uses Qwen3-VL-8B as its base, adapted with LoRA plus a regression head and trained on short-horizon trajectories from RoboCasa365, CALVIN and real-world demonstrations; in ablation the fine-tuned primitive-level estimator reaches 62.5% average success, versus 37.6% for zero-shot VLM monitoring and 18.8% for global progress estimation.

Subtask instructions from the planner must first pass an LLM-based instruction verifier confirming they match the language forms the underlying VLA saw in training. Semantically valid but unfamiliar wording makes the frozen VLA execute unreliably, and the verifier aligns high-level planning with the language interface the policy can actually execute. Removing the verifier drops RoboCasa365 average success from 62.5% to 34.5%.

Perspective

The demonstrated setting is 16 tasks each from the RoboCasa365 Composite-Seen and Composite-Unseen splits with five episodes per task, the subset of five-task CALVIN ABCDharD sequences selected for low baseline performance, and multi-task sequences executed on a Franka Research 3 arm under a CALVIN-style protocol; the intended readers are researchers and engineers working on long-horizon robot manipulation and VLA deployment.

The authors note that effectiveness depends on reliable visual grounding and on the underlying policy's short-horizon manipulation capability, and that recovery may be limited by irreversible execution failures; skill-aware replanning that accounts for policy capabilities, learning from successful recoveries, and adding tactile and other modalities are left to future work. On CALVIN, XR-0 is slightly higher than RePlan-VLA on complete five-task sequences (60.0% versus 58.0%). The CALVIN subset size is inconsistent across the source: the evaluation setup states 100 sequences, while Section 4.2.2 states 50 CALVIN ABCDD sequences, and this report adopts neither figure.