Bump the Camera and a Frozen Robot Policy Falters: Homeward Warps the View Back, Lifting Four VLAs on All 1,599 Camera Variants
Synopsis
When a third-person camera is bumped, remounted or given a new lens at deployment, frozen vision-language-action policies lose success sharply because they assume the camera still sits where demonstrations were recorded. Homeward leaves policy weights untouched and uses one training-view reference image plus the robot's kinematic model to reproject the live view back into the training view: unknown camera parameters and static scene depth are estimated by a monocular 3D foundation model, while the known robot arm is rendered directly from joint states. Across all 1,599 LIBERO-Plus camera variants, success rises from 62.5%, 76.5%, 68.0% and 3.6% to 77.6%, 95.1%, 91.2% and 26.4% for OpenVLA-OFT, π0, π0.5 and OpenVLA.
Interpretation
A frozen vision-language-action policy can recover much of its performance after the camera moves, provided the input image is restored to the training viewpoint rather than the policy being asked to adapt to the new one. These policies assume the deployment camera matches the one used to collect demonstrations; LIBERO-Plus identifies camera viewpoint shift as the most damaging perturbation for frozen policies, dropping OpenVLA-OFT to 62.5% overall across all 1,599 camera variants and to 22.0% under the largest shifts. On LIBERO-Goal, reprojecting perturbed views into the training view with simulator ground-truth depth and camera poses raises OpenVLA-OFT from 79.7% to 90.4%, and from 35.5% to 82.9% under the severest shifts, indicating the failure lies mainly in visual input space.
Calling an off-the-shelf 3D foundation model to do the restoration can be worse than nothing, because such models tear apart the thin, specular, moving robot arm. Replacing privileged geometry with a frozen VGGT leaves OpenVLA-OFT at 72.8% closed-loop success on LIBERO-Goal, below the 79.7% of the uncorrected policy. Frame inspection shows VGGT reprojects the static tabletop and objects plausibly while the arm is scattered across depth planes; the arm's 3D geometry is already known exactly from joint states and the kinematic model, so Homeward splits the image into an estimated scene layer and a kinematically rendered robot layer.
Homeward restores the view with a frozen policy, a frozen geometry backbone, an uncalibrated deployment camera and zero adaptation data, needing only the robot mesh model, one reference image and a one-time training-camera calibration. Earlier test-time view canonicalization either required calibrated cameras and depth sensors or fine-tuned a generative view-synthesis network on the target scenes. Evaluated over roughly 300,000 closed-loop episodes with four frozen models and five camera perturbation families; on all 1,599 LIBERO-Plus variants the four models gain 15.1, 18.6, 23.2 and 22.8 points, with 14 of 16 suite-policy combinations improving significantly and none degrading significantly.
Perspective
All evaluation is in the LIBERO simulator, covering four frozen models, five camera perturbation families and the 1,599 LIBERO-Plus camera variants; the method needs a robot mesh model, one training-view reference image and a one-time training-camera calibration, and requires no deployment-camera calibration or scene fine-tuning. The authors list extending Homeward from simulation to physical robots as the next step, which requires robustness to URDF-to-real photometric gaps, foreground occlusions in front of the manipulator, and contact-rich gripper-object interactions.
On AnyCamVLA's wide-orbit benchmark on LIBERO-Long, Homeward still trails AnyCamVLA with ground-truth cameras, and the gap widens at larger orbits, with diagnostics pointing to monocular translation error growing from 2 cm to 7 cm. On the LIBERO-Plus camera split, restoration lowers success by 4.7 points for OpenVLA-OFT+, which was already fine-tuned on perturbed data. OpenVLA does not improve under camera drift, and the Object suite is Homeward's weakest.
