Frozen visual world models can now police the path, not just the goal: RTS searches only on alarm and lifts safe arrival by up to 41.67 points
Synopsis
Pre-trained visual world models can turn current and goal observations into multi-step action plans, but a terminal-proximity objective still counts a trajectory as successful when it crosses a forbidden zone or slides along a collision. Risk-Triggered Search (RTS) leaves the frozen JEPA world model untouched, unrolls each candidate action sequence into future latent prefixes, trains a lightweight risk head on process-violation labels, and uses that same prefix-risk criterion as both the search trigger and the hard feasibility gate. Across five tasks, RTS-AP raises Safe Arrival Rate over Direct by 12.67, 41.67 and 27.34 percentage points on TwoRoom, Reacher and PointMaze, while 1,968 paired trajectories show identical actions to every-step search with 17.71%–70.41% less controller time.
Interpretation
Safe control shifts from counting goal arrival as success to auditing whether the path crosses a forbidden zone or slides along a collision, and RTS intervenes only when the frozen visual world model raises an alarm, so safe arrival rises. The earlier INTACT Direct optimized only terminal proximity, reaching 81.50% success but 8.50% safe arrival on Reacher; Guarded A searched on terminal distance and reached 81.17% success with only 9.50% safe arrival. On TwoRoom, Reacher and SimplePointMaze, RTS-AP reaches 88.17%, 50.17% and 77.34% safe arrival, gains of 12.67, 41.67 and 27.34 percentage points over Direct, while violation rates fall from 21.50%, 84.50% and 50.00% to 2.33%, 37.83% and 8.59%.
Risk is not read from a single state but from each candidate action sequence unrolled through the frozen world model into a future latent prefix, which the risk head then scores for process violation. Candidate plans sharing one observation look identical in static features, so the risk head cannot separate them; concatenating candidate actions into the features lifts within-context AUROC only from 0.500 to 0.547 on TwoRoom and 0.527 on Reacher. With predicted prefixes, within-context AUROC reaches 0.791 on both TwoRoom and Reacher, paired gains of +0.243 and +0.263, and task-specific risk heads rank candidates at 0.901–0.985 AUROC across all five tasks.
Correction after an alarm abandons plain Euclidean minimal modification for a step-magnitude preservation term, so obstacle-directed actions are not shrunk into stalling and instead detour tangentially. Pure Euclidean minimal modification drives obstacle-directed actions to zero amplitude, leaving 2.133 non-fallback zero-action stalls per episode on PointMaze; the amplitude-preserving metric cuts that to 0.734. On PointMaze, swapping only the final selector under a fixed generator raises safe arrival by 3.65 percentage points (95% CI [0.78,6.51]), and three independently trained risk heads give gains of +3.65, +3.39 and +3.12 points.
The search trigger and the final feasibility gate share one criterion, and the correction metric has a unique zero at the nominal plan, so executing a safe plan directly yields exactly the closed-loop trajectory of every-step search. Every-step CEM search preserves the backbone but carries heavy latency, while earlier triggered schemes classify the current state or mix task return into final selection and thereby alter safe nominal plans. Across 1,968 same-kernel paired trajectories on TwoRoom, Reacher, SimplePointMaze, Cube and PushT, RTS and Always-RTS produce 100% identical step-by-step actions with 17.71%–70.41% less controller time.
Perspective
Evaluation covers five tasks, TwoRoom, Reacher, SimplePointMaze, Cube and PushT, with frozen INTACT weights and geometric forbidden zones: 997 Cube tasks and 1,538 PushT tasks on the full test sets, and 128 tasks on 64 unseen SimplePointMaze maps, plus an independent confirmation cohort of 32 ordinary and 32 boundary contact tasks. The method targets visual control interfaces that expose only RGB observations and action histories; the authors propose risk-aware visual adapters, goal-masked dynamic audit horizons and adaptive search budgets as next directions.
Performance remains bounded by visual boundary resolution under downsampling, with RTS-AP at 56.50% safe arrival versus 62.00% for Direct on low-resolution TwoRoom; static threshold calibration in high-frequency PushT contact trades protection of safe nominal plans against recall of near-term violations; and local CEM proposals have limited reach in deep cul-de-sacs. Moderating the PointMaze search cap from 100% to 50% raises safe arrival from 75.78% to 83.85% while violation rate rises from 9.11% to 15.89%.
