Skip to main content
Back to timeline
arXivSource publication:

A new test for video world models exploring dynamic scenes: WanderBench runs eight models and finds camera control and source dynamics drop together on long clips

Synopsis

Video world models can take a real recording as condition and re-observe its dynamic scene along new camera paths, but there was no evaluation of whether the generated rollout stays faithful to that recording. WanderBench pairs 745 single-shot source videos with keyboard-specified camera routes and scores six dimensions over 24 sub-metrics; across eight models, longer clips keep the static scene and visual quality while camera control and source dynamics are lost, and no model keeps revisited places consistent. The training-free WanderLand anchors each frame to the same-instant source and fills revisit gaps with the model's own earlier static content, improving most dimensions on five models, by up to 8.3 points in source-dynamics fidelity.

Source-provided article image: WanderBench: Can World Models Explore Dynamic Worlds from Video?
arXiv

Interpretation

The object of video world model evaluation moves from a single image or text prompt to a real recording: the model must re-observe the scene and events in that recording along a new camera path, not merely generate plausible frames. Earlier benchmarks either start from a single image or text prompt, or mainly measure perceptual quality and camera following, without directly testing whether exploration preserves subject identity, event progression and scene consistency on revisiting a place. WanderBench pairs 745 single-shot source videos with camera routes given as keyboard actions, spans three difficulty tiers from 3 to 25 seconds, scores six dimensions with 24 human-validated sub-metrics, and compares output against the source frame of the same instant.

Eight video-conditioned world models share a failure mode on long clips: the static scene and visual quality remain, while camera control and source dynamics fall away. On the simple tier the frame-limited UniWorld-View obtains the highest overall score and the three full-length models span only 4.3 points; on the hard tier that span widens to 13.5 points and WorldForge drops to 10.4 in camera control. Source-world fidelity is the highest-scoring dimension on every tier, while source-dynamics fidelity falls to 37.2 on the hard tier; dynamic revisit consistency is the weakest dimension, with no model above 68.6 and full-length models between 50.0 and 54.5 even on the simple tier.

WanderLand leaves the parameters of pretrained video world models untouched and rebuilds only the camera and source render the model already reads, aligning every frame to the source at the same instant. Earlier conditioning used the first-frame camera or a single-frame reference, so generated events drifted out of sync with the recording on long clips and revisited places were redrawn. The camera comes from the model's own geometry front-end and the source frame is warped into it by the model's own point-cloud renderer; on revisits, Depth Anything 3 depth lifts the model's earlier outputs to 3D points to fill regions the source cannot cover, and pixels whose source content changes over time are discarded by camera-compensated color difference.

On WanderBench, WanderLand improves most dimensions on five backbone models and degrades none. Dynamic revisit consistency rises by 2.4 to 4.8 points across the five backbones, as holes at a revisit are filled with content the model has already shown instead of being redrawn. The largest single gain is on the full-length InSpatio-World 1.5, whose source-dynamics fidelity rises by 8.3 points; CameraAnything shows a 0.0 change in source-dynamics fidelity. World-in-World and WorldForge are not included because both are themselves training-free frameworks built on pretrained world models and do not expose the inputs WanderLand rebuilds.

Perspective

Evaluation and improvement are confined to conditioning on single-shot source videos whose camera is static or near-static, with camera routes given as keyboard actions and clips of 3 to 25 seconds; frame-limited models are scored only on the opening segment up to their maximum length (81 frames, 49 for TrajectoryCrafter), and the medium and hard tiers do not apply to them.

As clips grow longer, models keep the static scene and visual quality but lose camera control and source dynamics, a failure mode observed across the eight models tested; dynamic revisit consistency correlates with camera control, which the material reads as a model needing to reach the revisit pose before consistency there can be achieved. WanderLand reproduces the original output exactly where the evidence does not change, and its extra cost is one additional generation when the camera revisits a place.