Skip to main content
Back to timeline
arXivSource publication:

GRID gives UAV vision-language-action policies real 3D geometry, topping every TravelUAV split with 48.2% overall success

Synopsis

Long-range UAV vision-language-action policies previously took depth maps as extra input features, and on TravelUAV that left success essentially unchanged (42.5% versus 42.6%). GRID back-projects monocular depth along each viewing ray into body-frame 3D positions and, during training only, aligns visual features with a frozen multi-view geometry model, adding one depth pass at inference. Across 3,005 test episodes, overall success rises from AeroVLA's 41.7% to 48.2%, best on all three splits, with a 25.4% relative gain on the unseen map.

Source-provided article image: GRID-UAV: Geometry and Ray-depth Integrated Decision-making for Unmanned Aerial Vehicles
arXiv

Interpretation

On long aerial routes, feeding depth to the policy as extra features barely changes success, whereas lifting the same depth into 3D positions does improve it. AutoFly previously reported a 3.9-point success gain from depth features on routes averaging about 107 m, while TravelUAV routes average 256 m and reach 1.1 km. On TravelUAV, aerial-adapted depth passed through an AutoFly-style shared projector reaches 42.5% success against 42.6% for the no-depth baseline, with no significant difference on any split except the Hard routes of the unseen map.

Explicit 3D lifting and implicit geometric alignment fail at different stages of a flight, and GRID combines them to reduce each one's main failure. The implicit-only model more often never reaches the target, while the explicit-only model more often reaches it but fails to finish. On the unseen map, the explicit model reaches the target in 69.4% of episodes versus 61.5% for the implicit model, but completes it in 57.7% versus 62.3%; GRID reaches 8.0 points more often than the implicit model and completes 3.7 points more often than the explicit model.

GRID achieves the highest success rate on every TravelUAV test split while using only forward- and downward-facing RGB cameras at test time. The comparison is against UAV vision-language-action models such as AeroVLA that rely on 2D image encoders without 3D grounding. Over 3,005 test episodes overall success rises from 41.7% to 48.2%, with relative gains of 9.5% on Seen, 17.1% on Unseen Object and 25.4% on Unseen Map, and Hard unseen-map routes rise from 26.7% to 39.8%.

Perspective

All experiments run on the TravelUAV simulation benchmark, with only forward- and downward-facing RGB cameras at test time and commands for forward, vertical and yaw motion or landing. Evaluation covers only TravelUAV, because short-route benchmarks would not test the long-route failures studied here.

All results come from simulation, and adaptation to real sensor noise, wind and lighting changes remains untested. Each policy is trained once, so repeated evaluations and paired tests capture evaluation noise but not training variance. Stopping in the wrong place still affects 16% to 17% of episodes for all three models, and GRID gets lost slightly more often than the explicit model (3.8% versus 2.6%).