Stacked video agents often lose to plain inference: SAVE matches native Gemini 3 Flash on Minerva at about 60% of the cost
Synopsis
In a controlled evaluation that fixes the backbone to Gemini 3 Flash or Gemma 4 and accounts for tokens and cost, most existing agentic video-reasoning frameworks fail to consistently beat native inference and often cost more. SAVE keeps the simplest interaction loop and only changes the query unit from a single frame to a time range, adapting scene detection and visual resolution to the range length. On Minerva with Gemini 3 Flash, SAVE reaches 66.4% accuracy, comparable to 64.8% for native inference, while cutting inference cost by about 60%.
Interpretation
Once the multimodal backbone is held fixed, much of the apparent gain from video-reasoning agents disappears: with Gemini 3 Flash, none of the evaluated prior agents beats native inference on Minerva, EgoSchema or VSIBench, and several cost substantially more. Earlier agentic methods were evaluated with different backbones, preprocessing pipelines and compute budgets, so it was hard to tell whether performance came from the framework or from a stronger underlying model. With Gemini 3 Flash, native inference scores 64.8% on Minerva, 75.4% on EgoSchema and 59.2% on VSIBench, while VideoTree, VideoAgent, VideoLucy, VideoCuriousAgent, DeepVideoDiscovery and A.I.R. all fall below the corresponding native figure on the three benchmarks.
SAVE widens the agent's query action from a single frame to a contiguous time range, so the backbone specifies start and end frame indices and receives evidence from a whole interval at once. The minimal baseline queried one frame at a time, and that lack of temporal context was identified as its primary bottleneck. On Minerva, the minimal Active Exploration Baseline reaches 52.8%, sampling whole scenes raises it to 62.7%, adding high-resolution frames for fine-grained queries gives 63.3%, and the final adaptive-threshold version reaches 66.4%.
For each requested range, the system runs scene detection and returns the middle frame of each detected scene, with detection thresholds adjusted automatically to the range length. A fixed threshold can fail to isolate meaningful semantic boundaries in fine-grained motion sequences, leading the agent to query at the wrong granularity. If the initial threshold yields too few scenes, the threshold is lowered progressively until coverage suffices; if scenes are still too few, uniformly sampled frames fill the interval, and narrow ranges are returned at higher resolution to preserve detail.
SAVE outperforms all evaluated prior agentic methods in five of six backbone-benchmark settings while generally using substantially fewer resources. Prior agents did not consistently beat native inference under a shared backbone, so a simple design was needed that improves the accuracy-efficiency trade-off. On Minerva with Gemini 3 Flash, SAVE reaches 66.4%, 9.9 percentage points above the 56.5% of VideoLucy, the highest-accuracy evaluated prior agent, at 79% lower cost; with Gemma 4, SAVE reaches 62.4% at 19.7K tokens per question versus 72.0K for VideoLucy.
Perspective
The evaluation covers two backbones, Gemini 3 Flash and Gemma 4, and three benchmarks, Minerva, EgoSchema and VSIBench (tiny), with efficiency measured as token consumption and inference cost; the authors note that an ultra-long benchmark such as LVBench was not included and that end-to-end latency was not measured.
The authors note that standardizing heterogeneous methods required adapting backbone calls, prompts and tool interfaces, which may affect their behavior; the efficiency analysis covers only tokens and inference cost, not end-to-end latency, which is influenced by API rate limiting, network variability and heterogeneous hardware. MR Video was excluded because its preprocessing exceeded 3M tokens on the first Minerva video, and AVP because it relies on Gemini's native video interface.
