Skip to main content
Back to timeline
arXivSource publication:

CanopyKV keeps history beyond the crop: a quarter of Dense's KV, 10.47 points more accuracy on MME-Lite

Synopsis

Active visual reasoning models answer hard visual questions by repeatedly cropping images and calling tools, but each new crop and reasoning step enlarges the key-value cache, and compression can discard earlier evidence that later decisions still need. CanopyKV uses the parent-child crop relationships recorded by tool calls as retention signals, favoring parent-image context that the incoming crop does not cover and keeping selected history fixed across later branch visits. On MME-Lite it retains 24.63% of Dense's total KV, raises accuracy by 10.47 percentage points, and cuts complete-task time from 53.12 to 32.35 seconds.

Source-provided article image: CanopyKV: Remember beyond the Crop for Fast Active Visual Reasoning
arXiv

Interpretation

KV compression for active visual reasoning now chooses what to keep from the parent-child relationships between image crops, rather than from attention magnitude or recency alone. Earlier KV compression methods estimated retention value from attention statistics, recency, text priority, cross-modal relevance, or semantic diversity, and those signals did not separate evidence locally replaced by a new crop from complementary information still needed for later exploration. Across five evaluation groups from VisualProbe, V*, HR-Bench 4K/8K, and MME-Lite, CanopyKV's accuracy exceeds Dense and every other compression method in the table; on MME-Lite it retains 24.63% of total KV at 53.52% accuracy, 10.47 points above Dense and 3.70 points above the strongest baseline, MixKV.

Which history to keep comes directly from tool metadata, without extra attention scoring or forward passes. Source quotas divide the retention budget among the root image, each crop source, and text, then the crop parent chain shifts each parent's quota toward regions the new crop does not cover, while protected instruction, question, and recent control-state rows are always kept. In a replacement study of 320 samples across 290 source-image clusters, deleting parent evidence inside the incoming crop's footprint lowered fact-readout accuracy by only 1.56 percentage points, while deleting parent evidence outside it lowered accuracy by 24.84 points, identifying context beyond the crop as the retention priority.

Selected history stays fixed across later branch visits, so complementary evidence remains readable after the search focus moves. Resampling uniformly at each handoff reduced retained parent rows from 296 to 6 after eight visits to other branches and dropped readout accuracy to 22.46%, whereas fixed history held 90.82%. On fixed reasoning-stream replays, CanopyKV's cache work totals 3.72 seconds versus 11.91 for VL-Cache and 22.83 for MixKV; on identical streams it completes in 66.5 seconds versus Dense's 92.3.

Perspective

Evaluation uses Mini-o3-7B-v1 (built on Qwen2.5-VL) on the full question sets of VisualProbe, V*, HR-Bench 4K/8K, and MME-Lite, with one trajectory per question, BF16 greedy decoding, up to 32 rounds, and timing on H20 GPUs at batch size one. The method relies on parent-image and coordinate metadata recorded by cropping tools, fitting crop-based active visual reasoning pipelines.

The budget sweep shows a lower limit to the benefit: on a 358-question VisualProbe subset, retaining 12.65% of KV drops accuracy to 35.75%, raises search to 11.37 rounds, and raises time to 40.15 seconds. The authors also note that a smaller cache does not necessarily yield higher accuracy or shorter completion time, so retained content and execution efficiency must be considered together.