Frozen VLMs answer wrong but point right: SEE lifts every small-object benchmark across five backbones without training or a detector
Synopsis
Frozen vision-language models often point at the detail a question needs yet miss it when answering directly. SEE turns that into a training-free second look: when the initial answer has low confidence, the same model proposes candidate regions, answers again on enlarged crops, also answers with its own transcript of the full image, and aggregates every view with the original prediction by confidence-weighted voting. On the three small-object benchmarks V*Bench, HR-Bench 4K and 8K, SEE improves every model–benchmark pair across five frozen backbones by 2.3 to 13.1 points, raising the small-object mean from 72.4% to 78.2% and beating five training-free baselines.
Interpretation
A frozen vision-language model that answers a question incorrectly can often still box the region holding the answer, and SEE exploits that mismatch by making the model look again where it points when confidence is low. Earlier training-free approaches depend on attention maps, an external detector or iterative patch-tree search, or let a single second answering pass decide, so one bad crop can overturn an initially correct answer. On V*Bench questions that Qwen3.5-4B answers incorrectly, the medoid of five localization requests gives a box that overlaps the annotated target, which the authors call the elicitation gap; replacing SEE's self-proposed regions with random crops of the same size makes SEE worse than the direct answer, and a fixed grid recovers only part of SEE's gain.
SEE changes no weights and adds two kinds of view: enlarged crops of the regions the model itself names, and the model's own transcript of the full image placed before the question, after which all views vote with the original answer by confidence. Confidence is the geometric mean of the answer-token probabilities, and the extra inspection runs only when it falls below a threshold, so confident questions return without further calls. At the 4B backbone, adding zoom alone lifts V*Bench by 11.6 points but lowers InfographicVQA, while adding the transcript alone does the reverse; combining both with the vote raises the four-benchmark mean from 77.9 to 84.1, and adding the confidence gate gives 84.4 at about 5.9 times the direct-answer time.
Perspective
The demonstrated scope is high-resolution visual question answering and document understanding on five frozen backbones: V*Bench, HR-Bench 4K and 8K, and InfographicVQA, all as training-free inference-time enhancement with no weight updates. The authors position SEE for settings where details are small, the model is frozen and its token probabilities are available; combining attention-based zooming with SEE without giving up black-box access is named as the next step.
The authors report that gains concentrate on single objects, exceeding relation-question gains in some HR-Bench cells, because a zoom can crop away the second object being compared; the gate and vote are only as good as the confidence, thinking-mode GLM-4.1V is confident on almost every question so SEE rarely acts, and a confidently wrong view can overturn a right answer. The vote rule and threshold were compared on the full evaluation sets, and the authors note that a test set untouched by every choice would settle how much this matters; multi-image input, video and long answers are untested.
