Skip to main content
Back to timeline
arXivSource publication:

Visual Board splits what is seen from what is inferred in multimodal agents, and an 8B system beats a standalone 32B on all three benchmarks

Synopsis

In multi-step visual question answering, an early misreading persists as a premise in the dialogue history, and later reasoning stays coherent around it. Visual Board writes image observations, unresolved details and derived conclusions into separate fields of a capacity-bounded workspace that is replaced by snapshot, while the original image stays available for reinspection. Without task-specific training, it improves 4B, 8B, Qwen3-VL-Flash and 32B configurations on Euclid30K, SolidGeo and ChartQA bar charts by 6.10 to 24.82 percentage points, and the homogeneous All-8B system outperforms standalone Qwen3-VL-32B on all three benchmarks.

Source-provided article image: Keep the Image in Mind: A Visual Board for Multimodal Agents
arXiv

Interpretation

The bottleneck in multi-step visual reasoning is placed in how visual evidence is represented, not only in how much of it is gathered: recording observations, uncertain interpretations and derived conclusions separately makes conflicts between what is seen and what is inferred explicit. Earlier multi-step frameworks appended each observation and inference to a growing dialogue history, so an early misreading became a premise of every later step and was rarely revisited even when new evidence arrived. On Euclid30K, SolidGeo and ChartQA bar charts, without task-specific training, the 4B, 8B, Qwen3-VL-Flash and 32B configurations all improve over their standalone baselines, by 6.10 to 24.82 percentage points.

The workspace has three fields, image facts, uncertain image facts and reasoning steps, and the solver updates it by snapshot replacement rather than appending, with the original image kept accessible as the authoritative visual evidence. New tool observations enter only as proposals rather than automatic edits to the committed workspace; a refuted observation can be withdrawn into uncertainty, and derivations that depend on it are updated. In a 500-question Euclid30K comparison with solver, tools, critic and interaction limits fixed, the structured workspace beats shared free-form notes by 13.00, 9.35 and 6.67 points on All-4B, All-8B and Qwen3.8-Flash, and beats no-Board retries by 18.80, 7.35 and 8.47 points while using 7.9% to 38.2% fewer tokens.

Perspective

The demonstrated scope is visual question answering on Euclid30K, SolidGeo and ChartQA bar charts, with backbones covering 4B, 8B, Qwen3-VL-Flash and 32B, and a text-only critic that never sees the image. The ablation comparisons concentrate on 500 Euclid30K questions, and the 32B diagnostic study covers 50 questions. Supervised fine-tuning uses only the 1,556 geometry trajectories from the Euclid30K training split, with no overlap with ChartQA-Bar.

A text-only critic can miss premises that are visually wrong but internally consistent. The critic's benefit shifts with backbone strength: All-4B and All-8B gain 12.00 and 7.95 points, while Qwen3.8-Flash rises from 68.47% to 69.80% without the critic, using 59.8% fewer tokens and 57.6% less time. The harness adds inference cost through multi-round solver, visual-tool and critic calls. Supervised fine-tuning currently covers only geometry trajectories, and extending it to broader visual domains remains future work.