Long-video QA stops discarding frames first: NullTok calibrates evidence against same-position control queries and leads Video-MME on two backbones at a 64-frame budget
Synopsis
Long-video question answering is limited by the number of visual tokens, and a frame-first pipeline that drops a frame holding a small object or a brief action cannot recover its tokens later. NullTok instead assesses every sampled token first: it builds an empirical null from 64 group-matched irrelevant control queries at the same video position, converts target-query affinity into a right-tail Monte Carlo p-value, then applies a high-recall semantic guard, visual deduplication and joint frame-token budget allocation. With frozen LLaVA-Video-7B and Qwen3-VL-8B at a cap of 64 full-frame equivalents and no subtitles or audio, it reaches 65.81% and 68.85% on Video-MME, above every evaluated baseline under the same budget for both backbones.
Interpretation
Long-video input compression no longer keeps or drops whole frames; it first judges which tokens at all sampled spatiotemporal positions are genuinely specific to the current question, then decides what to keep. Earlier pipelines retrieved question-relevant frames or clips and then compressed their visual tokens, so fine-grained selection depended on the coverage of the initial retrieval, and a frame holding a small object, a brief action or a subtle attribute could be excluded before its tokens ever entered compression. At a cap of 64 full-frame equivalents without subtitles or audio, Video-MME accuracy is 65.81% with frozen LLaVA-Video-7B and 68.85% with Qwen3-VL-8B, above all evaluated baselines under the same budget for both backbones; against QCA+MMTok, which retrieves 256 frames and compresses them to the same budget, LLaVA-Video-7B gives 65.81% versus 65.07%.
Whether a token is worth keeping is decided by how exceptional it is at its own position relative to irrelevant queries, not by the absolute size of a similarity score. Raw query-token affinities have no consistent scale across videos and positions, since broad phrasing or visual texture can inflate them, so a reference is needed for how the same coordinate responds to structurally comparable but irrelevant queries. Each token is scored at the same coordinate by 64 group-matched control queries drawn from other videos, forming a local empirical null that turns the target score into a right-tail Monte Carlo p-value; under conditional exchangeability that p-value is tokenwise marginally super-uniform, a per-token statement that does not cover family-wise error over the final mask.
Calibrated candidates then pass through semantic protection, visual deduplication and one joint allocation so that temporal coverage and spatial detail share a single fixed budget. Candidates can remain visually redundant and temporally uneven, so scoring alone does not turn them into a compact input that fits the budget. A high-recall semantic guard retains high-scoring tokens, deterministic deduplication on frozen embeddings removes redundancy at a similarity threshold, and an allocator distributes the budget by frame relevance and candidate density; on MLVU with Qwen3-VL-8B at 16 full-frame equivalents it reaches 70.26%, 1.43% above MMTok and 1.38% above both OTPrune and ReDiPrune.
Perspective
The demonstrated scope is two frozen backbones, LLaVA-Video-7B and Qwen3-VL-8B, on Video-MME, LongVideoBench and MLVU at a main budget of 64 full-frame equivalents without subtitles or audio, plus a 16-full-frame-equivalent comparison on MLVU with Qwen3-VL-8B; it suits researchers and engineers working on input compression for long-video multimodal reasoning.
The statistical guarantee is tokenwise and marginal under conditional exchangeability, and family-wise or false-discovery-rate control over the final mask would need additional analysis and is not claimed; WFS remains stronger on LongVideoBench for both backbones and QCA is also stronger on Qwen, while on MLVU with LLaVA-Video-7B NullTok reaches 67.73%, below WFS at 71.73%.
