A small model reads first: SkimPrefill cuts prefill time 3.7 to 11.6 times on long image-text inputs
On long interleaved image-text inputs of about 100K tokens, vision-language models spend most of the wait for the first output token in prefill. SkimPrefill has Qwen3.5-0.8B read the whole input first at half image resolution, stopping at layer 7, then scores every image and paragraph with one attention head of that layer and hands the top whole units to the target as an ordinary shorter prompt. Keeping 10% of the input tokens on MMLongBench at 128K, it speeds up the prefill of Qwen3.5-9B, Gemma-4-12B and Qwen3-VL-32B by 3.7, 11.6 and 8.2 while holding their average accuracy.