Skip to main content
Search

# Visual Token Compression

5 related articles

Subscribe to updates
  1. arXiv

    Visual tokens need not be ranked alike: BtsPruner keeps 32 of 576 and retains 91.6% of the dense model's performance

    Large vision-language models process hundreds to thousands of visual tokens per image, and inference cost rises with them. Earlier training-free pruning methods ranked all visual tokens by one importance criterion, overlooking that tokens take on different roles as encoding proceeds. BtsPruner locates the role transition from changes in attention concentration, then picks globally referenced representative tokens from later layers and recursively retrieves local detail tokens from earlier layers, preserving both roles under a fixed budget; on LLaVA-1.5-7B it keeps only 32 of 576 visual tokens (5.6%) and retains 91.6% of the dense model's performance.

  2. arXiv

    Video LLMs stop comparing every token: codec motion and residuals cut matching by two-thirds

    Compressing visual tokens for video large language models costs feature comparisons in the compression decisions themselves. This method aligns reference-frame dependencies, motion vectors and residuals already present in compressed video to the visual token grid, uses them to propose small candidate sets, then verifies with local visual similarity. Across three video LLMs and four video-understanding benchmarks, performance matches the training-free baseline FlashVID and sometimes exceeds it, while comparisons for token protection and temporal matching drop substantially.

  3. arXiv

    V-CoLA compresses vision tokens using linear-attention intrinsic signals, retaining 88% performance at 12.5% tokens with up to 6.15x prefill speedup

    The authors propose V-CoLA, a training-free vision token compression framework for hybrid vision-language models with linear attention, deriving a uniqueness-aware importance criterion from the retention preferences of the bounded recurrent state, combined with adaptive chunk-wise merging and early exiting of deep-layer vision tokens; on Qwen3.5-9B and related models it reaches 99.5% of original performance at 50% vision tokens and over 88.0% at 12.5%, with 1.86 to 6.15 prefill speedup.

  4. arXiv

    Foresight lets a frozen Qwen3-VL-8B plan its own future perception in streaming video, reaching 23.0 joint F1 on OmniPro Online and beating the strongest trained baseline by 9.5%

    The authors introduce Foresight, a training-free dual-stream architecture with two Siamese LLMs sharing weights, input encoders, and a KV cache, where one copy continuously ingests the stream while the other runs ahead over the same causal state to write a plan deciding when to reason next, what to check then, and how densely to sample; with a frozen Qwen3-VL-8B backbone it reaches 23.0 mean joint F1 on OmniPro Online (versus 13.5 for the strongest trained baseline, MiniCPM-o 4.5), improves the backbone by 6.7 points on StreamingBench with a best overall score, and by 15.4 points on OVO-Bench, with the largest gain of 18.7 points on Forward Active Responding where evidence arrives later in the stream.

  5. arXiv

    Braco reframes visual-token compression from picking tokens to re-parameterizing them, reaching 95.2% normalized accuracy at 23–64x compression while cutting prefill FLOPs by 84.2%–86.7%

    The work recasts extreme visual-token compression as a token-parameterization problem, separating basis transformation and structured truncation (which fix the retained subspace and compressibility) from coordinate organization (which affects optimization and cross-modal alignment, i.e. learnability), and designs Braco, a lightweight four-step coder combining transform-basis truncation, input-independent basis-coordinate embeddings, budget-dependent orthogonal re-parameterization, and learned spatial residual tokens from lightweight pooling; experiments show Braco forms the favorable empirical accuracy–efficiency frontier under 23x–64x compression and remains competitive at 144x, reaching 95.2% accuracy while reducing prefill FLOPs by 84.2%–86.

You’re all caught up