Skip to main content
Search

# δ-Vision

1 related articles

Subscribe to updates
  1. arXiv

    δ-Vision rebuilds layer-wise visual memory with lightweight MLP adapters, cutting FLOPs to 17.90% while keeping every visual token

    Using low-rank interventions, the work finds that visual-to-text information flow in multimodal LLMs is concentrated in a low-dimensional subspace and that layer-specific visual states are highly predictable by lightweight MLPs; it therefore proposes δ-Vision, which freezes the language-model backbone and uses low-rank adapters to construct each layer's visual memory while preserving all visual tokens for text retrieval, reaching an average score of 74.4 on Qwen3-VL-4B (92.9% of uncompressed performance), 9.1 points above the strongest 5%-retention pruning baseline DivPrune, and requiring only 17.90% of the uncompressed model's FLOPs on Video-MME.

You’re all caught up