Skip to main content
Search

Anhui University

2 related articles

Subscribe to updates
  1. arXiv

    Pruned visual tokens keep the answer but shift the odds: DiFiT trains selection on the full model's response distribution

    Visual token pruning for multimodal large language models is usually judged by whether answers stay correct, yet pruning can change the probabilities of alternative responses even when greedy answers do not move. DiFiT learns which visual tokens to keep from both the full-token model's response distributions and reference answers, with one ranking serving several retention ratios. On Qwen2.5-VL-7B it cuts sampling-induced negative flips by 30.3% relative to FastV at 10% retention and preserves 98.74% of baseline performance on MME at 30% retention.

  2. arXiv

    Recurrent memory splits each write into two calls: whether to store a token, then how many slots to touch, lifting sparse-fact accuracy from 0.048 to 0.222

    Recurrent-memory models compress history into a fixed set of slots, and earlier multi-slot memories wrote every token to the same number of slots, merging whether to store a token with how many slots to update. Factorized Write Routing separates the two: a learned gate first decides whether the current token is written, then a slot router chooses a fixed or token-adaptive number of slots, while queries still read all slots. Across ten seeds on two synthetic long-context tasks with four slots per write, the result reverses with information density: always writing reaches 0.472 long-context accuracy on dense Feature Recall versus 0.337 for the learned gate, while the learned gate reaches 0.222 on sparse-fact Passkey versus 0.048 for always writing.

You’re all caught up