Skip to main content
Search

Yufei Xue

1 related articles

Subscribe to updates
  1. arXiv

    ProQuant pushes multimodal LLMs to 4-bit weights and activations, holding Qwen3-VL-2B within about one point of full precision

    Small multimodal LLMs for edge devices lose accuracy sharply under low-bit quantization. ProQuant splits quantization-aware training into two steps: it first regenerates responses for open-source multimodal data with the target model so fine-tuning data match the pretrained distribution, then quantizes vision-sensitive layers first and gradually enables text-sensitive layers. On Qwen3-VL-2B-Instruct at W4A8, average accuracy stays about one point below full precision; at W4A4 it reaches 59.83, while the PTQ baseline MBQ scores 6.37.

You’re all caught up