Skip to main content
Search

Minsoo Cheong

1 related articles

Subscribe to updates
  1. arXiv

    LoopVQ squeezes looped-transformer KV caches to 0.5 bit: Ouro-1.4B holds 75.82 on GSM8K where CQ drops to 46.78

    Looped transformers reuse the same layers and save parameters, but each loop step still stores its own keys and values, so the KV cache grows linearly with loop steps and becomes the memory bottleneck. LoopVQ packs channels from several consecutive loop steps into one sub-vector that shares a single codebook index, then fine-tunes the codebooks to match the full-precision output. On Ouro-1.4B at 0.5 bits per element it scores 75.82 on GSM8K, against 46.78 for CQ and 57.32 for TaSQ, two vector-quantization methods built for standard LLMs.

You’re all caught up