Skip to main content
Search

# KVPath

1 related articles

Subscribe to updates
  1. arXiv

    KVPath links each token's key-value states across layers into one path, cutting GQA cache by up to 68% without raising perplexity

    KVPath stops caching a full key-value representation at every Transformer layer and instead links a token's key-value states across depth into one path: the first layer stores a full-dimensional base key-value, and each later layer generates only a low-dimensional delta that updates the preceding state. On parameter-matched 12-layer models, the residual and rolling variants cut KV-cache storage by up to 68.65% under GQA and 54.90% under MLA, with average perplexity equal to or lower than standard GQA and MLA.

You’re all caught up