KVPath links each token's key-value states across layers into one path, cutting GQA cache by up to 68% without raising perplexity
KVPath stops caching a full key-value representation at every Transformer layer and instead links a token's key-value states across depth into one path: the first layer stores a full-dimensional base key-value, and each later layer generates only a low-dimensional delta that updates the preceding state. On parameter-matched 12-layer models, the residual and rolling variants cut KV-cache storage by up to 68.65% under GQA and 54.90% under MLA, with average perplexity equal to or lower than standard GQA and MLA.