Agents that hand off KV caches directly now carry a tamper-detecting watermark: 1,500 handoffs caught, reasoning accuracy unchanged
Synopsis
Latent multi-agent systems skip text and pass KV-cache tensors straight to the next agent, a continuous channel that previously had no integrity check. KVMark embeds a training-free, zero-bit keyed watermark into both the V-cache and K-cache at each handoff, and a detector uses correlation scores to decide whether the cache was altered. On a four-agent Qwen3-4B pipeline, all 1,500 handoffs were detected with zero false positives, and accuracy on four benchmarks including GSM8K stayed within sampling error.
Interpretation
A KV-cache handoff between agents can now be verified: before processing, the receiver can tell whether the cache came from the expected sender and whether it was modified in transit. Text watermarks rely on vocabulary sampling during decoding and agent watermarks target discrete action sequences, so neither acts on continuous high-dimensional cache tensors; transport-layer HMACs catch post-handoff substitution but not malicious caches signed in-band by a compromised endpoint. On a four-agent Planner-Critic-Refiner-Judger pipeline with Qwen3-4B, 500 GSM8K samples produced 1,500 handoff verifications; every correct-key check passed, random-key scores sat near zero, and a margin separated the minimum correct score from the maximum wrong-key score.
The watermark is added along key-selected dimensions with position-dependent signs, so a party without the key cannot use statistics to find which dimensions carry the signal. A constant bias along fixed dimensions is exposed by column-mean analysis, and identical signs across attention heads let head-averaging preserve the signal. Across suppressed and boosted V-cache layers and all K-cache layers, nine statistical strategies including column mean, variance, kurtosis, range, absolute mean and cross-cache correlation recovered at most 3.0 of 16 true watermark dimensions, against a chance expectation of 2.0 of 16.
Perspective
The demonstrated setting is a homogeneous pipeline sharing one transformer architecture: a four-agent LatentMAS flow on Qwen3-4B, plus a transfer test on Qwen3-8B with the same GSM8K benchmark; the watermark acts directly on cache tensors without reading model weights or the tokenizer.
The authors note that sign patterns are anchored to absolute token positions, so dynamic cache compression, sliding-window attention or token eviction disrupt sequence indexing and trigger false tamper alerts; the three-tier strength parameters are tuned to Qwen3's layer-wise magnitude growth and need a few forward passes of calibration on model families with different normalization; and the method guarantees only tensor-level transport integrity, not semantic steering by an agent that reads its input and still generates biased content.
