Open-source LLM watermarking moves to a private vocabulary: injected once on Alpaca, SAME keeps a 100% fixed trigger rate after downstream fine-tuning
Synopsis
Once open-source model weights are public, third-party fine-tuning rewrites parameters and internal representations, breaking watermarks that rely on specific trigger words or parameter traces. SAME pairs every native token with a fixed private embedding and uses dual-path training so the private vocabulary inherits native semantics, tying the watermark to the model's evolving semantic ability. Injected once on Alpaca into Qwen3-8B-Instruct, it keeps a 100% fixed trigger rate after independent fine-tuning on SST-2, AG News, MMLU and BoolQ, while BadEdit falls to 10.87% on MMLU.
Interpretation
An ownership watermark for open-source LLMs can now be injected once and carried through downstream fine-tuning to both classification and generation tasks, without redesigning triggers per task. Earlier trigger-response watermarks tied triggers to task-specific data, transferred poorly across downstream formats and were progressively overwritten by later benign fine-tuning; parameter-space watermarks depend on parameters and internal representations that fine-tuning alters. On Qwen3-8B-Instruct, SAME is injected only on Alpaca and then fine-tuned on the native path for each target dataset; fixed verification reaches a 100% trigger rate on SST-2, AG News, MMLU and BoolQ, whereas BadEdit, injected with the corresponding downstream data, reaches 10.87% on MMLU and VPI reaches 47.50% on AdvBench.
The watermark rides on a private vocabulary with a token-by-token counterpart to the native vocabulary, and the model learns to read private embeddings as inputs meaning the same as native tokens. The private embeddings are initialized independently and stay fixed throughout injection, so only model parameters are updated and the same private encoding is consistent across training examples. Each input is fed to the same model through both native and private embeddings, with an autoregressive cross-entropy loss on each path; training uses 52K Alpaca instruction-response pairs for 10 epochs, with 500 examples held out for evaluation.
Beyond predefined trigger-response pairs, the same private vocabulary can be composed into new queries shaped to a downstream task format, and ownership is checked by the semantic similarity of the answer to the native answer. Fixed verification follows the conventional trigger-response paradigm, while semantic verification does not rely on predefined responses and adapts to different task formats without redesigning the watermark. Semantic verification reaches 68.17% accuracy on SST-2, a BERTScore-F1 of 85.61 on Dolly-15K and 89.83 on Alpaca, all with the same private vocabulary.
Perspective
Verification requires access to the suspect model's input-embedding interface so that privately encoded inputs can be supplied; experiments use Qwen3-8B-Instruct as the main model and evaluate LLaMA3.2, Qwen3, Mistral and GPT-OSS families and scales, covering dense and mixture-of-experts architectures plus one multimodal setting.
The private-vocabulary initialization magnitude is chosen empirically, with no established rule tying it to the native embedding scale; MoE and other model families need model-specific tuning to achieve effective private-vocabulary alignment. The authors also note that the private vocabulary may expose behaviors unavailable through the native interface, and that security against adaptive recovery or extraction attacks is not established.
