Skip to main content
Back to timeline
arXivSource publication:

Poisoned fine-tuning can be repaired at two layers: harmful score falls from 11.54 to 0.08

Synopsis

In fine-tuning-as-a-service, malicious fine-tuning can weaken a model's refusal behavior while keeping its downstream task performance. SLDR replaces layer-wise safety diagnostics with a signed sensitivity spectrum, trains a LoRA recovery adapter only on the two layers with the highest and lowest sensitivity, and uses a representation-based maliciousness score at inference to decide when to activate it. On Llama3.1/SST2, the average harmful score drops from 11.54 to 0.08 while downstream accuracy matches undefended fine-tuning, and the harmful score stays near zero up to a poisoning ratio of 0.9.

Source-provided article image: SLDR: Defending Against Malicious Fine-tuning via Selective Layers Recovery and Dynamic Routing
Figure 1 · arXiv

Interpretation

A maliciously fine-tuned model can recover its refusal behavior by changing parameters in only two layers, without a whole-model rollback or pruning. Earlier post-fine-tuning defenses repaired the model at a coarse model level, for example by perturbing weights or pruning harmful parameters, which entangled safety repair with task-specific behavior and made it hard to restore refusal without disturbing downstream performance. Under the default Llama3.1/SST2 setting, the average harmful score falls from 11.54 for undefended fine-tuning to 0.08, with downstream accuracy of 92.32 against 92.39 for undefended fine-tuning; by comparison, Lisa and Antidote push the harmful score to 1.70 and 1.50 but drop downstream accuracy to 74.17 and 84.86.

The recovery adapter is attached only to the two ends of the sensitivity spectrum, and inference decides per query whether to use it. Earlier layer-wise safety diagnostics mainly reported where safety behavior is concentrated and did not distinguish whether strengthening a layer reinforces or suppresses refusal. Each layer's attention and feed-forward weights are scaled in isolation, and the change in refusal counts on an over-refusal dataset gives a signed sensitivity score; the highest- and lowest-scoring layers then receive the LoRA recovery adapter. At inference, the cosine-similarity gap between the last-token hidden state and harmful versus benign reference sets forms a maliciousness score, and only queries above the threshold of 0 use the recovered model. Ablations show that using either endpoint alone or other layers is worse: on Qwen2.5, the positive endpoint alone leaves a harmful score of 0.80, the negative endpoint alone 0.20, and other layers 0.90.

Perspective

The demonstrated scope is recovery after LoRA-based malicious fine-tuning: the downstream task stage can use either LoRA or full-parameter fine-tuning, but the recovery stage trains only a LoRA adapter, and experiments ran on a single A100-PCIE-40GB GPU without covering 70B-scale models. Evaluation spans four architectures (Llama3.1, Llama3, Qwen2.5, Mistral-v0.2), five downstream tasks (SST2, AGNEWS, GSM8K, AlpacaEval, Magicoder), and four harmful benchmarks (BeaverTails, DirectHarm4, HarmBench, HEx-PHI). Layer probing is a one-time offline cost of 0.3663 hours for 32 layers, and inference adds 0.0172 seconds per prompt.

Protection weakens noticeably under jailbreak attacks: SLDR reaches a harmful score of 61.53 on Zulu, lower than the original aligned model's 66.82 but still high overall, and 15.53 on IJP, above the original aligned model's 12.94. The robustness of the routing mechanism itself is not discussed, and an attacker could craft adversarial examples whose maliciousness score falls below the threshold to bypass dynamic routing. The recovery stage does not support full-parameter fine-tuning, and larger models and more complex out-of-distribution attacks remain untested.