Skip to main content
Back to timeline
arXivSource publication:

A fixed linear erasure layer can be bypassed: one rank-one affine update routes the target score back through an editable upstream model

Synopsis

Once an open-weight model is released, an attacker can modify the layers that produce its features while a fixed affine erasure layer stays in place. The work formalizes this setting: if the erasure layer still transmits even one direction visible to an allowed downstream linear classifier, the attacker can write a target score into that direction and carry it through, and for a nonconstant linear target score the required change is a rank-one affine update. Across six CIFAR-10 class-pair tasks, the route restores nearly all pre-erasure target performance while leaving a deployed benign linear head's scores unchanged to numerical tolerance.

Source-provided article image: A Rank-One Bypass of Fixed Linear Erasure
arXiv

Interpretation

A fixed affine erasure layer constrains only what linear classifiers can read from the current features, not what the model can compute after its upstream layers are changed. Prior linear concept-erasure guarantees target the fitting distribution and a stated linear classifier family, under which no predictor in that family beats a constant predictor. On six CIFAR-10 class-pair tasks using ImageNet-pretrained ResNet-18 features behind a LEACE layer with five seeds per pair, the analytic rank-one adapter recovers the pre-erasure target-score accuracy once route strength exceeds the threshold, while the head-only baseline and the constant-map control stay at each task's base rate.

The attacker does not recover the target from the erased representation; it computes a target score before the erasure layer and writes that score into one direction the fixed layer still transmits. For a nonconstant linear target score the required change is a rank-one affine update, modifying the upstream representation along a single direction plus a bias. In all six pairs a route exists that preserves the deployed benign linear head's scores, a softmax head over the eight non-target CIFAR-10 classes, with observed drift within numerical tolerance and benign-head predictions identical to the clean model.

Perspective

Experiments use benign class-pair proxies in discriminative image models: six CIFAR-10 pairs that share classes, one fine-tuning budget and two small nonlinear evaluators, so no direct quantitative claim is made about diffusion models, CSAM-domain data or harmful generation. Theorem 1 covers only routing through a fixed affine layer; nonlinear, noisy or refitted bottlenecks are outside it. The authors propose a benign generative proxy as the natural next test.

The authors note that tight edit limits sharply weaken the route, but an open-weight release has no mechanism to enforce them, so an edit limit is an evaluation budget rather than a defense. An eraser fitted in double precision still leaves appreciable accuracy on held-out data, so suppression on the fitting sample need not transfer to new data. The fine-tuning controls do not separate generic from target-specific pretraining, and an unsampled interval precedes the first sampled checkpoint, so a transient confined to that interval is not excluded.