Invisible perturbations wipe out visual place recognition recall: ARAFT lifts robust R@1 to 47.3 inside a 5-point clean-accuracy budget
Synopsis
Current DINOv2-based visual place recognition models lose almost all top-1 recall when the query image is imperceptibly perturbed: under a ranking attack, SuperVLAD and BoQ keep at most 0.2% R@1 on four benchmarks. ARAFT fine-tunes a released model with its original metric-learning loss on clean images, a listwise rank loss on each adversarial query's hardest positive, and an anchor that pulls the adversarial descriptor toward the clean descriptor of the frozen pretrained model, selecting checkpoints under a clean-accuracy budget fixed in advance. Within a 5-point validation clean R@1 budget, plain adversarial training has no admissible checkpoint, while ARAFT raises robust R@1 from zero to 47.3 on SuperVLAD and 55.3 on BoQ.
Interpretation
Released foundation-model visual place recognition systems largely fail to localise once the query image is slightly perturbed: an attacker who alters only the query can make the system report the wrong place. Earlier robustness work on visual place recognition studied natural changes such as city, season and illumination; prior work on adversarial perturbations reported the vulnerability and proposed detection, but no defense had been trained for visual place recognition. Under a ranking attack, SuperVLAD and BoQ keep at most 0.2% top-1 recall on the four benchmarks MSLS, Pitts30k, SPED and Nordland, and 0.0 at larger perturbation budgets.
ARAFT makes robustness and preservation of the pretrained geometry the same goal, which keeps the defense inside a small clean-accuracy cost. Plain adversarial training uses the same attack, positives and schedule but has no anchor and uses a hinge loss in place of the listwise rank loss. Within a 5-point validation clean R@1 budget, plain adversarial training has no admissible checkpoint on either model, so its budget model is the unmodified pretrained one with zero robustness; ARAFT's most robust checkpoint lies inside the budget, at 47.3 robust R@1 with a 4.6-point clean drop on SuperVLAD and 55.3 with a 4.7-point drop on BoQ.
On the test sets ARAFT trades less clean accuracy for robustness comparable to plain adversarial training, but only near the training budget. Without a budget, each at its most robust checkpoint, plain adversarial training gives up about ten more clean R@1 points on average. At perturbation budget 2 the two are level on average (46.1 vs. 46.4 on SuperVLAD, 49.8 vs. 47.2 on BoQ); at budget 4 plain adversarial training leads by 9.2 and 5.6 mean points, significant on every dataset under paired McNemar tests.
Perspective
The demonstrated scope is a white-box, query-only threat model under digital perturbations: the attacker has the model weights and the clean database descriptors and perturbs only the query image, while the database is built offline and read-only; physical and patch attacks and attacks on database images are out of scope, and all reported robustness is empirical rather than certified. Evaluation covers two architectures, SuperVLAD and BoQ, and four benchmarks, MSLS, Pitts30k, SPED and Nordland, with training on GSV-Cities and checkpoints selected under a 5-point clean R@1 budget fixed in advance on MSLS-val.
Robustness is empirical only: although several optimisers and budgets were tested with no sign of gradient masking, no gradient-free or transfer attacks were run against the ARAFT models, and certified guarantees are out of scope. The gradient-masking checks, the clean twin and the second seed are available for SuperVLAD only, and BoQ was trained and evaluated with one seed and the two headline attacks. At perturbation budget 4 on Nordland, robustness is near zero for every trained method, so only the budget-2 results indicate a meaningful defense on that benchmark. The 5-point budget is enforced on MSLS-val, and on the strongly shifted SPED and Nordland the clean cost reaches 17–24 points.
