Skip to main content
Back to timeline
arXivSource publication:

No SAR pretraining from scratch: SARVIA adapts off-the-shelf multimodal encoders in 600 steps and about 2 GPU-hours

Synopsis

Synthetic aperture radar (SAR) foundation models usually depend on costly large-scale domain pretraining. SARVIA starts elsewhere: it reuses the visual encoder of an existing multimodal large language model (MLLM), builds self-supervised visual questions with computable answers from unlabeled SAR observations, trains only low-rank adapters for brief specialization, and exports the encoder as a standalone backbone. In the default ViT-L setting, 600 adaptation steps and about 1.92 RTX PRO 6000 GPU-hours lead the compared foundation models on all six SAR detection and segmentation benchmarks.

Source-provided article image: SARVIA: Fast-Adapting the Strong SAR Foundation Models That Are Already in Your MLLMs
arXiv

Interpretation

A SAR domain backbone need not start from dedicated SAR pretraining; it can start from the visual encoder of an existing multimodal large language model plus a short domain adaptation. The prior route invested in large-scale SAR-specific pretraining to obtain transferable representations; under one protocol, the unadapted Qwen3.5-2B and Qwen3.5-4B encoders already reach 36.29 and 38.89 mIoU on the compact segmentation set, above TerraMind at 34.69 and SARMAE at 33.07. The adapted ViT-L backbone leads the compared methods on all six public benchmarks, RSAR, SARDet-100K, M4-SAR, OpenEarthMap-SAR, AIR-PolSAR-Seg-2.0 and WHU-OPT-SAR, while domain-training cost is orders of magnitude below dedicated pretraining.

The adaptation needs no manually annotated caption or question-answering corpus; supervision is constructed from the unlabeled observations themselves. Visual-instruction pretraining of the kind that relies on existing captioning, visual question answering, grounding and classification datasets makes annotated resources central to adaptation; SARVIA generates both questions and answers from unlabeled observations, known image operations and native radar measurements. The task pool holds 87 task variants drawn from TerraMesh, SARLO-80 and KuroSiwo; general image tasks cover spatial relations such as crop correspondence and patch ordering, while SAR-specific tasks cover native measurements such as relative power and heterogeneity and SAR-optical correspondence.

Adaptation jointly trains low-rank adapters in the visual encoder, the multimodal merger and the language model while base weights stay frozen, and only the visual encoder is exported afterward. Because answers can be written as text, a new task needs only a question generator and a computable answer, with no separate prediction head per task. The default setting adapts the Qwen3.5-2B visual encoder with rank-16 LoRA over 600 updates at global batch size 128, 76,800 question-answer draws in total; loss-driven sampling reallocates 10.55% of cumulative expected draws within fixed source budgets by step 600.

Perspective

The demonstrated scope is synthetic aperture radar detection and segmentation: adaptation uses unlabeled observations from TerraMesh, SARLO-80 and KuroSiwo, downstream evaluation covers the six public benchmarks RSAR, SARDet-100K, M4-SAR, OpenEarthMap-SAR, AIR-PolSAR-Seg-2.0 and WHU-OPT-SAR, and encoder comparisons use the OEM-SLC-Amplitude development set. The authors propose next to evaluate the recipe across domains and successive multimodal model generations under fixed adaptation budgets.

More adaptation is not better: extending rank-16 training to 1000 steps loses 1.18 mIoU and doubling the rank lowers the best score by 0.58; from step 600 to 800, diagnostic question-answering accuracy rises while development-set mIoU falls from 39.70 to 38.55. The authors also state that transfer beyond SAR remains untested and requires reliable, visually answerable tasks.