Mamba image restoration drops fixed scans: CoScan learns a per-image traversal and lifts three backbones
Synopsis
Mamba-based image restoration must flatten a 2D feature map into a 1D sequence, and a fixed scan order can split apart neighboring pixels in degraded images. CoScan learns the scan order from image content: a differentiable proxy evaluator supervises scan quality, multi-scale traversals are coarsened from a fine-level minimum spanning tree, and a frequency-domain saliency map picks the start. Plugged into MambaIR, EAMamba and MaIR, it improves low-light enhancement, deblurring, raindrop removal, dehazing and super-resolution benchmarks without adding computation overhead.
Interpretation
Causal state-space models for image restoration no longer rely on a fixed space-filling scan; the traversal now changes with the input image content. Earlier Z-shaped, serpentine, window-based, diagonal and Hilbert-curve scans all follow the space-filling principle but are fixed and content-agnostic, so under degradations they can cross boundaries frequently or make long jumps, fragmenting the context. On LOL low-light enhancement, CoScan on MaIR gains 0.87 dB PSNR over the MaIR backbone, 1.07 dB on MambaIR and 0.89 dB on EAMamba; CoScan+MaIR reaches 0.66 dB PSNR above Restormer, the best non-Mamba baseline.
The scan order is produced by a differentiable learning pipeline rather than searched at inference time. The minimum-spanning-tree construction is non-differentiable, so scan quality cannot be optimized directly by gradients; an evaluator predicts the scan-quality objectives and serves as a differentiable proxy signal. The evaluator is trained to regress ground-truth objective vectors computed from two weight configurations, one from Dafner and one from the generator; with the evaluator fixed, the generator is updated through the proxy predictions, with a margin constraint to the Dafner reference applied only to the autocorrelation term.
Multi-scale traversals come from coarsening a fine-level minimum spanning tree, and the start point comes from frequency-domain saliency. Directly downscaling a curve introduces breaks and jumps, while generating a curve independently at each scale is costly and can be inconsistent across scales. On LOL, starting from a baseline that replaces MaIR scans with the generated curve, adding multi-scale scan scaling, frequency-guided start selection and prefix global tokens yields PSNR gains of 0.45 dB, 0.54 dB and 0.19 dB respectively.
Perspective
The demonstrated scope is five restoration tasks: low-light enhancement, deblurring, raindrop removal, dehazing and super-resolution, covering LOL plus four real-world low-light datasets, GoPro and HIDE, Raindrop, RESIDE-6K, and the Set5, Set14, B100, Urban100 and Manga109 super-resolution test sets; CoScan is inserted as a scan replacement into the MambaIR, EAMamba and MaIR backbones. For high-resolution images, scan generation and inference run patch by patch.
At high resolution, each patch generates its own scan order independently, which may limit modeling of extremely long-range dependencies across distant image regions; results on high-level vision tasks are described only as promising, with no numbers given. In the ablation, removing mini-batch weight averaging instead brings a 0.34 dB PSNR improvement, a direction that differs from the other components.
