PUMA keeps match quality in the fusion loop: it beats listed representative methods on COVERAGE and NIST16
Synopsis
Image manipulation localization usually fuses semantic features with noise-forensic features across two branches, but the common Softmax normalization keeps only relative weights and discards how well the two feature sets match overall. PUMA replaces balanced matching with unbalanced optimal transport inside a local neighborhood, so transported mass can vary with match quality, and it retains signed mass residuals before normalization. Those residuals then set, position by position, how two propagation orders are mixed. On four test sets, PUMA scores above the published results of the representative methods in its comparison on COVERAGE and NIST16.
Interpretation
Cross-branch interaction no longer emits only normalized aggregation weights; it keeps the match quality between semantic and noise features as a usable signal. Earlier semantic–forensic fusion methods mostly used Softmax normalization to produce aggregation weights, preserving relative allocation among features but not the overall degree of matching between the two feature sets. On four test sets, PUMA scores above the published results of the representative methods in its comparison on COVERAGE and NIST16.
The interaction module first aligns features on a shared spatial grid, then uses spatially constrained unbalanced optimal transport to establish cross-branch correspondences. Unbalanced transport relaxes mass conservation, so the amount of transported feature mass can vary with bilateral match quality instead of forcing all information to be moved. Matching is restricted to a truncated RGB neighborhood of radius 2 around each noise position, with at most 25 candidates, permitting small local offsets while controlling erroneous correspondences and computational cost.
Outputs of the two propagation orders are mixed by a position-wise gate driven by mass residuals, and noise features are updated through a gated residual path. Existing methods commonly fix the execution order of within-branch aggregation and cross-branch transport, so they cannot adapt to location-dependent differences such as those between normal regions and manipulation boundaries. The module is inserted at the pre-fusion interface while the backbone and prediction pathway stay unchanged, and the updated noise features plus the original semantic features go to the baseline's existing fusion and localization head.
Perspective
The demonstrated scope is pixel-level image manipulation localization, evaluated on four test sets and several post-processing conditions; the module sits at the pre-fusion interface while the backbone and localization head follow the baseline.
The experiments support the utility of local matching states and adaptive sequential interaction for cross-source localization and several post-processing conditions, but that conclusion is limited to the reported test sets and comparison list.
