Skip to main content
Back to timeline
arXivSource publication:

Two photos blended into one can be split again: training-free diffusion sampling recovers both sources and the blend map

Synopsis

Image deblending must split one composite image back into its source images, and a single observation admits many plausible decompositions, making the problem ill-posed. Training-free diffusion posterior sampling had addressed only single-output reconstruction, while supervised methods need domain-specific training and mostly assume a spatially constant blend coefficient. This work formulates deblending as a joint inverse problem, denoises two source images in parallel with one pretrained diffusion prior, constrains the solution with text prompts and a second-order polynomial alpha map, and reports the best PSNR, SSIM and LPIPS on unseen domains while staying competitive with supervised baselines on known-scalar benchmarks.

AI-generated editorial illustration: Image Deblending with Diffusion Models

Interpretation

A composite image formed by blending two images pixel-wise through an alpha map can now be split back into both sources, together with the blend weights, without task-specific training. Earlier training-free diffusion posterior sampling addressed single-output inverse problems, reconstructing one unknown image at a time; supervised decomposition methods need domain-specific training and mostly assume the blend coefficient is spatially constant. In the known-scalar setting, LCD scores 23.06 dB PSNR on Animals and drops to 17.21 dB on the unseen Vehicles domain, DAD drops from 17.42 to 11.09 dB, while this method moves from 21.36 to 20.23 dB.

Both source images are denoised in parallel by the same pretrained diffusion model and coupled only through a data-consistency term from the blending equation. The joint conditional score splits into independent prior scores and one shared likelihood gradient, which is obtained by estimating clean images from noisy samples with Tweedie's formula and backpropagating through the blending residual, so no network takes two images as input. The pipeline has two stages: a latent stage using Stable Diffusion v1.4 with cross-negative prompting for semantic separation, and a pixel stage using an unconditional DDPM trained on ImageNet for detail; evaluation uses 300 test pairs each from Fruits and Animals in the BIDeN benchmark plus 100 pairs each from Vehicles and Flowers.

The blend weights are no longer free per pixel but constrained to a second-order polynomial, so the alpha map cannot absorb image edges. Initial experiments with pixel-wise alpha maps showed a failure mode in which the optimization assigned high-frequency edge content to the alpha map while the recovered sources stayed blurred. The polynomial cuts the parameter space from one value per pixel to 9 coefficients and can express spatially constant, linear-gradient and radial blends; its four corner values are constrained to 0 to 1, but interior pixels are not strictly bounded.

Perspective

The demonstrated scope is blind decomposition of two-layer alpha blends, evaluated on 300 test pairs each from Fruits and Animals in the BIDeN benchmark and 100 pairs each from the unseen Vehicles and Flowers domains; the method targets offline use, and generating three Stage 1 candidates plus refining the selected one takes 50.6 to 71.0 minutes per decomposition, excluding selector overhead.

Text guidance makes the main comparison not strictly information-matched against prompt-free baselines; PickScore is trained for text-to-image preference and is an imperfect proxy for deblending quality; the polynomial alpha parameterization favors smooth blend maps, the spatially varying benchmark is drawn from the same quadratic family, and performance degrades gradually on smooth mattes outside it; the current formulation supports only two source layers.