Image deblending is the task of decomposing a composite image back into its source images, which is an ill-posed problem, as a single blended image can be decomposed into multiple plausible source layers. While diffusion models are often used to solve inverse problems, they typically address single-output reconstruction problems. In this work, we formulate deblending as a joint inverse problem, employing diffusion posterior sampling to simultaneously reconstruct two source images, demonstrating that a single diffusion prior is expressive enough to disentangle two full images at once rather than just one. Drawing inspiration from human perception, where semantic understanding helps disentangle visual inconsistencies, we propose a training-free framework that leverages text prompts to constrain this solution space. We address the blind deblending task through a two-stage architecture: text-guided latent diffusion performs semantic separation using prompts as high-level cues, followed by a pixel-space refinement to recover fine details. Stochastic sampling produces multiple plausible decompositions that can be ranked without ground truth. Extensive experiments show that our training-free method achieves state-of-the-art results on unseen domains and extends to spatially-varying alpha maps, while remaining competitive with supervised baselines on standard scalar blending benchmarks.
Figures & tables
Figure 1: A single blended image is separated into two sources and a spatially varying α -map, without training. Top: (a,b) ground-truth sources, (c) ground-truth α -map, (d) blended input y . Bottom: our recovered sources (e,f), recovered α -map (g), and the re-blended reconstruction y~ (h).
Figure 2: Overview of our two-stage training-free blind deblending framework. Top: Stage 1 separates the blended image y in latent space, starting from E(y) noised to ts and guided by prompts c+1,c+2 . It runs from Ncand seeds, and PickScore selects one candidate pair by scoring each output against its prompt. Stage 2 re-noises that pair to tr and refines it in pixel space without prompts, keeping α fixed. Bottom: the guided step shared by both stages. Each source is denoised by a pretrained model and decoded ( D ; identity in Stage 2). In Stage 1 the model is conditioned on cross-negative prompts ( c−1=c+2 , c−2=c+1 ). The estimates are re-blended with the polynomial αϕ , and the gradient of ∥y−A∥2 updates both sources; in Stage 1, ϕ is also updated every k steps. Prompts: “Close-up image of green, red and orange apples” and “Head of a rooster, side view”.
Figure 3: Failure of unconstrained pixel-wise α . (a,b) source images; (c) ground-truth α -map; (d) blended input. (e-h) Result of optimizing a free per-pixel α -map jointly with the sources. The blended result (h) matches the input (d), but the predicted map (g) clearly exhibits “edge stealing,” where high-frequency details leak into the mixing mechanism, demonstrating that spatial optimization fails without proper constraints. Prompts: “ A close up photo of a lemon cut in half ”, “ A white bunny sleeping .”
In-Distribution (Seen Domains)
Generalization (Unseen Domains)
Fruits
Animals
Vehicles
Flowers
Scenario
Method
TF
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Known Scalar
DAD
✗
17.47
0.618
0.342
17.42
0.646
0.426
11.09
0.432
0.503
11.45
0.387
0.522
GIP
✗
17.77
0.588
0.380
17.61
0.587
0.493
14.07
0.509
0.508
14.09
0.482
0.504
BIDeN
✗
20.10
0.737
0.186
19.90
0.715
0.233
15.44
0.552
0.385
15.69
0.542
0.379
LCD
✗
23.04
0.812
0.132
23.06
0.795
0.249
17.21
0.689
0.293
17.19
0.655
0.298
Table 1: Quantitative evaluation on seen (Fruits, Animals; 300 pairs) and unseen (Vehicles, Flowers; 100 pairs) domains. Sources are matched to the ground truth by minimum MSE. TF: training-free. Bold: best per scenario, domain and metric. LCD and GIP support only a known scalar α . Ours and DoubleDIP select one of three candidates with PickScore; ours selects without ground truth, while DoubleDIP’s outputs are first put in ground-truth order ( Sec. C.4.5 of the supplementary material). Only ours uses text prompts ( Sec. 5.1 ).
Figure 4: Qualitative alpha-deblending comparison for Known Scalar ( KS ), Unknown Scalar ( US ), and Spatially Varying ( SV ) mixing (rows) on a seen (Market + Pigeons) and an unseen (Lego + Purple Flower) pair, chosen by eye as representative examples; further examples are in the supplement. Each method shows its two recovered sources side by side; GT gives the two ground-truth sources. LCD and GIP apply only to KS, and N/A marks inapplicable methods. Baselines leave residual traces of one source in the other, which our method largely avoids.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Qualitative comparison on seen domains. Results for Fruits vs. Animals across mixing scenarios: Known Scalar ( KS ), Unknown Scalar ( US ), and Spatially Varying ( SV ). The four blocks represent distinct test cases: Redberries–Fox , Strawberry–Giraffe , Strawberry–Squirrel , and Fruit–Duck . Each block is one blended pair.
Figure 6: Qualitative comparison on unseen domains. Results for Vehicles vs. Flowers across three mixing scenarios: Known Scalar ( KS ), Unknown Scalar ( US ), and Spatially Varying ( SV ). The three blocks represent: Gray Car–Pink Flower , Police–White Flower , and Tank–Purple Flower . Each block is one blended pair; the baselines were not trained on these domains.
Sample 1
Sample 2
Scenario
Method
CFG ( ω )
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Known Scalar
Baseline
0.0
17.7258
0.7207
0.2215
17.6530
0.7166
0.2666
Cross-neg. (Ours)
1.0
20.1523
0.7842
0.1494
20.0426
0.7811
0.1890
2.0
20.5356
0.7852
0.1499
20.3608
0.7864
0.1829
3.0
19.5746
0.7800
0.1564
19.5845
0.7787
0.1946
Standard CFG
1.0
20.3293
0.7897
0.1447
20.1637
0.7871
0.1827
Appendix
Table 2: Ablation of text-guidance strategies on 100 Fruits/Animals pairs with Ncand=1 . Baseline is prompt-free sampling ( ω=0 ); Ours uses the opposing source’s description as the negative prompt (cross-negative); Standard CFG uses an empty negative prompt; Class-Level uses generic class labels. Sample 1 and Sample 2 are the two recovered sources. Bold marks the best value per scenario and column. Cross-negative prompting is clearly better than standard CFG on spatially varying α , while the two are comparable for a known scalar.
Figure 7: Qualitative comparison of text guidance methods. We compare three text guidance strategies across different guidance strengths ( ω ) for two datasets: Apples–Cat (top) and Fruits–Kangaroo (bottom). The methods are: Ours (cross-negative prompting: the other source’s description is the negative prompt), Standard CFG (empty negative prompt), and Class-Level (cross-negative prompting with generic class labels instead of descriptions). ω=0.0 is prompt-free sampling and is identical across methods. Ours is the top row of each block; quantitative results are in Tab. 2 .
Prompts
PSNR ↑
SSIM ↑
LPIPS ↓
None ( ω=0 )
14.98
0.650
0.310
Auto (Qwen2-VL)
15.63
0.658
0.296
Manual
16.31
0.700
0.249
Appendix
Table 3: End-to-end automated-prompt study on 80 spatially varying blends with three Stage 1 candidates and one Stage 2 refinement. Auto prompts are generated by the VLM from the blended observation alone.
Figure 8: Qualitative automated-prompt examples from the 80-pair end-to-end study. The VLM receives only the blended observation ( Input ) and writes both prompts shown under the Auto decompositions; Manual shows the same blend decomposed with the human-written descriptions used elsewhere in the paper. The two examples are the pairs closest to the median Auto–Manual PSNR gap ( Representative : Auto 12.00 dB vs. Manual 12.52 dB) and to the 10th-percentile gap ( Failure : Auto 16.10 dB vs. Manual 20.55 dB), selected by these pre-specified quantiles rather than by visual inspection. Both illustrate source misidentification: the VLM describes the fruit basket as a landscape and the owl as a person’s face, and the corresponding Auto layers retain content from the other source.
Domain
Scenario
Selector
PSNR ↑
SSIM ↑
LPIPS ↓
Seen
Known scalar
CLIP
20.7634
0.8074
0.1493
PickScore
21.3595
0.8134
0.1418
Unknown scalar
CLIP
19.1261
0.7440
0.2078
PickScore
19.7825
0.7634
0.1902
Spatially varying
CLIP
17.5881
0.7189
0.2261
PickScore
18.0521
0.7325
0.2118
Appendix
Table 4: Stage 1 selector comparison on 300 seen (Fruits, Animals) and 100 unseen (Vehicles, Flowers) pairs per scenario. Metrics are computed from the matching same-identifier Stage 2 refinement and averaged over both recovered sources. Higher PSNR/SSIM and lower LPIPS are better.
Scenario
Ncand
PSNR
SSIM
LPIPS
Known scalar
1
21.08
0.769
0.236
3
21.57
0.802
0.198
16
22.97
0.829
0.180
Unknown scalar
1
19.90
0.735
0.232
3
20.51
0.762
0.234
16
22.02
0.783
0.201
Appendix
Table 5: Representative points from the candidate-budget study, averaged over 10 input pairs per scenario. Scalar rows use the given α (see text). Lower LPIPS is better; higher PSNR and SSIM are better.
Scenario
Selector
PSNR ↑
SSIM ↑
LPIPS ↓
Comparison (300 pairs)
Known Scalar
CLIP
14.42
0.515
0.537
PickScore
14.58
0.521
0.535
Unknown Scalar
CLIP
14.66
0.511
0.545
PickScore
14.69
0.516
0.552
Spatially- Varying
CLIP
13.68
0.457
0.597
Appendix
Table 6: Best-of-three selector comparison for the training-free DoubleDIP baseline. CLIP and PickScore independently select from the same three samples for each input; outputs are paired with prompts in ground-truth source order (see text). Bold marks the best value within each evaluation set and mixing scenario. Lower LPIPS is better; higher PSNR and SSIM are better.
n1\n2
2
4
6
8
10
4
14.520/.565/.357
14.923/.597/.335
15.131/.631/.312
15.426/.637/.312
15.529/.651/.290
6
14.816/.587/.325
15.252/.618/.301
15.515/.647/.279
15.765/.658/.267
15.794/.661/.266
8
15.560/.625/.295
16.000/.656/.270
16.282/.671/.259
16.737/.690/.241
16.868 / .695 /.239
10
15.326/.631/.280
15.705/.659/.259
15.964/.674/.250
16.259/.689/.236
16.445/ .695/.232
Appendix
Table 7: Recurrence-count ablation on 20 spatially varying Fruits/Animals blends with Ncand=1 . Rows vary Stage 1 recurrences n1 and columns vary Stage 2 recurrences n2 ; each cell reports PSNR ↑ /SSIM ↑ /LPIPS ↓ . Bold denotes the best value and ties in the displayed digits are bold in every tied cell, and underlining marks the adopted setting (n1,n2)=(8,8) . Metrics are recomputed from saved outputs at 256×256 after permutation-invariant source matching.
Figure 9: Performance across Stage 1 starting timestep ts (rows, out of Ts=100 ) and Stage 2 starting timestep tr (columns, out of Tr=600 ) on 20 spatially varying Fruits/Animals blends with Ncand=1 . The starred box marks the adopted setting (ts,tr)=(80,350) . Each panel has its own colour scale.
Scenario
Stage
PSNR ↑
SSIM ↑
LPIPS ↓
Known Scalar
1
17.3872
0.5910
0.3175
2
21.3595
0.8134
0.1418
Unknown Scalar
1
16.7536
0.5668
0.3450
2
19.7825
0.7634
0.1902
Spatially- Varying
1
15.6474
0.5453
0.3634
2
18.0521
0.7325
0.2118
Appendix
Table 8: Effect of Stage 2 refinement on 300 Fruits/Animals inputs per mixing scenario. Stage 1 is the PickScore-selected candidate (downsampled to 256×256 ) and Stage 2 its refinement. Metrics are averaged over both recovered sources.
Figure 10: Qualitative results of the two-stage pipeline. Stage 1 performs coarse semantic separation; Stage 2 restores fine details and better matches the ground truth.
GT matte
Estimator
PSNR ↑
SSIM ↑
LPIPS ↓
α MAE ↓
Polynomial
Quadratic
14.18
0.504
0.586
0.180
Grid 4×4
14.52
0.518
0.565
0.142
Gaussian
Quadratic
13.91
0.500
0.599
0.265
Grid 4×4
14.11
0.507
0.603
0.237
Reflection-like
Quadratic
13.52
0.482
0.625
0.242
Grid 4×4
13.66
0.496
0.616
0.216
Appendix
Table 9: Stage 1 results (one candidate per input) on 20 Fruits/Animals pairs per ground-truth matte family. Bold marks the better estimator for each ground-truth matte; the paired 95% confidence intervals of the PSNR differences include zero. Alpha MAE is the mean absolute error between the recovered and ground-truth α -maps.
Diffusion models for image restoration use generative priors to denoise, deblur, upscale, and reconstruct images from incomplete or corrupted measurements.
Pretrained diffusion models provide powerful image priors for training-free posterior sampling in image restoration. To guide this sampling process, frequency-aware methods progressively incorporate measurement information across frequency bands, facilitating coarse-to-fine reconstruction. However, existing methods typically do not explicitly separate frequency activation from degradation-induced attenuation, leaving attenuation differences among inactive frequencies insufficiently modeled. In this work, we propose frequency-decoupled posterior guidance to separate frequency activation from attenuation-aware spectral regularization. Specifically, a progressive low-to-high frequency schedule determines the active measurement band, while a kernel-derived attenuation map defines a selective spectral prior over inactive components. To stabilize the sampling process, we also introduce a local trajectory regularizer that suppresses spatially irregular state-to-clean deviations. For a fixed endpoint energy, we provide a KL-regularized path-space interpretation. In practice, we construct time-dependent guidance through local energy corrections using a Tweedie plug-in approximation. Experiments on natural-image benchmarks demonstrate strong PSNR and SSIM performance across challenging non-blind deblurring settings, even at higher measurement noise levels.
Sihan Wang, Jinshu Huang, Haibin Su +1
School of Mathematical Sciences, Nankai University, Tianjin, China. · Yau Mathematical Sciences Center, Tsinghua University, Beijing, China.
Text-to-image diffusion models can generate individual concepts well, but they often omit or merge concepts incorrectly with multiple concepts. We trace these failures to an early coordination bottleneck: before denoising begins, prompt-conditioned attention may allocate different concepts to strongly overlapping spatial support, which can keep their attention coupled as denoising proceeds. This observation motivates treating compositional generation as a boundary-condition problem rather than repeatedly controlling the evolving trajectory. To this end, we propose Rectify-then-Diffuse (RTD), a training-free framework that rectifies the initial allocation once before standard denoising. Firstly, we propose Soft-Overlap Disentanglement (SOD), which converts normalized overlap between pilot concept maps into a differentiable and layout-agnostic separation objective. Secondly, we introduce Isotropic Gradient Rectification (IGR), which normalizes the SOD gradient and applies a bounded latent displacement with a consistent scale across prompts and initializations. Extensive experiments show that RTD achieves state-of-the-art compositional fidelity and robust gains. On the AE-Bench object pair subset, RTD improves BLIP-VQA by 45.8% and ImageReward by 19.6% over CO3 while running 2.3× faster. Code will be released at https://github.com/Z-yiwei/rectify-then-diffuse
Ning Zhu, An Chen, Mengfei Zhao +4
Glasgow College, University of Electronic Science and Technology of China · School of Mathematical Sciences, University of Electronic Science and Technology of China
Image acquisition with a camera involves several degradations due to the optical system, sensor, or low-level processing steps. We address blind deblurring in professional photography: we aim to invert unknown isotropic blur without knowledge of the degradation kernel. For such inverse problems,where some high-frequency information is lost, it is challenging to use generative models to produce details that are both photo-realistic and faithful to the input. We propose SuperSharpen, a diffusion-based blind deblurring method offering explicit control over restoration strength through a blur measure. We compare two conditioning strategies: a ControlNet-style adapter on a frozen backbone, and full finetuning of the diffusion prior. Our experiments show that finetuning achieves better fidelity with fewer hallucinated details. We validate our approach on synthetic and real-world blur, demonstrating improved perceptual quality and controllable restoration strength.
Imane Si Salah, Emile Cribelier, Thomas Veit +2
DxO Labs, France · LTCI, Télécom Paris, IP Paris, France