Blind Image Restoration (BIR) aims to restore degraded images without knowledge of either the type or severity of the degradation. This is a challenging ill-posed inverse problem for which an image prior is essential. We view the prior of an image class as a probability distribution supported on a low-dimensional manifold embedded in the high-dimensional image space. Existing generative approaches primarily model the probability distribution, while the geometric structure is implicit or disregarded. Conversely, geometric approaches do not account for the non-uniform probability distribution on the manifold. We propose a geometric-probabilistic framework for BIR that models these aspects. The probability distribution on the manifold is approximated using a kernel-based method, while a distance from the manifold function induces a geometry-based probability function on the ambient space. This formulation leads to the (Latent) Manifold-Probabilistic Projection Model ((L)MPPM), a restoration model defined in both image and latent spaces. (L)MPPM induces a flow guided by a score field that jointly incorporates geometric and probabilistic information. Evaluated on image restoration tasks and datasets, (L)MPPM consistently outperforms existing approaches, providing empirical evidence for the benefit of incorporating both geometric and probabilistic information into the image prior.
Figures & tables
Figure 1: Illustration of our manifold-aware restoration approach. The blue path shows direct projection onto manifold M using distance function DM(x) , while the red-green path represents encoding-decoding through latent space Rd via functions F and G . Ideally, both paths converge to the same manifold point, ensuring geometrically consistent restoration.
Figure 2: An illustration of the kernel approximation Pker(z) of the probability distribution P(z) in the latent space.
Figure 3: The manifold M is illustrated as the curved line. xi∗ is the closest point to x on the manifold. Gˉ(x) is depicted as well and is not necessarily a point on the manifold.
Figure 4: The manifold M is the unit circle lying in the xy -plane and is parametrized by the azimuth angle θ . It is sampled according to a normal distribution centered at θ0 indicated by the red line. The reconstruction trajectory is shown in dark red. Note that the final result of the iterations on x does not converge to x∗ which is the closest point on the circle. Instead, it is influenced by the data distribution on the manifold through the effect of Gˉ(x) .
Figure 5: Comparison between denoising autoencoder (DAE) and our proposed MPPM, this example uses the same setup as in Fig. 4 . The error was computed as the deviation from the unit circle in 2D. In regions of the circle with lower probability density, the DAE is more prone to error than the proposed MPPM method.
Figure 6: Left: Diffusion model trajectory, Right: MPPM trajectory
Figure 8: Left panel: noise σ=0.7 ; middle panel: elastic ( α=0.34 , σ=1.8 ); right panel: downsampling factor =0.35 . In all panels, from left to right: degraded, DAE, LDM, LMMPM (ours), and original.
Figure 9: Top: Digit generation from pure noise, with an FID of 19.53 computed over 2000 images. Bottom: Progression of digit generation over 16 steps.
Figure 10: Noise σ=0.1 applied on CelebA-HQ-256. From left to right: degraded, DAE, LDM, DiffBir stage 1,DiffBir, LMPPM, LMMPM +DiffBir stage2, original.
Figure 11: Noise σ=0.2 . From left to right: degraded, DAE, LDM, DiffBir stage 1, DiffBir, LMPPM, LMMPM +DiffBir stage2, original.
Figure 12: Missing pixels (0.08). From left to right: degraded, DAE, LDM, DiffBir stage 1, DiffBir, LMPPM, LMMPM +DiffBir stage2, original.
Figure 13: Images with 5 scribbles. From left to right: degraded, DAE, LDM, DiffBir stage 1, DiffBir, LMPPM, LMMPM +DiffBir stage2, original.
Figure 14: Nose Masking. From left to right: degraded, DAE, LDM, DiffBir stage 1, DiffBir, LMPPM, LMMPM +DiffBir stage2, original.
Figure 15: Eye Masking. From left to right: degraded, MPPM with no ID loss, MPPM, DiffBir stage 1, DiffBir, LMPPM, LMMPM +DiffBir stage2, original.
Figure 16: Inference convergence (scribble). Top: distance to manifold DS . Bottom: probability P(z) . Iterations terminate once probability plateaus.
Elastic 2.3
Elastic 1.8
SSIM ↑
FID ↓
SSIM ↑
FID ↓
DAE
0.66
69.36
0.59
134.60
LDM
0.64
66.52
0.58
124.05
LMPPM (ours)
0.63
12.61
0.59
16.38
Ablation lmppm
0.63
12.83
0.59
16.27
Ablation dae
0.17
522.39
0.15
527.25
Table 1: Quantitative results on MNIST
Noise 0.1
Noise 0.2
Miss pixels 0.08
LPIPS ↓
FID ↓
LPIPS ↓
FID ↓
LPIPS ↓
FID ↓
DAE
0.047
41.48
0.058
41.52
0.073
41.55
LDM
0.155
22.49
0.352
28.27
0.666
43.81
DiffBIR Stage 1
0.059
14.51
0.334
37.48
0.426
47.11
DiffBIR (Full)
0.024
13.52
0.354
29.80
0.545
46.67
LMPPM
0.042
18.68
0.058
22.72
0.063
26.51
Table 2: Quantitative results on SCUT-FBP5500
Noise 0.1
Noise 0.2
Miss pixels 0.08
LPIPS ↓
FID ↓
LPIPS ↓
FID ↓
LPIPS ↓
FID ↓
DAE
0.018
44.34
0.075
44.43
0.129
43.82
LDM
0.205
27.24
0.512
34.51
0.736
45.93
DiffBIR Stage 1
0.063
19.43
0.459
32.74
0.436
42.27
DiffBIR (Full)
0.047
19.90
0.623
34.05
0.612
44.75
LMPPM
0.079
24.72
0.086
25.24
0.107
28.36
Table 3: Quantitative results on CelebA-HQ-256
Figure 17: Ablation Eye Masking. From left to right: degraded, LMPPM with no ID loss, LMPPM, original.
Figure 18: Ablation nose Masking. From left to right: degraded, LMPPM β=0 , LMPPM, original.
Figure 19: Ablation noise, σ=0.2 . From left to right: degraded, LMPPM α=0 , LMPPM, original.
Table 10: LMPPM training parameters across all experiments, determined through preliminary grid search, diffusion steps are define the number of steps in algorithm 2
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, School of Information Science and Technology, University of Science and Technology of China