Lipschitz-Controlled Flow Matching for Robust Image Restoration
Organizations: Paderborn University · TU Dortmund University
Abstract
Flow-matching models can produce high-quality image restorations, but their learned dynamics may amplify small input perturbations and make reconstructions unstable. We introduce LipsFlower to reduce this amplification. We derive stability bounds for Flow-from-, which initializes the ordinary differential equation from a noise-perturbed observation, as well as for observation-conditioned posterior sampling, which conditions the velocity field on the observation throughout integration. Based on this analysis, we develop LipHDiT, a Lipschitz-aware hourglass diffusion transformer that serves as the vector-field architecture of LipsFlower. LipHDiT incorporates stabilized approximate spectral normalization, bounded modulation, and convex residual mixing. We also use adversarial finite-radius gain regularization at sampled interpolation states to penalize excessive sensitivity to the state and, for conditional sampling, the observation. We test both formulations on blind face restoration, measuring adversarial robustness and the perceptual quality of the reconstructions. LipsFlower reduces perturbation amplification while keeping restoration quality competitive with the evaluated baselines. These results suggest that Lipschitz-aware flow matching can improve the stability of generative image restoration.
Figures & tables
| Method | SPSA– (RMS budget) | SPSA– | |||||||||
| Mean | P95 | Max | LPIPS drift | Mean | P95 | Max | LPIPS drift | ||||
| (I) | CodeFormer | 2.44 | 4.04 | 4.577 | 7.17 | 63.375 | 2.91 | 4.82 | 5.495 | 9.35 | 53.101 |
| GFPGAN | 3.01 | 5.13 | 5.664 | 7.53 | 110.573 | 3.60 | 6.45 | 7.122 | 9.57 | 98.214 | |
| RestoreFormer++ | 1.77 | 3.75 | 4.445 | 6.79 | 44.146 | 2.10 | 4.56 | 5.406 | 8.53 | 38.113 | |
| OSDFace | 4.99 | 6.49 | 6.898 | 8.13 | 186.731 | 5.83 | 7.77 | 8.262 | 9.87 | 155.168 | |
| PMRF | 0.47 | 1.11 | 1.455 | 2.91 | 9.033 | 0.47 | 1.03 | 1.355 | 2.96 | 5.454 | |
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | PSNR | LPIPS | FID | Precision | DEG | LMD | |
| (I) | CodeFormer | 25.146 | 0.343 | 52.521 | 0.3512 | 37.282 | 2.470 |
| GFPGAN | 24.986 | 0.364 | 46.160 | 0.3212 | 36.036 | 2.444 | |
| RestoreFormer++ | 25.313 | 0.344 | 40.611 | 0.4089 | 29.634 | 2.043 | |
| OSDFace | 23.381 | 0.364 | 43.567 | 0.4631 | 33.907 | 2.158 | |
| PMRF | 25.821 | 0.367 | 35.842 | 0.3855 | 31.027 | 2.037 | |
| ELIR | 25.817 | 0.374 | 39.944 | 0.2441 | 34.258 | 2.308 | |
| Scale | Probability | Radius distribution |
| Small | ||
| Moderate | ||
| Severe |
| Setting | Flow-from- | Conditional |
| Number of GPUs | ||
| Batch size per GPU | ||
| Gradient accumulation | ||
| Effective batch size | ||
| Regularization interval | ||
| Selected examples per GPU |
| Setting | Backbone | Gain reg. | SPSA– (RMS budget) | SPSA– | ||||||||
| Mean | P95 | Max | LPIPS drift | Mean | P95 | Max | LPIPS drift | |||||
| A: Baseline | HDiT | – | 0.525 | 0.743 | 0.849 | 1.740 | 15.803 | 0.523 | 0.737 | 0.830 | 1.660 | 9.188 |
| B: Regularizer only | HDiT | Yes | 0.559 | 0.791 | 0.904 | 1.584 | 14.040 | 0.558 | 0.770 | 0.878 | 1.631 | 8.660 |
| C: Architecture only | LipHDiT | – | 0.486 | 0.688 | 0.775 | 1.309 | 12.530 | 0.489 | 0.674 | 0.750 | 1.277 | 7.772 |
| E: LipsFlower | LipHDiT | Yes | 0.475 | 0.667 | 0.747 | 1.158 | 11.649 | 0.480 | 0.658 | 0.727 | 1.118 | 7.360 |
| Setting | Backbone | Gain reg. | PSNR | LPIPS | FID | Precision | DEG | LMD |
| A: Baseline | HDiT | – | 25.784 | 0.350 | 37.050 | 0.3797 | 30.602 | 1.995 |
| B: Regularizer only | HDiT | Yes | 25.298 | 0.384 | 35.306 | 0.4005 | 31.145 | 2.045 |
| C: Architecture only | LipHDiT | – | 25.474 | 0.379 | 39.906 | 0.2625 | 34.148 | 2.345 |
| E: LipsFlower | LipHDiT | Yes | 25.475 | 0.388 | 40.769 | 0.2711 | 34.143 | 2.351 |
| Method | Inference configuration |
| CodeFormer (a) | Fidelity weight ; aligned inputs; upscale factor ; background upsampling disabled. |
| GFPGAN (b) | Released v1.3 checkpoint with the clean architecture; restoration weight ; aligned inputs; upscale factor . |
| RestoreFormer++ (c) | Released RestoreFormer++ checkpoint; aligned inputs; upscale factor . |
| OSDFace (d) | Stable Diffusion 2.1-base backbone; one U-Net evaluation at timestep ; merged LoRA with rank and scale parameter ; FP32 inference. |
| DiffBIR (e) | Version 2 face-restoration pipeline; spaced sampler with steps; classifier-free guidance scale ; captioning disabled; FP32 inference. |
| PMRF (f) | Official implementation and pretrained checkpoint; flow integration steps. |
| Setting | Flow-from- | Conditional |
| Image resolution | ||
| Number of GPUs | ||
| Batch size per GPU | ||
| Accumulation steps | ||
| Effective batch size | ||
| Learning rate |
Flow Matching
Flow matching learns continuous-time vector fields that transport simple noise distributions into data for efficient generative modeling.
See all 702 papers