How far have we gone in Generative Image Restoration? A study on its capability, limitations and evaluation practices

Xiang Yin; Jinfan Hu; Zhiyuan You; Kainan Yan; Yu Tang; Chao Dong; Jinjin Gu

cs.CVMar 5, 2026

How far have we gone in Generative Image Restoration? A study on its capability, limitations and evaluation practices

Authors: Xiang Yin, Jinfan Hu, Zhiyuan You, Kainan Yan, Yu Tang, Chao Dong, Jinjin Gu

Organizations: Fudan University · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · University of the Chinese Academy of Sciences · Multimedia Laboratory, The Chinese University of Hong Kong · Shenzhen University of Advanced Technology · INSAIT, Sofia University “St. Kliment Ohridski”

Abstract

Generative Image Restoration (GIR) has achieved impressive perceptual realism, but how far have its practical capabilities truly advanced compared with previous methods? To answer this, we present a large-scale study grounded in a new multi-dimensional evaluation pipeline that assesses models on detail, sharpness, semantic correctness, and overall quality. Our analysis covers diverse architectures, including diffusion-based, GAN-based, PSNR-oriented, and general-purpose generation models, revealing critical performance disparities. Furthermore, our analysis uncovers a key evolution in failure modes that signifies a paradigm shift for the perception-oriented low-level vision field. The central challenge is evolving from the previous problem of detail scarcity (under-generation) to the new frontier of detail quality and semantic control (preventing over-generation). We also leverage our benchmark to train a new IQA model that better aligns with human perceptual judgments. Ultimately, this work provides a systematic study of modern generative image restoration models, offering crucial insights that redefine our understanding of their true state and chart a course for future development.

Figures & tables

Figure 2: Distribution of annotated total scores across semantic scene groups. The horizontal width of each box indicates the percentage of samples within each score interval. The light gray region indicates low overall scores, representing generally unacceptable results.

Figure 3: Detail and sharpness score distributions across semantic scenes. The width of each box corresponds to the percentage of samples. The red line indicates the balance point; scores above it ( Over $\uparrow$ ) denote over-generation, and scores below it ( Less $\downarrow$ ) denote under-generation.

Model	Architecture	Model	Architecture
HYPIR [ 6 ]	Diffusion-based	S3Diff [ 85 ]	Diffusion-based
SUPIR [ 5 ]	Diffusion-based	TSD-SR [ 86 ]	Diffusion-based
PiSA-SR [ 31 ]	Diffusion-based	ResShift [ 29 ]	Diffusion-based
SeeSR [ 32 ]	Diffusion-based	FLUX [ 87 ]	Generation
OSEDiff [ 33 ]	Diffusion-based	Nano Banana [ 88 ]	Generation
CCSR [ 89 ]	Diffusion-based	BSRGAN [ 25 ]	GAN-based

Table 1: Restoration or generation models used in the benchmark.

Dataset	# Images	# IR Model	# GIR Models	Multi. Aspect
SRIQA-Bench [ 9 ]	1.1K	10	8	✗
ISRGen-QA [ 8 ]	0.72K	15	10	✗
Ours	7K	20	18	✓

Table 2: Comparison of existing datasets for GIR assessment.

Panel of Figure 4 — Figure 4: Illustration of our annotation criteria. The rows (top to bottom) indicate: Detail (under-generated [-2] to over-generated [+2]), Sharpness (blurred [-2] to over-sharpened [+2]), and Semantic correctness (severe failure [0] to fully consistent [4]).

Figure 5: Distribution of semantic consistency scores across scenes. Box widths represent the sample percentage at each score. The light gray region marks unacceptable results.

Panel of Figure 6 — Figure 6: Common failure modes in hard cases: distorted faces in crowd scenes (top), structural errors in hands/feet (middle), and illegible characters in text (bottom).

Figure 7: Diffusion models exhibit a dual tendency in detail generation: sometimes under-generating textures, leading to overly smooth results (top row), and at other times over-generating, creating implausible or redundant details (middle and bottom rows)

Figure 8: Examples illustrating over-generation, semantic infidelity, and degradation unremovement in generation models: more whiskers and railings are generated (first), the geometry of the face is not restored, and the position of the shoes is completely changed (second and third), and artifacts and noise are not eliminated (last).

Panel of Figure 9 — Figure 9: Examples of hard degradations ( motion blur , old film ).

Figure 10: Averaged overall scores of different model families across various degradation types.

Table 3: Diffusion-based models’ parameter sensitivity. Mean scores show that the optimal configuration is scene- and degradation-dependent.

Figure 11: Effect of parameter configurations on restoration behavior in diffusion-based models (HYPIR as an example).

	NIQE	PI	MANIQA	MUSIQ	CLIP-IQA	DeQA-Score	Ours
SRCC	0.164	0.178	0.361	0.376	0.389	0.409	0.662
PLCC	0.206	0.230	0.391	0.427	0.420	0.464	0.677

Table 4: Performance comparison of existing IQA methods and our trained IQA model on our dataset.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 12: Distribution of annotated overall scores across semantic scene groups. The horizontal width of each box indicates the percentage of samples within each score interval. The light gray region indicates low overall scores, representing generally unacceptable results.

Model	Config	Configurations
	ID
HYPIR	1	sharpness:800, noise:600
	2	sharpness:4000, noise:200
	3	sharpness:4000, noise:600
PiSA-SR	1	lambda_pix:1.0, lambda_sem:0.3
	2	lambda_pix:2.0, lambda_sem:1.0

Appendix

Table 6: Configurations used in our parameter sensitivity analysis. All other parameters remain at their official defaults.

Model	Config	SmallFace	Crowd	Vehicles	Street	Aerial	Hands/	Complex	Text	Digital	OP
Model	ID	Face	Crowd	Vehicles	View	View	Feet	Texture	Text	Zoom	(Color)
HYPIR	1	2.83	2.00	3.00	2.71	2.90	2.68	2.58	2.50	2.17	2.23
	2	2.94	2.50	3.10	2.86	2.90	3.05	2.75	2.61	2.61	2.43
	3	2.61	2.10	3.20	2.71	2.70	2.73	2.58	2.50	2.26	2.14
PiSA-SR	1	1.78	1.60	2.30	2.00	1.80	2.64	2.17	2.22	2.15	1.67
	2	1.67	1.30	2.40	2.21	1.70	2.68	2.00	2.11	2.04	1.94

Appendix

Table 7: Average overall scores of different parameter configurations across semantic categories. Bold numbers indicate the best-performing configuration within each model for a given category

Model	Avgerage Overall Score $\uparrow$	Avgerage Rank $\downarrow$
HYPIR	2.775	3.127
PiSA-SR	2.508	5.547
TSD-SR	2.467	7.487
DiffBIR	2.428	5.181
PASD	2.38	6.382
OSEDiff	2.377	6.173

Appendix

Table 8: Overall performance ranking of all evaluated restoration models. We report each model’s mean overall quality score across the full dataset and its average per-image ranking position.

Test set	NIQE	MANIQA	MUSIQ	CLIP-IQA	DeQA-Score	Ours
LIVEW	0.81/0.77	0.85/0.83	0.79/0.83	0.83 / 0.81	0.89 / 0.88	0.83 / 0.81
AGIQA	0.72/0.65	0.72/0.64	0.72/0.63	0.74/0.69	0.81 / 0.73	0.81 / 0.76
CSIQ	0.70/0.65	0.62/0.63	0.77/0.71	0.77/0.72	0.79 / 0.74	0.86 / 0.83

Appendix

Table 9: Performance comparison of our proposed method against popular IQA models on three external datasets. Results are reported in terms of SRCC/PLCC. The best and second-best results are highlighted in bold and underlined respectively.

Panel of Figure 13 — Figure 13: More low-quality examples of different semantic categories. Zoom in for a better view.

Panel of Figure 14 — Figure 14: More low-quality examples of different semantic categories. Zoom in for a better view.

Panel of Figure 15 — Figure 15: More visual examples of different degradation categories. Zoom in for a better view.

Panel of Figure 16 — Figure 16: More visual examples of different degradation categories. Zoom in for a better view.

Panel of Figure 17 — Figure 17: Results of different prompts used in generation models. Zoom in for a better view. Prompt 1: Perform high-quality image restoration and super-resolution in this image: enhance clarity, remove noise and scratches, reduce blur, and sharpen fine details—while preserving a natural, realistic appearance without over-processing or artificial artifacts. Prompt 2: Restore this image with enhanced clarity and sharpness. Reduce noise, fix compression artifacts, and improve detail while strictly preserving the original content, style, and semantics. Prompt 3: Clean and restore the image by removing artifacts, noise, and blur. Maintain full fidelity to the original composition and avoid creating new visual elements. Prompt 4: Improve this image’s perceptual quality: refine edges, correct degradation, and enhance fine details. Do not alter identity, expression, or any semantic attributes.

Panel of Figure 18 — Figure 18: Results of different prompts used in generation models. Zoom in for a better view. Prompt 1: Perform high-quality image restoration and super-resolution in this image: enhance clarity, remove noise and scratches, reduce blur, and sharpen fine details—while preserving a natural, realistic appearance without over-processing or artificial artifacts. Prompt 2: Restore this image with enhanced clarity and sharpness. Reduce noise, fix compression artifacts, and improve detail while strictly preserving the original content, style, and semantics. Prompt 3: Clean and restore the image by removing artifacts, noise, and blur. Maintain full fidelity to the original composition and avoid creating new visual elements. Prompt 4: Improve this image’s perceptual quality: refine edges, correct degradation, and enhance fine details. Do not alter identity, expression, or any semantic attributes.

Panel of Figure 19 — Figure 19: Distribution of different scores across various semantic scene groups. The horizontal width of each box indicates the percentage of samples within each score interval. The red line indicates the balance point; scores above it ( Over $\uparrow$ ) denote over-generation, and scores below it ( Less $\downarrow$ ) denote under-generation. The light gray region indicates low overall scores, representing generally unacceptable results.

Panel of Figure 20 — Figure 20: Distribution of different scores across various degradation types. The horizontal width of each box indicates the percentage of samples within each score interval. The light gray region indicates low overall scores, representing generally unacceptable results. The red line indicates the balance point; scores above it ( Over $\uparrow$ ) denote over-generation, and scores below it ( Less $\downarrow$ ) denote under-generation.

Panel of Figure 21 — Figure 21: Effect of parameter configurations on restoration behavior in diffusion-based models. Zoom in for a better view.

Panel of Figure 22 — Figure 22: Effect of parameter configurations on restoration behavior in diffusion-based models. Zoom in for a better view.

Panel of Figure 23 — Figure 23: Qualitative analysis of image quality assessment methods. Zoom in for a better view.

Panel of Figure 24 — Figure 24: Qualitative analysis of image quality assessment methods. Zoom in for a better view.

Panel of Figure 25 — Figure 25: Qualitative analysis of image quality assessment methods. Zoom in for a better view.

Panel of Figure 26 — Figure 26: Failure cases of semantic errors. Zoom in for a better view.

Panel of Figure 27 — Figure 27: Failure cases of semantic errors. Zoom in for a better view.

Panel of Figure 28 — Figure 28: Failure cases of over-generation. Zoom in for a better view.

Panel of Figure 29 — Figure 29: Failure cases of hard degradations. Zoom in for a better view.

Explore similar work

GGT-100K: Generative Ground Truth for Generalizable Real-World Image Restoration
May 29, 2026Xiangtao Kong, Jixin Zhao, Lingchen Sun +2Synthetic Data Augmentation Image Restoration
The Hong Kong Polytechnic University · OPPO Research Institute
Can Nano Banana 2 Replace Traditional Image Restoration Models? An Evaluation of Its Performance on Image Restoration Tasks
Apr 3, 2026Weixiong Sun, Xiang Yin, Chao DongImage Restoration All-in-One Image Restoration
Shenzhen University of Advanced Technology · Fudan University · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences
LL-Bench: Rethinking Low-Level Vision Evaluation in the Era of Large-Scale Generative Models
Jun 1, 2026Lu Liu, Huiyu Duan, Chenxin Zhu +6Image Restoration Image Quality Assessment
Shanghai Jiao Tong University, Shanghai, China

May 29, 2026J/K move · Enter open · S save

GGT-100K: Generative Ground Truth for Generalizable Real-World Image Restoration

Xiangtao Kong, Jixin Zhao, Lingchen Sun, Rongyuan Wu+1

The Hong Kong Polytechnic University · OPPO Research Institute

Synthetic Data Augmentation Image Restoration Multimodal Foundation Models

Open paper AskPDF arXiv

Real-world image restoration (IR) is bottlenecked by the scarcity of high-quality paired training data. Synthetic datasets are abundant but often fail to model real-world degradations, while real-world paired datasets are expensive and difficult to capture. As a result, IR models trained on these datasets show limited generalization in real-world scenarios. In this work, we propose Generative Ground Truth (GGT) by using generative multimodal foundation models (MFMs) to produce high-quality (HQ) targets from real-world low-quality (LQ) images. We first conduct a systematic evaluation of nine state-of-the-art MFMs, including Nano-Banana-2 and GPT-Image-2, on images of various scenes and degradation types. The results demonstrate that Nano-Banana-2 with VLM-based adaptive prompting shows the highest capability to synthesize perceptually realistic and content-faithful HQ targets, which can serve as the GGT for the LQ input. We then employ Nano-Banana-2 to build a GGT synthesis pipeline, which involves multi-stage quality control to ensure data reliability, and construct GGT-100K, an LQ-HQ paired dataset comprising 103,707 training pairs and covering diverse scenes and complex real-world degradations. A test set of 500 image pairs is also established. Extensive experiments show that GGT-100K consistently improves the real-world generalization of a wide range of IR models, with particularly strong benefits for finetuning generative models for IR tasks. Our results suggest that MFMs can serve as practical tools for restoration-oriented data generation, and GGT-100K is a useful resource to expand the generalization boundaries of real-world IR models.

Abstract

Explore similar work

GGT-100K: Generative Ground Truth for Generalizable Real-World Image Restoration

Can Nano Banana 2 Replace Traditional Image Restoration Models? An Evaluation of Its Performance on Image Restoration Tasks

LL-Bench: Rethinking Low-Level Vision Evaluation in the Era of Large-Scale Generative Models