Total Parameters Trainable Parameters Non-trainable Parameters 5,416,159 (20.66 MB) 5,414,751 (20.66 MB) 1,408 (5.50 KB) 3.4.5 Architecture of Deblurring Network ()
In this work, one of our major contributions was the introduction of SMFD-UNet (Semantic Mask Fusion Deblurring UNet), a lightweight architecture designed to deblur a blurry and noisy
49
facial image. This model introduced a revolutionary way to take assistance of the semantic face mask to deblur the blurry and noisy image. The model architecture is shown in Figure 3.8
Figure 3.8: SMFD Model Architecture
The SMFD-UNet was also built very lighter compared to existing deblurring models. Table 3.2 shows the parameter counts of this model.
Table 3.2: SMFD-UNet Parameter Summary
() Parameter Type Count (Size) () Total Parameters Trainable Parameters Non-trainable Parameters 7,532,601 (28.73 MB) 7,531,641 (28.73 MB) 960 (3.75 KB) ()
50
3.4.5.1 Encoder
The model consisted of two encoder branches: one for processing the semantic face mask input and another for the input image. With feature extraction using Residual Dense Convolution Blocks (RDC) both branches had a similar architecture. Every encoder branch consisted of four convolution and pooling steps. The input is first run through an RDB to extract hierarchical features, then max-pooling helps to shrink spatial dimensions at every level. Concatenated by stacking one over another on the channel axis, outputs of matching stages from both branches were fused at different stages, therefore allowing the model to efficiently employ both visual and semantic aspects.
3.4.5.2 Decoder
From the fused feature maps, the decoder rebuilded the high-resolution output. It uses Attention Upsample blocks to combine upsampling via the PixelShuffle technique with attention meth-ods to hone skip connections. Using Convolutional Block Attention Module (CBAM) , the upsampled features were combined with skip connections from the encoder so stressing signif-icant features. An RDB followed every upsampling stage to improve the features even further. This progressive upsampling and refining technique guaranteed the structural information’s and fine detail’s recovery.
3.4.5.3 Final Layers
A sequence of convolutions and post-processing techniques made up the latter levels of the model. To improve feature representations, the decoder’s output was passed via a CBAM layer and a last RDB. The segmentation result is generated by then using a 1 × 1 convolution. Another 1 × 1 convolution helped to improve this output; next, a post-processing phase modified the resulting image’s brightness and contrast. The last result was a polished, high-quality image fit for jobs such segmentation or deblurring of facial images.
3.4.6 Experimental Setup
We proposed two architectures to deblur facial images holistically in this work. A blurry facial image is used to construct a semantic face mask in the first architecture. The second archi-tecture, the Semantic Mask-Guided face Deblurring (SMFD) model, creates a high-quality de-
51
blurred face image from the blurry image and the semantic mask. We trained our main de-blurring model, SMFD, using dataset ground-truth semantic face masks and our semantic mask generating model masks to objectively evaluate their efficacy. This dual-training strategy helps us assess the SMFD model’s semantic input quality reliability and performance. Though the SMFD deblurring model was trained on 2,000 samples, we trained the semantic mask generat-ing model on 3,000 samples. Despite differences in assessment metrics and loss functions for semantic mask prediction and picture deblurring, both models’ experimental setups are similar. These distinctions ensure that each model is examined according to its goals, optimizing each model and assessing our suggested structure.
3.4.6.1 Real-time Data Augmentation
Data augmentation was necessary to generalize the model. It also helped to reduce overfitting. Our real-time data augmentation included geometric transformations such as random rotations (up to ± 30 ◦ ), random horizontal flips, and random crops (0.8x to 1.0x). It also included a few photometric augmentations, such as brightness adjustments with a factor of up to 0.7 to 1.3, contrast variations up to a factor of 0.7 to 1.3.
3.4.6.2 Cross-Validation
Five-fold cross-validation verified our model’s reliability. The dataset was initially split 80:20 into training/validation and testing sets to properly split each data type. Five equal-sized folds comprised the training/validation dataset. Every cycle, one fold was validated and the model was trained using the other four. Five repeats of this stage ensured each fold was validated once. Each fold received 50 epochs of training in a 16-fold batch. Our training technique re-duced overfitting by using many data sources. The model’s efficacy was assessed by combining performance assessments from each fold.
3.4.6.3 Hyperparameter Tuning
A little bit different hyperparameters were used for the Mask Generator and the deblurring SMFD-UNet model. These hyperparameters were set by being influenced by existing works on this task and through a trial-and-error mechanism. We adjusted several hyperparameters to reach optimal performance and training economy. The learning rate was first set to 0.001, same for both models, and a method of decrease was followed. The validation Dice coefficient
52
for Mask Generator and validation SSIM for SMFD-UNet were constant for five consecutive epochs, the learning rate was reduced to 20% of its current value. This approach continued until the learning rate dropped to a minimum of 1 × 10 − 9. An early stopping condition was developed to prevent overfitting; it halted training when the validation Dice coefficient for Mask Generator and validation SSIM for SMFD-UNet did not improve for ten consecutive epochs. Moreover, model checkpointing was used during training to save the weights of the model when the validation Dice coefficient for Mask Generator and the validation SSIM for SMFD-UNet increase, thereby preserving the optimal model configuration for the last assessment.
3.4.6.4 Evaluation Metrics and Loss Function
Different evaluation metrics and loss functions were used for the two models. These evaluation metrics and loss functions were described in detail in chapter 2.
Semantic Mask Generator Model’s Evaluation Metrics and Loss Function: We eval-uated mask generator model using the Dice Coefficient and Jaccard Index. The loss function consisted in dice loss. Every metric was computed channel wise independently before a mean value was derived.
De-blurring SMFD Model’s Evaluation Metrics and Loss Function: As assessment measures we applied PSNR, SSIM, LPIPS, MSE, NIQE, FID. PSNR assesses output pixel-wise accuracy while SSIM gauges output structural similarity with reference to the genuine value. We employed mean square error as our loss function for this work.
3.5 Conclusion
Using the CelebA dataset with synthetically generated blur through a randomized pipeline pro-ducing 1.74 trillion permutations, this chapter described the proposed technique for face picture deblurring. Combining blurry pictures with semantic face masks, preprocessed to 256x256x3 with simplified 5-value masks, the proposed SMFD-UNet is a dual-input U-Net-based model. Important elements include Residual Dense Convolution Blocks, PixelShuffle Upsample, At-tention Upsample, and CBAM for efficient feature extraction and refining; then, post-processing improves visual quality. This method addresses identity retention and texture shortage, there-fore laying a good basis for experimental analysis. Performance evaluation of this technique will be covered in the future chapter.
53
Chapter 4
Result Analysis and Discussion
4.1 Introduction
This chapter provides a comprehensive evaluation of the proposed Semantic Mask Fusion De-blurring UNet model, which reconstructs facial images with semantic face masks generated directly from image blurriness. The performance was evaluated using peak signal-to-noise ra-tio (PSNR) and structural similarity index (SSIM), as well as qualitative visual comparisons to state-of-the-art algorithms such as CodeFormer, RestoreFormer, RestoreFormer++, and GFP-GAN. The examination also includes an ablation study to emphasize the contributions of key components like the semantic mask branch and attention-based upsampling. Training valida-tion curves and real-world deblurring results were also examined in order to demonstrate model generalization and strength. The results demonstrate how successfully the lightweight design provides remarkable structural similarity and competitive naturalism, overcoming the concerns of face image deblurring raised in previous chapters.
4.2 Results Analysis
4.2.1 Result Obtained by Our Model
Our approach mostly depended on the acquisition of ideal semantic face mask. Still, we sug-gested a technique to create face masks from the blurry pictures. Still, the replica differs from the original in some ways. To deblur facial photos, we thus have compared our deblurring technique with produced masks and original masks provided in the dataset.
54
4.2.1.1 Result in Generating Semantic Face Mask Our semantic mask generation was doing pretty good. It achieved satisfactory dice co-efficeint of 64.84% and jaccard index of 55.72%. The sores were not that much good. This was due to the severe blurriness that was causing misinterpretation of a few portions. A visual representation of our score is given in Figure 4.1.
Figure 4.1: A Visual Representation of Our Result in Generating Semantic Face Masks
A comparative study was done with two state-of-the-art models for segmentation, UNet and UNet++ [54]. Table 4.1 shows the performance of each model along with its parameters. This table shows that our model, although it achieved lower scores, is computationally more efficient, which aligned more closely with our objective of making the architecture lightweight. Also, the differences in scores were negligible. Table 4.1: Quantitative Comparison of Our Model with Recent State-of-the-art Segmentation Models
() Model Dice ↑ Jaccard ↑ Parameters ↓ () UNet 67.21% 58.00% 31,401,605 UNet++ 68.20% 58.74% 36,643,333 Ours 64.84% 55.72% 5,416,159 4.2.1.2 Result in Deblurring Facial Images ()
Using Peak Signal-to- Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM), the performance of the proposed Semantic Mask Fusion Deblurring UNet was assessed against
55
many state-of- the-art techniques including CodeFormer, RestoreFormer, RestoreFormer++, and GFPGAN. The findings, collected in Table 4.2, show how well our model restores noisy and blurry facial images guided by semantic face masks and multi-stage feature fusion. Table 4.2: Quantitative Comparison of Our Model with Recent State-of-the-art Deblurring and Restoration Models by Zero-shot Prediction
() Model (Publisher, Year) PSNR ↑ SSIM ↑ MSE ↓ NIQE ↓ LPIPS ↓ FID ↓ () CodeFormer (NeurIPS, 2022) 22.59 61.80% 417.59 0.2565 0.3211 119.31 RestoreFormer (CVPR, 2022) 22.91 63.28% 388.53 0.2613 0.3450 129.25 RestoreFormer++ (TPAMI, 2023) 22.88 62.65% 395.52 0.2599 0.2368 89.70 GFPGAN (CVPR, 2021) 22.99 63.70% 381.64 0.2644 0.2295 97.42 Ours with Original Mask 24.37 70.81% 330.19 0.2664 0.2933 110.45 Ours with Generated Masks 23.82 69.34% 390.07 0.2682 0.3251 123.54 ()
As we suggested a dual input model for deblur facial photographs. Among its inputs is semantic facial mask. We therefore also suggested a semantic face mask generating model from blurry images. We also examined result using original face mask to ignore the dependence with mask generator model, so verifying the deblurring model how good it is. Figure 4.2 shows a visual comparison of our model with the state-of- the modern mod-els. These findings show the benefit of including semantic face mask guidance with dual-input multi-stage feature fusion in order to enable more exact reconstruction of facial details and enhanced perceptual quality.
Figure 4.2: A Visual Comparison of Our Result with Other Recent State-of-the-art Models by Their Zero-shot Prediction
56
A few real-world blurry photographs, images containing motion blur mainly, were taken of a famous actor, Rowan Atkinson, from the sets of a Hollywood movie named ”Johnny En-glish” [55]. The results of deblurring these photographs using our suggested model and other state-of-the-art models are provided in Figure 4.3. These samples illustrate that our models’deblurred photos were not so clear, but it has improved the structural likeness of the human. Other state-of-the-art models deblurred photos are sharp, but they are synthetic images where structural dissimilarity may be found, and there is also a change of facial expression. As for these real-world blurry images, no ground truth was present, hence no quantitative compari-son was possible. So, a qualitative evaluation was performed by taking a survey among a few people, and their votes also match the above explanations.
Figure 4.3: Deblurring Real-world Blurry Images
The enhancements in PSNR and SSIM emphasize the model’s ability to effectively reduce noise and recover fine facial features, proving the efficacy of the suggested architecture in ad-dressing the issues of facial deblurring with a lighter architecture. Lighter because existing works like GFPGAN, RestoreFormer, RestoreFormer++, CodeFormer, and others are mainly GAN, Diffusion, or Transformers, and they are typically heavier than CNNs and need a huge collection of data to be well trained, and also a huge amount of time. Additionally, our proposed model was trained well with a smaller dataset, and performance across real-world blurriness proved in favor of this.
In Figure 4.4, the training and validation curves of the training time of the deblurring SMFD-UNet model are shown. From the curves, it is clear that the model is well-trained. Since there are no significant gaps between the training and validation curves, the SSIM validation curve is superior to the training curve, supporting the model’s generalization capability. Addition-ally, both training and validation scores are satisfactory, indicating no issues with overfitting
57
or underfitting. Additionally, the smooth, incremental curves reflect the optimal selection of hyperparameters.
Figure 4.4: Training and Validation Curves
4.2.2 Ablation Study An ablation study was performed by methodically changing the design and monitoring the per-formance using Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measures (SSIM) in order to assess the influence of every component in our suggested model. Table 4.3 shows the outcomes. Comprising the backbone devoid of the semantic mask branch, the baseline configuration obtained an SSIM of 66.25% and a PSNR of 22.46. By including the semantic mask branch with conventional upsampling, performance was much enhanced, and a PSNR of 23.91 and an SSIM of 69.22% resulted. This shows how much semantic direction helps to improve reconstruction quality. Table 4.3: Performance Comparison of Different Configurations Based on PSNR and SSIM Metrics
() Configuration PSNR ↑ SSIM ↑ () Backbone without Semantic Mask Branch 22.46 66.25% Backbone + Semantic Mask Branch + Traditional Upsampling 23.91 69.22% Backbone + Semantic Mask Branch + Attention Upsampling (Transpose) 23.89 70.20% Backbone + Semantic Mask Branch + Attention Upsampling (Pixel Shuffle) 24.14 70.42% Backbone + Semantic Mask Branch + Attention Upsampling + Post Processing Block 24.37 70.81% ()
58
Replacing conventional upsampling with attention-based upsampling also showed further results. While pixel shuffle-based attention upsampling somewhat improved performance to a PSNR of 24.14 and an SSIM of 70.42%, transpose-based attention upsampling produced a PSNR of 23.89 and an SSIM of 70.20%. These results emphasize how well attention processes capture fine-grained features during upsampling.
Adding a post-processing block to the attention upsampling (pixel shuffle) setup achieved optimal performance (PSNR 24.37, SSIM 70.81%). The ablation study shows that the se-mantic mask branch, attention-based upsampling, and post-processing block each significantly contribute to the model’s performance, with their combined effect yielding the best results.
A layer-wise feature map visualization has been shown for the Mask Generator model in Figure 4.5 and for SMFD-UNet in Figure 4.6. As feature maps could be multi-channel, proper understanding is not possible, though it gives a little understanding of how the proposed archi-tectures reach their destination. For channels like 3, the output is plotted as an RGB image; for 5 channels, anti-one-hot encoding reduces it to a single channel; otherwise, only the first channel is displayed. The outputs in both figures changed over time because of the architecture’s de-sign, which uses Residual Dense Connection (RDC) layers to keep and improve features across layers, which helped keep more detail. Max pooling kept important information while reducing spatial dimensions, and attention up-sampling made relevant features stronger, which helped the network produce useful outputs like semantic masks or images that have been deblurred. The model learned more complex patterns as the channel depth slowly increased (for example, from 64 to 256). The fusion stages in the deblurring process combined semantic guidance with image data, which made the images clearer. The network’s ability to learn hierarchical features can be seen by the consistent improvement in output quality, which went from blurry to sharp or masked to accurate.
59
Figure 4.5: Layer-wise Feature Map Visualization for the Mask Generator Model
60
Figure 4.6: Layer-wise Feature Map Visualization for the SMFD-UNet
61
4.2.3 Cross-Validation Scores At the end, to evaluate the performance of the proposed model, we conducted a 5-fold cross-validation experiment, shown in Table 4.4. Two key metrics were used to assess the quality of the reconstructed images: Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). This proves the generalization of our model across diverse kinds of blurry inputs.
Table 4.4: PSNR and SSIM values across folds with mean and standard deviation
Fold 1 24.4046 70.61 Fold 2 24.1845 69.50 Fold 3 24.7568 71.22 Fold 4 24.2840 70.27 Fold 5 24.7137 71.05 Mean ± SD 24.4687 ± 0.2289 70.53 ± 0.61 4.3 Discussion () Our model outperformed all other state-of-the-art restoration models in terms of SSIM and PSNR. Our model’s mean square error was superior for deblurring with the original face mask, but it lagged behind the MSE of GFPGANs and RestoreFormer when using the generated face mask. Our model lags behind for every other metric. This is because our model concentrated on structural similarity, while other models prioritized naturalness and sharpness. Other mod-els likely achieved better naturalness due to their emphasis on generative techniques, such as those in GFPGAN and RestoreFormer, which prioritize texture synthesis and sharpness through adversarial training or advanced loss functions like perceptual loss. These approaches enhance visual appeal by generating finer details, often at the cost of structural fidelity, as they may overfit to statistical patterns rather than preserving original structural elements like identity traits and expressions. On the other hand, identity traits and expressions were preserved in our model’s deblurred images, despite the fact that they were less crisp and natural. First, the ro-bust multi-stage fusion technique, which combined the semantic masks with the blurry input in the encoder to combine features at each stage, was the reason for our model’s performance. 62 The RDC block then continued to have a significant influence on both the mask generation and deblurring processes. It is made up of residual connections and dense convolutions. Dense convolution enabled feature reuse and better information flow by feed-forward connecting each layer to every one after it. Dense convolution concatenated all previous layer outputs as input, improving gradient flow and minimizing vanishing gradient problems in contrast to traditional convolution, which only receives input from the previous layer. As a result, deeper networks with fewer parameters were made possible by more effective parameter usage. By addressing vanishing gradient problems and enhancing feature propagation, residual connections made it simpler to train deep networks by adding a layer’s input straight to its output. Better upsampling with fewer parameters was made possible by pixel-shuffle upsampling. In order to improve fea-ture quality, CBAM used channel and spatial attention to highlight significant high-resolution features and suppress those that weren’t relevant. This CBAM feature improved our model’s upsampling process when combined with Pixel-shuffle upsampling. The final post-processing block enhanced the perceived sharpness and visual quality. 4.4 Conclusion Validated in this chapter, the Semantic Mask Fusion Deblurring UNet model outperformed state-of-the-art methods, including CodeFormer and GFPGAN, with a PSNR of 24.4687 ± 0.2289 and SSIM of 70.53 ± 0.61 % using natural masks. A Dice coefficient of 64.84% and Jaccard index of 55.72% were obtained by the semantic mask generating model. The ablation in-vestigation underlined the relevance of the post-processing block, attention-based upsampling, and semantic mask branch. These results show how successfully the model repairs blurry facial images with competitive naturalness and enhanced structural similarity. This thesis book will draw to a close in the next chapter. 63 Chapter 5 Conclusion and Future Scopes 5.1 Introduction The main results and applications of the Semantic Mask Fusion Deblurring UNet (SMFD-UNet) model for face image deblurring are compiled in this chapter. It presents a thorough overview of the thesis results, emphasizes the need of the suggested approach, and lists possible future topics of research. The chapter ends by considering the whole influence of the work in advancing facial image restoration and its uses in computer vision. 5.2 Thesis Conclusion and Summarization Using semantic face masks to guide the restoration process, this thesis presented SMFD-UNet, a novel lightweight framework meant to solve important problems in facial image deblurring. The work effectively addressed constraints in current literature, including dependence on high-quality reference images, computational inefficiencies, and the failure to preserve identity under strong blur circumstances. The main results and contributions of this effort are compiled as follows: • Novel Deblurring Framework : Using original masks, SMFD-UNet achieved excep-tional performance with a Peak Signal-to-Noise Ratio (PSNR) of 24.4687 ± 0.2289 and Structural Similarity Index Measure (SSIM) of 70.53 ± 0.61 % by combining blurry im-ages with semantic face masks under a dual-input architecture, surpassing existing works. • Reference-Free Semantic Mask Generation : A UNet-based semantic mask genera- 64 tor was built to directly extract facial component masks (e.g., eyes, nose, mouth) straight from blurry images, obtaining a Dice coefficient of 64.84% and a Jaccard index of 55.72%. This solved a main restriction in previous semantic-guided methods by removing the re-quirement for clear reference images. • Robust Blurring Pipeline : By use of combinations of Gaussian blur, motion blur, reso-lution degradation, and noise addition, a randomized blurring pipeline was presented to simulate almost 1.74 trillion deterioration scenarios. This guaranteed the resilience of the model to certain real-world blur circumstances. • Lightweight and Efficient Architecture : The model is light and efficient, compared to existing state-of-the-art models. The Mask Generating model has a parameter count of 5,416,159, and the SMFD-UNet has a parameter count of 7,532,601. Residual Dense Convolution Blocks (RDC), PixelShuffle Upsampling, and Convolutional Block Atten-tion Modules (CBAM) produce a computationally efficient model fit for low-resource environments. • Multi-Stage Feature Fusion : To improve the reconstruction of fine facial characteristics and maintain structural integrity, a unique multi-stage fusion technique was presented to merge semantic and visual aspects at several scales. The ablation investigation verified the important part each component— semantic mask branch, attention-based upsampling, and post-processing block—plays in obtaining high-quality deblurring results. Tests using real-world blurry photos proved the generalizability of the pro-posed model in real-world settings. SMFD-UNet excels in pixel-wise and structural similarity yet unlike rival models that generate crisper but synthetic outputs with changed facial expres-sions retains competitive naturalness measures (NIQE, LPIPS, FID). This work offered a strong, reference-free, and effective approach, therefore advancing the field of face image deblurring. Through its lightweight architecture and great performance, its applications encompass facial recognition, forensic analysis, photographic improvement, and medical imaging, thus providing both academic and industry value. 65 5.3 Evaluation of Hypotheses This study was guided by four hypotheses concerning the efficacy of a lightweight, reference-free model for deblurring heavily distorted facial images using semantic masks. The experi-mental results are evaluated below to assess the extent to which each hypothesis was supported. H1: Semantic masks generated from blurry images can effectively guide identity-preserving deblurring. The results moderately support this hypothesis. Semantic masks, derived directly from blurry input images, moderately guided the deblurring process by pre-serving critical facial features such as eyes, nose, and mouth contours. The Dice and Jaccard scores were not satisfactory; that’s why it is called moderate. H2: Artificial uniform blurriness can mimic real-world nonuniform blurriness to a satisfactory level. This hypothesis was partially supported. As was tested on real-world blurry images, the outputs were good but not perfect, and obviously retrieval of the original image is never possible. H3: Reference-free semantic deblurring performs competitively with reference-based methods. The findings largely support this hypothesis. The proposed lightweight CNN ar-chitecture was trained and tested using both original semantic masks and generated semantic masks derived from blurry input images, with both configurations yielding closely comparable performance. H4: Lightweight CNN architectures can achieve high deblurring quality with lower computational costs. This hypothesis was fully supported by the experimental results. The proposed lightweight CNN architecture achieved superior deblurring performance compared to state-of-the-art transformer-based and GAN-based models, as evidenced by quantitative metrics and qualitative evaluations. 5.4 Future Scopes The results of this work provide several interesting research paths and provide means for several improvements to further deblurring of facial images and their uses: • Keep the Same Color as Input : Proposed model’s output was struggling to keep the same color as the input image. Future works can focus on this. 66 • Unwanted Colourful Pixels in Background : Proposed model sometimes failed to cor- rectly output the background and created these unwanted colorful pixels. • Enhanced Semantic Mask Generation : The semantic mask generator, with a Dice co- efficient of 64.84%, could be improved. • Expanded Degradation Models : The current blurring pipeline covers a vast range of scenarios, but future work could incorporate additional degradation types, such as occlu-sions, complex lighting variations, or atmospheric distortions, to further enhance model generalization. • Larger and More Diverse Real-World Datasets : Training and testing on larger, more diverse real-world datasets could improve the model’s generalization, particularly for underrepresented facial features or complex real-world scenarios. These directions aim to build on the strengths of SMFD-UNet, addressing remaining chal-lenges and expanding its applicability across diverse domains and use cases. 5.5 Conclusion This chapter gave a thorough overview of the contributions of the SMFD-UNet model to fa-cial image deblurring, stressing its better performance in PSNR (24.4687 ± 0.2289) and SSIM (70.53 ± 0.61 %), reference-free semantic mask generation, and strong blurring pipeline. Im-proved by multi-stage feature fusion and attention techniques, the lightweight design provides a sensible answer for practical uses. Future scholars can focus on enhancing mask generation, handling various degradations, and optimizing for real-time applications, thereby strengthening the impact of the model. With great possibilities for both academic study and commercial use in computer vision, this work lays a strong basis for improving facial image restoration. 67 REFERENCES [1] R. Jones, “Blur degradation in imaging systems: A review,” Journal of Imaging Science , vol. 59, no. 3, pp. 123–135, 2015. [2] C.-H. Lee, Z. Liu, L. Wu, and P. Luo, “Maskgan: Towards diverse and interactive facial image manipulation,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2020. [3] K. Grm, W. J. Scheirer, and V. Štruc, “Face hallucination using cascaded super-resolution and identity priors,” IEEE Transactions on Image Processing , vol. 29, pp. 2150–2165, 2019. [4] K. Zhang, W. Ren, W. Luo, W.-S. Lai, B. Stenger, M.-H. Yang, and H. Li, “Deep im-age deblurring: A survey,” International Journal of Computer Vision , vol. 130, no. 9, pp. 2103–2130, 2022. [5] L. Xu, J. S. Ren, C. Liu, and J. Jia, “Deep convolutional neural network for image decon- volution,” Advances in neural information processing systems , vol. 27, 2014. [6] M. Munsif, M. Sajjad, M. Ullah, A. N. Tarekegn, F. A. Cheikh, P. Tsakanikas, and K. Muhammad, “Optimized efficient attention-based network for facial expressions anal-ysis in neurological health care,” Computers in Biology and Medicine , vol. 179, p. 108822, 2024. [7] A. Levin, Y. Weiss, F. Durand, and W. T. Freeman, “Understanding and evaluating blind deconvolution algorithms,” IEEE Conference on Computer Vision and Pattern Recogni-tion , pp. 1964–1971, 2009. [8] B. Wang, F. Xu, and Q. Zheng, “A survey on facial image deblurring,” Computational Visual Media , vol. 10, pp. 3–25, Feb 2024. 68 [9] X. Wang, Y. Li, H. Zhang, and Y. Shan, “Towards real-world blind face restoration with generative facial prior,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pp. 9168–9178, 2021. [10] T. Varanka, T. Toivonen, S. Tripathy, G. Zhao, and E. Acar, “Pfstorer: Personalized face restoration and super-resolution,” in Proceedings of the IEEE/CVF Conference on Com-puter Vision and Pattern Recognition , pp. 2372–2381, 2024. [11] Z. Wang, J. Zhang, R. Chen, W. Wang, and P. Luo, “Restoreformer: High-quality blind face restoration from undegraded key-value pairs,” in Proceedings of the IEEE/CVF con-ference on computer vision and pattern recognition , pp. 17512–17521, 2022. [12] Z. Wang, J. Zhang, T. Chen, W. Wang, and P. Luo, “Restoreformer++: Towards real-world blind face restoration from undegraded key-value pairs,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 12, pp. 15462–15476, 2023. [13] S. Zhou, K. Chan, C. Li, and C. C. Loy, “Towards robust blind face restoration with code-book lookup transformer,” Advances in Neural Information Processing Systems , vol. 35, pp. 30599–30611, 2022. [14] X. Li, C. Chen, S. Zhou, X. Lin, W. Zuo, and L. Zhang, “Blind face restoration via deep multi-scale component dictionaries,” in European conference on computer vision , pp. 399–415, Springer, 2020. [15] Z. Shen, W.-S. Lai, T. Xu, J. Kautz, and M.-H. Yang, “Deep semantic face deblur-ring,” in Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 8260–8269, 2018. [16] J. Pan, Z. Hu, Z. Su, and M.-H. Yang, “Deblurring face images with exemplars,” in Com-puter Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VII 13 , pp. 47–62, Springer, 2014. [17] C. Shi, X. Zhang, X. Li, I. Mumtaz, and J. Lv, “Face deblurring based on regularized struc-ture and enhanced texture information,” Complex & Intelligent Systems , vol. 10, no. 2, pp. 1769–1786, 2024. 69 [18] Y.-Q. Liu, X. Du, H.-L. Shen, and S.-J. Chen, “Estimating generalized gaussian blur ker-nels for out-of-focus image deblurring,” IEEE Transactions on circuits and systems for video technology , vol. 31, no. 3, pp. 829–843, 2020. [19] E. S. Gedraite and M. Hadad, “Investigation on the effect of a gaussian blur in image filtering and segmentation,” in Proceedings ELMAR-2011 , pp. 393–396, 2011. [20] S. Cho, Y. Matsushita, and S. Lee, “Removing non-uniform motion blur from images,” in 2007 IEEE 11th International Conference on Computer Vision , pp. 1–8, 2007. [21] L. Jiao and J. Zhao, “A survey on the new generation of deep learning in image process- ing,” IEEE Access , vol. 7, pp. 172231–172263, 2019. [22] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep con-volutional neural networks,” Advances in neural information processing systems , vol. 25, 2012. [23] T. Hey and A. Trefethen, “The data deluge: An e-science perspective,” in Grid computing: Making the global infrastructure a reality , pp. 809–824, Wiley, 2003. [24] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to doc-ument recognition,” Proceedings of the IEEE , vol. 86, no. 11, pp. 2278–2324, 2002. [25] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convo-lutional networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 4700–4708, 2017. [26] M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, Septem-ber 6-12, 2014, Proceedings, Part I 13 , pp. 818–833, Springer, 2014. [27] H. Gao, H. Yuan, Z. Wang, and S. Ji, “Pixel transposed convolutional networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 42, no. 5, pp. 1218–1227, 2019. [28] W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang,“Real-time single image and video super-resolution using an efficient sub-pixel convolu-tional neural network,” in Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 1874–1883, 2016. 70 [29] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017. [30] J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 7132–7141, 2018. [31] H. Zhao, Y. Zhang, S. Liu, J. Shi, C. C. Loy, D. Lin, and J. Jia, “Psanet: Point-wise spatial attention network for scene parsing,” in Proceedings of the European conference on computer vision (ECCV) , pp. 267–283, 2018. [32] S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, “Cbam: Convolutional block attention mod-ule,” in Proceedings of the European Conference on Computer Vision (ECCV) , pp. 3–19, 2018. [33] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in International Conference on Machine Learning , pp. 448–456, PMLR, 2015. [34] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,”in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 770–778, 2016. [35] S. R. Dubey, S. K. Singh, and B. B. Chaudhuri, “Activation functions in deep learning: A comprehensive survey and benchmark,” Neurocomputing , vol. 503, pp. 92–108, 2022. [36] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 , 2014. [37] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention– MICCAI 2015 (N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi, eds.), (Cham), pp. 234–241, Springer International Publishing, 2015. [38] R. Bokade, A. Navato, R. Ouyang, X. Jin, C. Chou, S. Ostadabbas, and A. V. Mueller, “A cross-disciplinary comparison of multimodal data fusion approaches and applications: Ac-celerating learning through trans-disciplinary information sharing,” Expert Systems With Applications , vol. 165, p. 113885, 2020. 71 [39] O. Kupyn, V. Budzan, M. Mykhailych, D. Mishkin, and J. Matas, “Deblurgan: Blind motion deblurring using conditional adversarial networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 8183–8192, 2018. [40] O. Kupyn, T. Martyniuk, J. Wu, and Z. Wang, “Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better,” in Proceedings of the IEEE/CVF international conference on computer vision , pp. 8878–8887, 2019. [41] X. Tao, H. Gao, X. Shen, J. Wang, and J. Jia, “Scale-recurrent network for deep image deblurring,” in Proceedings of the IEEE conference on computer vision and pattern recog-nition , pp. 8174–8182, 2018. [42] J. ChoSJ et al. [49] C. H. Sherman and J. L. Butler, Transducers and Arrays for Underwater Sound , vol. 4. New York: Springer, 2007. [50] Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale structural similarity for im-age quality assessment,” in The Thirty-Seventh Asilomar Conference on Signals, Systems Computers, 2003 , vol. 2, pp. 1398–1402, IEEE, November 2003. [51] A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a “completely blind” image quality analyzer,” IEEE Signal Processing Letters , vol. 20, no. 3, pp. 209–212, 2012. [52] K. Zhang, W. Zuo, and L. Zhang, “Ffdnet: Toward a fast and flexible solution for cnn-based image denoising,” IEEE Transactions on Image Processing , vol. 27, no. 9, pp. 4608–4622, 2018. [53] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” in Advances in Neural Information Processing Systems , vol. 30, Curran Associates, Inc., 2017. [54] Z. Zhou, M. M. Rahman Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: A nested u-net architecture for medical image segmentation,” in Deep Learning in Medical Image Analy-sis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4 , pp. 3–11, Springer International Publishing, 2018. [55] P. Howitt, “Johnny english,” 2003. Film. 73