DiNAT-IR: Exploring Dilated Neighborhood Attention for High-Quality Image Restoration
Organizations: Texas A&M University
Abstract
Transformers, with their self-attention mechanisms for modeling long-range dependencies, have become a dominant paradigm in image restoration tasks. However, the high computational cost of self-attention limits scalability to high-resolution images, making efficiency-quality trade-offs a key research focus. To address this, Restormer employs channel-wise self-attention, which computes attention across channels instead of spatial dimensions. While effective, this approach may overlook localized artifacts that are crucial for high-quality image restoration. To bridge this gap, we explore Dilated Neighborhood Attention (DiNA) as a promising alternative, inspired by its success in high-level vision tasks. DiNA balances global context and local precision by integrating sliding-window attention with mixed dilation factors, effectively expanding the receptive field without excessive overhead. However, our preliminary experiments indicate that directly applying this global-local design to the classic deblurring task hinders accurate visual restoration, primarily due to the constrained global context understanding within local attention. To address this, we introduce a channel-aware module that complements local attention, effectively integrating global context without sacrificing pixel-level precision. The proposed DiNAT-IR, a Transformer-based architecture specifically designed for image restoration, achieves competitive results across multiple benchmarks, offering a high-quality solution for diverse low-level computer vision problems.
Figures & tables
| Method | GoPro | HIDE | Model Complexity | |||
| PSNR ↑ | SSIM ↑ | PSNR ↑ | SSIM ↑ | Params (M) ↓ | FLOPs (G) ↓ | |
| SRN 2018 | 30.26 | 0.934 | 28.36 | 0.904 | 3.76 | 35.87 |
| DBGAN 2020 | 31.10 | - | 28.94 | - | 11.59 | 379.92 |
| MT-RNN 2020 | 31.15 | - | 29.15 | - | 2.64 | 13.72 |
| DMPHN 2022 | 31.20 | - | 29.09 | - | 86.80 | - |
| CODE 2023 | 31.94 | - | 29.67 | - | 12.18 | 22.52 |
| Method | Indoor Scenes | Outdoor Scenes | Combined | ||||||
| PSNR ↑ | SSIM ↑ | MAE ↓ | PSNR ↑ | SSIM ↑ | MAE ↓ | PSNR ↑ | SSIM ↑ | MAE ↓ | |
| EBDB S 2017 | 25.77 | 0.772 | 0.040 | 21.25 | 0.599 | 0.058 | 23.45 | 0.683 | 0.049 |
| DMENet S 2019 | 25.50 | 0.788 | 0.038 | 21.43 | 0.644 | 0.063 | 23.41 | 0.714 | 0.051 |
| JNB S 2015 | 26.73 | 0.828 | 0.031 | 21.10 | 0.608 | 0.064 | 23.84 | 0.715 | 0.048 |
| DPDNet S 2020 | 26.54 | 0.816 | 0.031 | 22.25 | 0.682 | 0.056 | 24.34 | 0.747 | 0.044 |
| KPAC S 2021 | 27.97 | 0.852 | 0.026 | 22.62 | 0.701 | 0.053 | 25.22 | 0.774 | 0.040 |
| Method | Rain100H | Rain100L | Test2800 | Test1200 | Test100 | |||||
| PSNR ↑ | SSIM ↑ | PSNR ↑ | SSIM ↑ | PSNR ↑ | SSIM ↑ | PSNR ↑ | SSIM ↑ | PSNR ↑ | SSIM ↑ | |
| SEMI 2019 | 16.56 | 0.486 | 25.03 | 0.842 | 24.43 | 0.782 | 26.05 | 0.822 | 22.35 | 0.788 |
| DIDMDN 2018 | 17.35 | 0.524 | 25.23 | 0.741 | 28.13 | 0.867 | 29.65 | 0.901 | 22.56 | 0.818 |
| UMRL 2019 | 26.01 | 0.832 | 29.18 | 0.923 | 29.97 | 0.905 | 30.55 | 0.910 | 24.41 | 0.829 |
| RESCAN 2018 | 26.36 | 0.786 | 29.80 | 0.881 | 31.29 | 0.904 | 30.51 | 0.882 | 25.00 | 0.835 |
| PreNet 2019 | 26.77 | 0.858 | 32.44 | 0.950 | 31.75 | 0.916 | 31.36 | 0.911 | 24.81 | 0.851 |
| Method | DnCNN | BM3D | VDN | MIRNet | MPRNet | DAGL | Uformer | Restormer | MambaIR | DiNAT-IR |
| 2017a | 2007 | 2019 | 2020 | 2021 | 2021 | 2022 | 2022 | 2024 | (Ours) | |
| PSNR ↑ | 23.66 | 25.65 | 39.28 | 39.72 | 39.71 | 38.94 | 39.89 | 40.02 | 39.89 | 39.89 |
| SSIM ↑ | 0.583 | 0.685 | 0.956 | 0.959 | 0.958 | 0.953 | 0.960 | 0.960 | 0.960 | 0.960 |
| Networks | Distortion | Perception | Params | MACs | ||
| PSNR ↑ | SSIM ↑ | FID ↓ | LPIPS ↓ | (M) ↓ | (G) ↓ | |
| Restormer (baseline) | 30.32 | 0.934 | 15.16 | 0.137 | 3.0 | 17.30 |
| NA w/o CAM | 31.56 | 0.948 | 11.12 | 0.111 | 3.0 | 16.64 |
| NA w/ CAM | 31.97 | 0.952 | 10.86 | 0.105 | 3.0 | 16.71 |
| DiNA w/o CAM | 32.03 | 0.953 | 10.17 | 0.103 | 3.0 | 16.64 |
| DiNA w/ CAM | 32.06 | 0.952 | 10.14 | 0.103 | 3.0 | 16.71 |
Image Restoration
Image restoration uses computational methods to recover useful visual content from noisy, blurred, low-resolution, or otherwise degraded images.
See all 395 papers