UCAN: Unified Convolutional Attention Network for Expansive Receptive Fields in Lightweight Super-Resolution
Authors: Cao Thien Tan, Phan Thi Thu Trang, Do Nghiem Duc, Ho Ngoc Anh, Hanyang Zhuang, Nguyen Duc Dung
Organizations: Ho Chi Minh City Open University · AI Tech Lab, Ho Chi Minh City University of Technology · Code Mely AI Research Team · Ha Noi University of Science and Technology · University of Manitoba · Global College, Shanghai Jiao Tong University
Hybrid CNN-Transformer architectures achieve strong results in image super-resolution, but scaling attention windows or convolution kernels significantly increases computational cost, limiting deployment on resource-constrained devices. We present UCAN, a lightweight network that unifies convolution and attention to expand the effective receptive field efficiently. UCAN combines window-based spatial attention with a Hedgehog Attention mechanism to model both local texture and long-range dependencies, and introduces a distillation-based large-kernel module to preserve high-frequency structure without heavy computation. In addition, we employ cross-layer parameter sharing to further reduce complexity. On Manga109 (4×), UCAN-L achieves 31.63 dB PSNR with only 48.4G MACs, surpassing recent lightweight models. On BSDS100, UCAN attains 27.79 dB, outperforming methods with significantly larger models. Extensive experiments show that UCAN achieves a superior trade-off between accuracy, efficiency, and scalability, making it well-suited for practical high-resolution image restoration.
Figures & tables
Figure 1: Performance comparison of PSNR versus model parameters on the Manga109 ( × 4) dataset. Our method is evaluated alongside state-of-the-art super-resolution approaches. Green triangles represent CNN-based methods, yellow squares denote Transformer-based methods, blue circles indicate SSM-based methods, and red diamonds show our proposed approach.
Figure 2: Comparison of feature maps output by Linear Attention (Using ReLU, ELU feature maps) and Hedgehog Attention. All experiments are conducted based on an image with N = 256 and d = 48. The full rank of matrices in the figure is 64. This figure shows that our model outperforms previous methods in diversity.
Figure 3: Detailed architecture of (a) Shared and Received Hybrid Attention (SHA and RHA) and (b) Large Kernel Distillation (LKD). LKD contains a Triple Feature Extraction block with three branches, which are the Large Kernel Spatial Branch, the Channel Branch, and the Small Kernel Spatial Branch. SHA and RHA employ Shared and Received Window Multi Head Self Attention (Shared WMHSA and Received WMHSA) to capture local information, and include a Shared Dual Fusion Layer (SDFL) and a Dual Fusion Receiver Layer (DFRL) to aggregate global context. The Dual Fusion Layer comprises two sub branches, which are Hedgehog Attention (HgA) and Channel Attention (CA).
Set5
Set14
BSDS100
Urban100
Manga109
Method
scale
#param
MACs
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
MambaIR-light [ 10 ]
2×
905K
334.2G
38.13
0.9610
33.95
0.9208
32.31
0.9013
32.85
0.9349
39.20
0.9782
OmniSR [ 35 ]
2×
772K
194.5G
38.22
0.9613
33.98
0.9210
32.36
0.9020
33.05
0.9363
39.28
0.9784
SRFormer-light [ 46 ]
2×
853K
236.3G
38.23
0.9613
33.94
0.9209
32.36
0.9019
32.91
0.9353
39.28
0.9785
ATD-light [ 40 ]
2×
753K
380.0G
38.29
0.9616
34.10
0.9217
32.39
0.9023
33.27
0.9375
39.52
0.9789
HiT-SRF [ 42 ]
2×
847K
226.5G
38.26
0.9615
34.01
0.9214
32.37
0.9023
33.13
0.9372
39.47
0.9787
Table 1: Quantitative comparison on lightweight image super-resolution with state-of-the-art methods. The best and second-best results are shown in bold and underlined , respectively.
Figure 4: Visual comparison between ground truth and different methods on Urban100.
Figure 5: Visualize in detail ERF of MambaIR [ 10 ] , MambaIRv2 [ 8 ] and UCAN
Figure 6: Local attribution maps (LAM) comparison of different super-resolution methods at scale × 4.
Table 2: Comparison of Latency and Parameters between Naive Self-Attention, Flash Attention [ 3 ] , and the Dual Fusion Layer (DFL)
Block
Case
Set5
Urban100
PSNR
SSIM
PSNR
SSIM
High performance attention (HPA)
w/o HPA
38.27
0.9616
32.90
0.9346
window 16×16
38.32
0.9617
33.04
0.9364
WMHA
WS 8×8
38.32
0.9617
33.02
0.9361
Dual Fusion Layer (DFL)
Using ReLU
38.33
0.9618
33.16
0.9374
Using ELU + 1
38.33
0.9618
33.16
0.9373
Table 3: Ablation study. We train all models on DIV2K for 400K iterations, and test on Set5 and Urban100 ( × 2). The final result is shown in the last row.
kcore
Dilation d
kextra
Final ERF
3
1
–
5
5
1
–
9
5
2
–
13
5
3
11
47
5
3
13
53
5
3
17
65
Table 4: LKSA module configurations and resulting effective kernel sizes (1D ERF). For rows with kextra=– , we use the FLK-S formula ( 16 ); otherwise, we use the FLK-L formula ( 17 ).
Figure 7: Visualization of attention maps for Linear Attention using ReLU and ELU+1 (computed with sequence length N=256 ). The lack of normalization leads to high-magnitude artifacts.
Figure 8: Ranking consistency analysis. We compare the output ranking of Linear Attention using standard ReLU, Symmetric ReLU, and the Hedgehog Feature Map (sequence length N=256 ). While adding negative information (Sym-ReLU) improves consistency, Hedgehog achieves superior performance through learnable stability.
Case
Configuration
Set5
Urban100
PSNR
SSIM
PSNR
SSIM
Hybrid Block (HB)
depth=3
38.30
0.9617
33.05
0.9363
depth=2
38.24
0.9615
32.95
0.9353
depth=1
38.16
0.9612
32.71
0.9336
Large Kernel Distillation (LKD)
LKD depth=4
38.33
0.9618
33.12
0.9369
LKD depth=3
38.33
0.9618
33.08
0.9366
Table 5: More ablation study. We train all models on DIV2K for 400K iterations, and test on Set5 and Urban100 ( ×2 ).
Set5
Set14
BSDS100
Urban100
Manga109
Method
scale
#param
FLOPs
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
SwinIR-light [ 19 ]
2×
910K
244.2G
38.14
0.9611
33.86
0.9206
32.31
0.9012
32.76
0.9340
39.12
0.9783
ELAN [ 43 ]
2×
621K
245.2G
38.27
0.9616
33.94
0.9207
32.30
0.9012
32.76
0.9340
39.11
0.9782
IPG-Tiny [ 34 ]
2×
621K
203.1G
38.17
0.9611
34.24
0.9236
32.35
0.9018
33.04
0.9359
39.31
0.9786
PFT-light [ 23 ]
2×
776K
278.3G
38.36
0.9620
34.19
0.9232
32.43
0.9030
33.67
0.9411
39.55
0.9792
UCAN (Our)
2×
689K
146.3G
38.34
0.9618
34.27
0.9242
32.39
0.9025
33.22
0.9379
39.54
0.9790
Table 6: Quantitative comparison on lightweight image super-resolution with state-of-the-art methods. The best and second-best results are shown in bold and underlined , respectively.
Figure 9: Visual comparison between the ground truth and different methods on Set5 - baby.
Figure 10: Visual comparison between the ground truth and different methods on Set14 - man.
Figure 11: Visual comparison between the ground truth and different methods on B100 - 300091.
Figure 12: Visual comparison between the ground truth and different methods on Manga109 - Yumeko Cooking.
Figure 13: Visual comparison between the ground truth and different methods on Manga109 - Gakuen Noise.
Figure 14: Visual comparison between the ground truth and different methods on Manga109 - Yasakii Akuma.
Figure 15: Visual comparison between the ground truth and different methods on Urban100 - 015.
Figure 16: Visual comparison between the ground truth and different methods on Urban100 - img19.
Figure 17: Visual comparison between the ground truth and different methods on Urban100 - img24.
Figure 18: Visual comparison between the ground truth and different methods on Urban100 - img72.
This paper introduces a lightweight image super-resolution (SR) network, termed the Multi-scale Spatial Adaptive Attention Network (MSAAN), to address the common dilemma between high reconstruction fidelity and low model complexity in existing SR methods. The core of our approach is a novel Multi-scale Spatial Adaptive Attention Module (MSAA), designed to jointly model fine-grained local details and long-range contextual dependencies. The MSAA comprises two synergistic components: a Global Feature Modulation Module (GFM) that learns coherent texture structures through differential feature extraction, and a Multi-scale Feature Aggregation Module (MFA) that adaptively fuses features from local to global scales using pyramidal processing. To further enhance the network's capability, we propose a Local Enhancement Block (LEB) to strengthen local geometric perception and a Feature Interactive Gated Feed-Forward Module (FIGFF) to improve nonlinear representation while reducing channel redundancy. Extensive experiments on standard benchmarks (Set5, Set14, B100, Urban100, Manga109) across ×2, ×3, and ×4 scaling factors demonstrate that both our lightweight (MSAAN-light) and standard (MSAAN) versions achieve superior or competitive performance in terms of PSNR and SSIM, while maintaining significantly lower parameters and computational costs than state-of-the-art methods. Ablation studies validate the contribution of each component, and visual results show that MSAAN reconstructs sharper edges and more realistic textures.
Recently, Vision Transformer (ViT)-based models have exhibited remarkable performance in image super-resolution. However, the quadratic computational complexity of ViTs with respect to spatial resolution severely constrains their efficiency, leading to high latency and massive memory consumption. To alleviate this, various window-based attention mechanisms have been proposed; yet, they inherently compromise the long-range dependency modeling that is the primary advantage of ViTs. To overcome these limitations, we propose the Clustered Unit-level Similarity Transformer (CUST), a novel architecture that efficiently integrates global and local information. Specifically, CUST enables each patch to aggregate and attend to similar patches within a broadened regional scope outside its local window, thereby capturing extensive contextual understanding. Furthermore, it employs overlapping attention windows to capture local dependencies, while explicitly extracting high-frequency details by computing the residual difference between the original features and their downsampled-upsampled counterparts. Comprehensive experiments demonstrate that our proposed model achieves a practical balance between computational efficiency and restoration performance. It achieves a lower memory footprint and faster inference speed compared to recent global context or lightweight models under realistic constraints. Code is available at [https://github.com/jwgdmkj/CUST].
Single-Image Super-Resolution (SISR) aims to reconstruct a High-Resolution (HR) image from a Low-Resolution (LR) observation, a fundamentally ill-posed problem where high-frequency details are severely degraded at large upscaling factors. Recent advances have been driven by transformer-based architectures and diffusion models improve global context modeling and perceptual quality at the cost of increased computational complexity. In contrast, this work focuses on enhancing the expressivity of local operators under minimal resources. We propose SRGAN--CKAN, a hybrid super-resolution framework that integrates Convolutional Kolmogorov--Arnold Networks (CKAN) into an adversarial learning setting reformulating convolution as a nonlinear patch-based transformation. The proposed operator replaces linear local mappings with spline-based functional representations, allowing expressive modeling of complex local structures and high-frequency textures using minimal hardware resources. Experimental results demonstrate that the proposed approach improves perceptual quality while preserving reconstruction fidelity, achieving a favorable balance between distortion-based and perceptual metrics. These results are obtained under constrained computational settings, highlighting the efficiency of the proposed formulation. Overall, this work introduces a complementary direction to existing approaches by improving the representational power of local transformations, providing an efficient and scalable alternative to globally intensive architectures.
Roberto Isai Navaro-Aviña, Eduardo Said Merin-Martinez, Andres Mendez-Vazquez +1
Cinvestav, Unidad Guadalajara Av. del Bosque 1145, El Bajio Zapopan, Jalisco, 45017, Mexico · Cinvestav, Unidad Tamaulipas Km. 5.5 Carretera Victoria - Soto La Marina Victoria, 87139, Tamaulipas, Mexico