RPT-SR: Regional Prior attention Transformer for infrared image Super-Resolution

Youngwan Jin; Incheol Park; Yagiz Nalcakan; Hyeongjin Ju; Sanghyeop Yeo; Shiho Kim

cs.CVFeb 17, 2026

RPT-SR: Regional Prior attention Transformer for infrared image Super-Resolution

Authors: Youngwan Jin, Incheol Park, Yagiz Nalcakan, Hyeongjin Ju, Sanghyeop Yeo, Shiho Kim

Organizations: Yonsei University · BK21 Graduate Program in Intelligent Semiconductor Technology

Abstract

General-purpose super-resolution models, particularly Vision Transformers, have achieved remarkable success but exhibit fundamental inefficiencies in common infrared imaging scenarios like surveillance and autonomous driving, which operate from fixed or nearly-static viewpoints. These models fail to exploit the strong, persistent spatial priors inherent in such scenes, leading to redundant learning and suboptimal performance. To address this, we propose the Regional Prior attention Transformer for infrared image Super-Resolution (RPT-SR), a novel architecture that explicitly encodes scene layout information into the attention mechanism. Our core contribution is a dual-token framework that fuses (1) learnable, regional prior tokens, which act as a persistent memory for the scene's global structure, with (2) local tokens that capture the frame-specific content of the current input. By utilizing these tokens into an attention, our model allows the priors to dynamically modulate the local reconstruction process. Extensive experiments validate our approach. While most prior works focus on a single infrared band, we demonstrate the broad applicability and versatility of RPT-SR by establishing new state-of-the-art performance across diverse datasets covering both Long-Wave (LWIR) and Short-Wave (SWIR) spectra

Figures & tables

Figure 1: Comparison of attention mechanisms. (a) Standard self-attention computes global relationships at a high computational cost. (b) Window self-attention limits computation to local windows but misses global context. (c) Our proposed Regional Prior Attention (RPA) fuses a persistent, learnable Regional Prior (R.P.) token with a Local token. The R.P. token learns the scene’s static layout over epochs, providing a strong structural guide for the reconstruction.

Figure 2: The overall architecture of our proposed RPT-SR. (Top) The model consists of a shallow feature stem, a deep body of RPA Blocks, and a reconstruction head. (Bottom Left) A detailed view of the Regional Prior Attention (RPA) module. A dynamic Local Token, summarized from the input, is fused with a learnable, static Regional Prior Token. These are processed by a Hierarchical Attention (HAT) block to guide the reconstruction. (Bottom Right) The hierarchical windowing strategy, where the attention window size increases in deeper layers.

Method	Publication	LPIPS $\downarrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	FLOPs (G) $\downarrow$	Params (M) $\downarrow$
BSRGAN [ 40 ]	CVPR’21	$0.237\,155$	$33.612\,97$	$0.164\,349$	$297.730$	$16.700$
SwinIR [ 21 ]	ICCVW’21	$0.206\,968$	$31.855\,01$	$0.144\,053$	$192.080$	$11.900$
HAT [ 1 ]	CVPR’23	$0.111\,827$	$39.626\,91$	$0.244\,789$	$345.632$	$20.510$
HAT-s [ 1 ]	CVPR’23	$0.111\,712$	$40.070\,24$	$0.243\,761$	$163.313$	$9.359$
DAT [ 3 ]	ICCV’23	$0.108\,384$	$40.245\,81$	$0.247\,306$	$245.182$	$14.802$
DAT-light [ 3 ]	ICCV’23	$0.158\,623$	$38.163\,57$	$0.238\,152$	$9.106$	$0.600$

Table 1: Quantitative comparison for

\times 4

super-resolution on the M3FD dataset. The best results are marked in bold and the second-best are underlined .

Figure 3: Qualitative comparison for $\times$ 4 super-resolution on the M3FD dataset. Our method (RPT-SR) reconstructs sharper details and more plausible textures compared to existing state-of-the-art methods, particularly in restoring fine structures like human figures, building facades, and distant objects.

Method	LPIPS $\downarrow$
	RASMD	TNO
BSRGAN [ 40 ]	$0.188\,441$	$0.409\,504$
SwinIR [ 21 ]	$0.241\,546$	$0.479\,653$
HAT [ 1 ]	$0.156\,022$	$0.247\,531$
HAT-S [ 1 ]	$0.160\,292$	$0.254\,256$
DAT [ 3 ]	$0.162\,312$	$0.253\,371$

Table 2: Comparison of LPIPS scores on RASMD and TNO datasets (

\times 4

Method	LPIPS $\downarrow$
	RASMD	M3FD
BSRGAN [ 40 ]	$0.145\,707$	$0.131\,405$
SwinIR [ 21 ]	$0.035\,362$	$0.073\,155$
HAT-S [ 1 ]	$0.028\,607$	$0.073\,064$
RGT [ 4 ]	$0.038\,265$	$0.074\,580$
RGT-S [ 4 ]	$0.040\,890$	$0.075\,927$

Table 3: Comparison of LPIPS scores on M3FD and RASMD datasets (

\times 2

Method	M3FD x4			TNO x4			RASMD x4 (SWIR)			FLOPS	PARAMS
Method	LPIPS $\downarrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	LPIPS $\downarrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	LPIPS $\downarrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	FLOPS	PARAMS
Baseline	0.105057	41.2597	0.26091	0.25048	36.4154	0.24596	0.154442	44.39395	0.223607	231.366G	25.454M
Static	0.105324	41.6872	0.26034	0.25257	38.2838	0.24903	0.154457	44.72269	0.225821	229.100G	25.377M
RPT (Ours)	0.103793	41.8049	0.26213	0.25014	38.5862	0.25125	0.153543	44.91027	0.226634	237.783G	25.830M

Table 4: Ablation study of the proposed Regional Prior Transformer (RPT). We evaluate the impact of using only local tokens ( Baseline ), only static prior tokens ( Static ), and their fusion ( RPT (Ours) ). Best results are highlighted in bold red .

Figure 4: Attention maps at the last RPA layer from the M3FD test set. From left to right we show the local-only baseline, the static-prior variant, and our full model.

Explore similar work

Jan 13, 2026

From Local Windows to Adaptive Candidates via Individualized Exploratory: Rethinking Attention for Image Super-Resolution

Single Image Super-Resolution (SISR) is a fundamental computer vision task that aims to reconstruct a high-resolution (HR) image from a low-resolution (LR) input. Transformer-based methods have achieved remarkable performance by modeling long-range dependencies in degraded images. However, their feature-intensive attention computation incurs high computational cost. To improve efficiency, most existing approaches partition images into fixed groups and restrict attention within each group. Such group-wise attention overlooks the inherent asymmetry in token similarities, thereby failing to enable flexible and token-adaptive attention computation. To address this limitation, we propose the Individualized Exploratory Transformer (IET), which introduces a novel Individualized Exploratory Attention (IEA) mechanism that allows each token to adaptively select its own content-aware and independent attention candidates. This token-adaptive and asymmetric design enables more precise information aggregation while maintaining computational efficiency. Extensive experiments on standard SR benchmarks demonstrate that IET achieves state-of-the-art performance under comparable computational complexity.

Chunyu Meng, Wei Long, Shuhang Gu

University of Electronic Science and Technology of China

Efficient ViTs Sparse Attention Image Super-Resolution

Ask PDF

Apr 11, 2026

Dual-Branch Remote Sensing Infrared Image Super-Resolution

Remote sensing infrared image super-resolution aims to recover sharper thermal observations from low-resolution inputs while preserving target contours, scene layout, and radiometric stability. Unlike visible-image super-resolution, thermal imagery is weakly textured and more sensitive to unstable local sharpening, which makes complementary local and global modeling especially important. This paper presents our solution to the NTIRE 2026 Infrared Image Super-Resolution Challenge, a dual-branch system that combines a HAT-L branch and a MambaIRv2-L branch. The inference pipeline applies test-time local conversion on HAT, eight-way self-ensemble on MambaIRv2, and fixed equal-weight image-space fusion. We report both the official challenge score and a reproducible evaluation on 12 synthetic times-four thermal samples derived from Caltech Aerial RGB-Thermal, on which the fused output outperforms either single branch in PSNR, SSIM, and the overall Score. The results suggest that infrared super-resolution benefits from explicit complementarity between locally strong transformer restoration and globally stable state-space modeling.

Xining Ge, Gengjia Chang, Weijun Yuan +6

Hangzhou Dianzi University · Hefei University of Technology · Jinan University +1

Image Super-Resolution Remote Sensing Image Super-Resolution Image Restoration

Ask PDF

Aug 4, 2026

FaithIR: Rethinking Infrared Image Super-Resolution from Perceptual Sharpness to Task Relevant Fidelity

Infrared image super-resolution (IISR) is important for downstream tasks such as object detection and semantic segmentation. Existing IISR methods often produce artificial textures, over-sharpened edges, and spurious high-frequency details that distort authentic thermal structures and semantic information. To address this issue, we propose FaithIR, a faithful infrared super-resolution framework for reliable machine perception. FaithIR consists of a patch-level conditioning branch that captures global thermal and structural information and a pixel-level restoration branch that performs dense local reconstruction under structural guidance. The entire restoration process is performed directly in the pixel domain to preserve infrared-specific structures and task-relevant information. Extensive experiments on FLIR-IISR, M3FD, and FMB demonstrate strong reconstruction fidelity, cross-dataset generalization, and superior performance in object detection and semantic segmentation. These results show that demonstrate that preserving faithful infrared structure preservations is more important for reliable machine perception than merely pursuing perceptual sharpness alone.

Axi Niu, Zhenguo Wu, Kang Zhang +3

School of Computer Science, Northwestern Polytechnical University, Xi’an, Shaanxi, China · School of Electrical Engineering, KAIST, Daejeon, Republic of Korea · School of Aeronautics and Astronautics, Northwestern Polytechnical University, Xi’an, Shaanxi, China

Image Super-Resolution

Ask PDF