Cross-Scale Pansharpening via ScaleFormer and the PanScale Benchmark

Ke Cao; Xuanhua He; Xueheng Li; Lingting Zhu; Yingying Wang; Ao Ma; Zhanjie Zhang; Man Zhou

cs.CVFeb 28, 2026

Cross-Scale Pansharpening via ScaleFormer and the PanScale Benchmark

Authors: Ke Cao, Xuanhua He, Xueheng Li, Lingting Zhu, Yingying Wang, Ao Ma, Zhanjie Zhang, Man Zhou, +2 more

Organizations: HFIPS, Chinese Academy of Sciences · University of Science and Technology of China · The Hong Kong University of Science and Technology · The University of Hong Kong · Xiamen University · Zhejiang University

Abstract

Pansharpening aims to generate high-resolution multi-spectral images by fusing the spatial detail of panchromatic images with the spectral richness of low-resolution MS data. However, most existing methods are evaluated under limited, low-resolution settings, limiting their generalization to real-world, high-resolution scenarios. To bridge this gap, we systematically investigate the data, algorithmic, and computational challenges of cross-scale pansharpening. We first introduce PanScale, the first large-scale, cross-scale pansharpening dataset, accompanied by PanScale-Bench, a comprehensive benchmark for evaluating generalization across varying resolutions and scales. To realize scale generalization, we propose ScaleFormer, a novel architecture designed for multi-scale pansharpening. ScaleFormer reframes generalization across image resolutions as generalization across sequence lengths: it tokenizes images into patch sequences of the same resolution but variable length proportional to image scale. A Scale-Aware Patchify module enables training for such variations from fixed-size crops. ScaleFormer then decouples intra-patch spatial feature learning from inter-patch sequential dependency modeling, incorporating Rotary Positional Encoding to enhance extrapolation to unseen scales. Extensive experiments show that our approach outperforms SOTA methods in fusion quality and cross-scale generalization. The datasets and source code are available at https://github.com/caoke-963/ScaleFormer.

Figures & tables

Figure 1: Challenges in cross-scale pansharpening. (a) Learning/modeling at limited resolution while supporting inference on high-resolution images. (b) GPU memory increases rapidly with resolution, especially for Transformer-based models. (c) Tiled inference can introduce block artifacts in some methods. (d) A scale-induced distribution shift exists between training-scale inputs and large-scale inference.

Figure 2: Comparison of convolution, Transformer, and the proposed ScaleFormer. (a) Convolution: linear in image size but with a limited receptive field on high-resolution inputs; enlarging kernels helps but adds quadratic cost in kernel size. (b) Transformer: uniform patchification enables global context, yet self-attention scales quadratically with patch count, driving computation up on large images. (c) ScaleFormer (ours): introduces a sequence axis, factorizing global modeling into fixed-size spatial processing plus a scalable sequence dimension, yielding controllable complexity and mitigating scale-induced distribution shifts for robust performance across resolutions.

Figure 3: PanScale dataset: composition and distribution, resolution range, geographic sampling locations, and representative scene visualizations.

Datasets	Type	Real data	Train	MRT	LST
PanCollection [ 7 ]	RR and FR	✓	✓	✗	✗
NBU [ 32 ]	FR	✓	✓	✗	✗
PAirMax [ 42 ]	RR and FR	✓	✗	✗	✓
PanScale	RR and FR	✓	✓	✓	✓

Table 1: Comparison between the PanScale dataset and widely used pansharpening datasets, focusing on data type, whether the data is real, the inclusion of a training set, and support for Multi-Resolution Testing (MRT) and Large Scale Testing (LST).

Figure 4: The overall architecture of ScaleFormer, which primarily consists of three key components: the Scale-Aware Patchify module implementing the bucket sampling strategy, the Single Transformer Modules for intra-modal feature modeling, and the Cross Transformer Modules for inter-modal feature interaction and fusion.

	Jilin-Ave				Landsat-Ave				Skysat-Ave
Method	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	Q $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	Q $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	Q $\uparrow$
GS	25.4386	0.7055	5.6091	0.5849	35.0781	0.8952	4.5954	0.5873	39.0558	0.9317	2.3673	0.6371
IHS	25.5950	0.7023	5.8071	0.5803	35.8777	0.8915	4.3234	0.5554	39.5900	0.9304	2.1756	0.6271
GFPCA	25.6134	0.7185	5.0460	0.4509	36.2597	0.8963	3.6041	0.4747	37.6663	0.9280	2.4973	0.4773
MSDCNN	36.5711	0.9574	1.4965	0.9092	38.8173	0.9592	2.6570	0.6737	42.5298	0.9696	2.5983	0.7261
SFINet	36.2845	0.9523	1.6129	0.8958	38.8276	0.9567	2.8075	0.6739	41.7457	0.9518	2.9933	0.6950

Table 2: Quantitative Results on PanScale Datasets. The best results are highlighted in bold . The second-best results are highlighted in underline .

\uparrow

indicates that the larger the value, the better the performance, and

\downarrow

indicates that the smaller the value, the better the performance. ARConv uses tiled inference due to OOM, and other methods are evaluated with train–inference scale misalignment.

Figure 5: Comparison results across increasing resolution scales. The four subplots respectively depict: (a) PSNR score on the Landsat dataset, (b) PSNR score on the Skysat dataset, (c) model computational complexity (measured in GFLOPs) as a function of input resolution, and (d) Memory variation concerning input resolution.

	Full-Jilin-Ave			Full-Landsat-Ave			Full-Skysat-Ave
Method	$D_{\lambda}$ $\downarrow$	$D_{S}$ $\downarrow$	QNR $\uparrow$	$D_{\lambda}$ $\downarrow$	$D_{S}$ $\downarrow$	QNR $\uparrow$	$D_{\lambda}$ $\downarrow$	$D_{S}$ $\downarrow$	QNR $\uparrow$
GS	0.0812	0.2021	0.7338	0.0568	0.1110	0.8393	0.0623	0.1112	0.8347
IHS	0.0971	0.2124	0.7116	0.0961	0.1233	0.7936	0.0974	0.1318	0.7852
GFPCA	0.0424	0.0985	0.8633	0.0649	0.0539	0.8850	0.0433	0.0624	0.8976
MSDCNN	0.0477	0.0420	0.9126	0.0488	0.0326	0.9214	0.0419	0.0222	0.9403
SFINet	0.0538	0.0439	0.9050	0.0425	0.0342	0.9255	0.1346	0.0334	0.8379

Table 3: Evaluation of our method on real-world full-resolution scenes from three datasets.

Figure 6: Comparative visual experiments of several methods on Jilin datasets.

Methods	MSDCNN	SFINet	MSDDN	PanFlowNet	HFIN	Pan-mamba	ARConv	Ours
Params(M)	0.2390	0.0484	0.3649	0.0873	1.9836	0.1828	4.4147	0.5151
GFLOPs(G)	9.5488	1.8785	7.6938	13.9469	46.2067	7.3459	38.3152	20.5679

Table 4: Evaluation of parameter count and GFLOPs.

	Landsat-200		Landsat-400		Landsat-800		Landsat-1600
Ablation	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$
w/o RoPE	40.4625	0.9682	40.9542	0.9704	40.7619	0.9699	40.6924	0.9701
SeqT -> SpaT	40.9061	0.9700	41.2964	0.9724	40.7206	0.9714	40.5065	0.9716
w/o SAP	40.5279	0.9660	40.9325	0.9692	40.6186	0.9688	40.3895	0.9689
Baseline	40.6063	0.9677	41.3715	0.9725	41.1326	0.9719	41.0335	0.9722

Table 5: Ablation study of our method on the Landsat test set. The best are highlighted in bold , and the second are underlined .

Figure 7: The construction pipeline of the PanScale dataset, which consists of three main stages: accessing satellite imagery via Google Earth Engine, data preprocessing, and data cropping.

	Jilin (4 $\times$ )			Landsat (2 $\times$ )			Skysat (2.5 $\times$ )
	Samples	Scale	Res(m)	Samples	Scale	Res(m)	Samples	Scale	Res(m)
Train	1055	200	2	2484	256	30	2370	200	2
RR Test	34	800,400 200	2	96	1600,800 400,200	30	88	1600,800 400,200	2
FR Test	121	2000,1600 1200,800	2	48	2000,1600 1200,800	30	34	2000,1600 1200,800	2

Table 6: Detailed Information of the PanScale Datasets, where "samples" denotes the number of samples at each scale, "scale" represents the height or width of the square PAN image, and "res" indicates the image resolution used for dataset creation.

Figure 8: Comparative visual experiments of several methods on Jilin datasets.

Figure 9: Comparative visual experiments of several methods on Landsat datasets.

Figure 10: Comparative visual experiments of several methods on Skysat datasets.

	Jilin-200						Jilin-400
Method	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$	SCC $\uparrow$	Q $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$	SCC $\uparrow$	Q $\uparrow$
GS	25.7681	0.7128	0.0573	5.3592	0.8592	0.6002	25.6914	0.7099	0.0570	5.5151	0.8683	0.5805
IHS	25.9327	0.7081	0.0590	5.6410	0.8488	0.5930	25.8671	0.7063	0.0588	5.7120	0.8618	0.5754
GFPCA	25.6482	0.7106	0.0761	5.1018	0.8730	0.4579	25.8554	0.7252	0.0733	4.9484	0.8882	0.4469
MSDCNN	36.5984	0.9555	0.0277	1.5336	0.9885	0.9146	37.0869	0.9588	0.0265	1.4275	0.9907	0.9056
SFINet	36.2407	0.9508	0.0306	1.6494	0.9872	0.9017	36.6916	0.9532	0.0301	1.5592	0.9895	0.8915

Table 7: Multi-resolution results on the Jilin datasets (Part 1), with numbers indicating the size of the PAN image. The best are highlighted in bold .

	Jilin-800
Method	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$	SCC $\uparrow$	Q $\uparrow$
GS	24.8563	0.6939	0.0605	5.9530	0.8700	0.5739
IHS	24.9852	0.6926	0.0611	6.0682	0.8659	0.5726
GFPCA	25.3367	0.7198	0.0738	5.0877	0.8974	0.4479
MSDCNN	36.0279	0.9578	0.0263	1.5284	0.9907	0.9074
SFINet	35.9211	0.9528	0.0295	1.6301	0.9901	0.8942

Table 8: Multi-resolution results on the Jilin datasets (Part 2), with numbers indicating the size of the PAN image. The best are highlighted in bold .

	Landsat-200						Landsat-400
Method	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$	SCC $\uparrow$	Q $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$	SCC $\uparrow$	Q $\uparrow$
GS	36.6649	0.9049	0.0413	4.1725	0.8227	0.5872	36.1124	0.8994	0.0481	4.0663	0.8504	0.5853
IHS	37.3551	0.9001	0.0364	4.1847	0.7969	0.5566	36.7972	0.8952	0.0398	4.0051	0.8263	0.5547
GFPCA	37.1018	0.8968	0.0432	3.7622	0.8255	0.4707	36.9330	0.8975	0.0431	3.3999	0.8967	0.4744
MSDCNN	38.8386	0.9590	0.0407	2.8789	0.8242	0.6667	38.6542	0.9597	0.0396	2.7907	0.8442	0.6678
SFINet	38.7072	0.9523	0.0486	3.3100	0.8428	0.6666	39.1371	0.9569	0.0484	2.7937	0.8649	0.6683

Table 9: Multi-resolution results on the Landsat datasets (Part 1), with numbers indicating the size of the PAN image. The best are highlighted in bold .

	Landsat-800						Landsat-1600
Method	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$	SCC $\uparrow$	Q $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$	SCC $\uparrow$	Q $\uparrow$
GS	34.9034	0.8949	0.0517	4.6372	0.8873	0.5911	32.6318	0.8817	0.0633	5.5056	0.9064	0.5857
IHS	35.6661	0.8915	0.0407	4.2507	0.8640	0.5616	33.6923	0.8791	0.0461	4.8532	0.8755	0.5486
GFPCA	36.1215	0.8963	0.0413	3.5643	0.9074	0.4790	34.8823	0.8946	0.0418	3.6901	0.9320	0.4748
MSDCNN	38.7256	0.9600	0.0388	2.6008	0.8973	0.6809	39.0508	0.9582	0.0430	2.3577	0.9553	0.6792
SFINet	39.1135	0.9575	0.0461	2.5864	0.9167	0.6804	38.3526	0.9600	0.0389	2.5399	0.9314	0.6803

Table 10: Multi-resolution results on the Landsat datasets (Part 2), with numbers indicating the size of the PAN image. The best are highlighted in bold .

	Skysat-200						Skysat-400
Method	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$	SCC $\uparrow$	Q $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$	SCC $\uparrow$	Q $\uparrow$
GS	39.4693	0.9354	0.0326	2.2604	0.9121	0.6343	39.2596	0.9346	0.0310	2.3096	0.9357	0.6478
IHS	40.1768	0.9354	0.0264	2.0909	0.8934	0.6259	39.7388	0.9326	0.0260	2.1468	0.9206	0.6362
GFPCA	38.0562	0.9289	0.0454	2.4658	0.8896	0.4720	37.8788	0.9275	0.0441	2.4482	0.9213	0.4867
MSDCNN	43.0041	0.9758	0.0406	2.8975	0.8886	0.7373	43.1001	0.9788	0.0399	2.2283	0.9259	0.7450
SFINet	41.6073	0.9417	0.1010	3.4414	0.8496	0.6763	41.3735	0.9418	0.0939	3.1682	0.8911	0.6829

Table 11: Multi-resolution results on the Skysat datasets (Part 1), with numbers indicating the size of the PAN image. The best are highlighted in bold .

	Skysat-800						Skysat-1600
Method	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$	SCC $\uparrow$	Q $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$	SCC $\uparrow$	Q $\uparrow$
GS	38.9661	0.9302	0.0322	2.4344	0.9474	0.6367	38.5282	0.9264	0.0328	2.4648	0.9555	0.6294
IHS	39.3872	0.9280	0.0275	2.2406	0.9361	0.6247	39.0572	0.9255	0.0269	2.2241	0.9468	0.6217
GFPCA	37.4581	0.9282	0.0457	2.5427	0.9345	0.4751	37.2721	0.9272	0.0446	2.5326	0.9442	0.4755
MSDCNN	43.0467	0.9792	0.0391	2.1924	0.9347	0.7424	40.9681	0.9444	0.0851	3.0750	0.9211	0.6798
SFINet	41.3829	0.9452	0.0813	2.9911	0.9136	0.6787	42.6192	0.9786	0.0391	2.3726	0.9423	0.7419

Table 12: Multi-resolution results on the Skysat datasets (Part 2), with numbers indicating the size of the PAN image. The best are highlighted in bold .

	Full-Jilin-800			Full-Jilin-1200			Full-Jilin-1600			Full-Jilin-2000
Method	$D_{\lambda}$ $\downarrow$	$D_{S}$ $\downarrow$	QNR $\uparrow$	$D_{\lambda}$ $\downarrow$	$D_{S}$ $\downarrow$	QNR $\uparrow$	$D_{\lambda}$ $\downarrow$	$D_{S}$ $\downarrow$	QNR $\uparrow$	$D_{\lambda}$ $\downarrow$	$D_{S}$ $\downarrow$	QNR $\uparrow$
GS	0.0795	0.1936	0.7433	0.0791	0.2012	0.7362	0.0814	0.2055	0.7304	0.0846	0.2081	0.7253
IHS	0.0955	0.2060	0.7191	0.0961	0.2121	0.7126	0.0972	0.2150	0.7092	0.0997	0.2165	0.7056
GFPCA	$\underline{0.0464}$	0.1001	0.8584	0.0420	0.0980	0.8641	0.0405	0.0980	0.8654	0.0407	0.0979	0.8654
MSDCNN	0.0537	0.0499	0.8995	0.0460	0.0416	0.9144	0.0455	0.0394	0.9170	0.0454	0.0371	0.9194
SFINet	0.0596	0.0515	0.8925	0.0519	0.0430	0.9075	0.0517	0.0415	0.9092	0.0519	0.0395	0.9109

Table 13: Multi-resolution results on the Full-Jilin datasets, with numbers indicating the size of the PAN image. The best are highlighted in bold , and the second are underlined .

	Full-Landsat-800			Full-Landsat-1200			Full-Landsat-1600			Full-Landsat-2000
Method	$D_{\lambda}$ $\downarrow$	$D_{S}$ $\downarrow$	QNR $\uparrow$	$D_{\lambda}$ $\downarrow$	$D_{S}$ $\downarrow$	QNR $\uparrow$	$D_{\lambda}$ $\downarrow$	$D_{S}$ $\downarrow$	QNR $\uparrow$	$D_{\lambda}$ $\downarrow$	$D_{S}$ $\downarrow$	QNR $\uparrow$
GS	0.0519	0.1145	0.8405	0.0589	0.1155	0.8334	0.0567	0.1076	0.8425	0.0598	0.1064	0.8406
IHS	0.0862	0.1235	0.8012	0.0972	0.1286	0.7883	0.0982	0.1205	0.7945	0.1028	0.1205	0.7905
GFPCA	0.0731	0.0577	0.8737	0.0623	0.0543	0.8870	0.0627	0.0525	0.8883	0.0615	0.0510	0.8910
MSDCNN	0.0523	0.0378	0.9135	0.0476	0.0316	0.9234	0.0485	0.0307	0.9234	0.0468	0.0303	0.9254
SFINet	0.0493	0.0405	0.9137	0.0410	0.0331	0.9278	0.0404	0.0314	0.9299	0.0392	0.0317	0.9307

Table 14: Multi-resolution results on the Full-Landsat datasets, with numbers indicating the size of the PAN image. The best are highlighted in bold , and the second are underlined .

	Full-Skysat-800			Full-Skysat-1200			Full-Skysat-1600			Full-Skysat-2000
Method	$D_{\lambda}$ $\downarrow$	$D_{S}$ $\downarrow$	QNR $\uparrow$	$D_{\lambda}$ $\downarrow$	$D_{S}$ $\downarrow$	QNR $\uparrow$	$D_{\lambda}$ $\downarrow$	$D_{S}$ $\downarrow$	QNR $\uparrow$	$D_{\lambda}$ $\downarrow$	$D_{S}$ $\downarrow$	QNR $\uparrow$
GS	0.0533	0.1033	0.8499	0.0657	0.1163	0.8273	0.0653	0.1129	0.8304	0.0649	0.1123	0.8311
IHS	0.0937	0.1282	0.7916	0.0973	0.1343	0.7834	0.0993	0.1333	0.7822	0.0994	0.1314	0.7836
GFPCA	0.0427	0.0633	0.8974	0.0445	0.0633	0.8956	0.0428	0.0620	0.8983	0.0432	0.0610	0.8989
MSDCNN	0.0449	0.0262	0.9344	0.0407	0.0282	0.9422	0.0422	0.0187	0.9391	0.0398	0.0155	0.9455
SFINet	0.1357	0.0331	0.8373	0.1304	0.0343	0.8412	0.1363	0.0333	0.8362	0.1361	0.0327	0.8368

Table 15: Multi-resolution results on the Full-Skysat datasets, with numbers indicating the size of the PAN image. The best are highlighted in bold , and the second are underlined .

Explore similar work

G-ZAP: A Generalizable Zero-Shot Framework for Arbitrary-Scale Pansharpening
Mar 15, 2026Zhiqi Yang, Shan Yin, Jingze Liang +1Zero-Shot Learning Remote Sensing
University of Electronic Science and Technology of China
GSPan: A Continuous Gaussian Primitive Representation for Arbitrary-Scale Pansharpening
Jun 16, 2026Fangyi Li, Xiaoyuan Yang, Yixiao Li +3Remote Sensing Image Super-Resolution Remote Sensing
Beihang University, School of Mathematical Sciences, Beijing, 102206, China · Tsinghua University, Department of Automation, Beijing, 100084, China · National Research Council - Institute of Methodologies for Environmental Analysis, CNR-IMAA, Tito, 85050, Italy
Universal Pansharpening Model
Mar 4, 2026Hebaixu Wang, Jing Zhang, Haonan Guo +4Multi-Source Image Fusion Remote Sensing
School of Electronic Information, Wuhan University, Hubei, 430072, Wuhan, China. · Zhongguancun Academy, Beijing, 100094,China. · School of Computer Science, Wuhan University, Hubei, 430072, Wuhan, China. +2

Mar 15, 2026J/K move · Enter open · S save

G-ZAP: A Generalizable Zero-Shot Framework for Arbitrary-Scale Pansharpening

Zhiqi Yang, Shan Yin, Jingze Liang, Liang-Jian Deng

University of Electronic Science and Technology of China

Zero-Shot Learning Remote Sensing

Open paper AskPDF arXiv

Pansharpening aims to fuse a high-resolution panchromatic (PAN) image and a low-resolution multispectral (LRMS) image to produce a high-resolution multispectral (HRMS) image. Recent deep models have achieved strong performance, yet they typically rely on large-scale pretraining and often generalize poorly to unseen real-world image pairs. Prior zero-shot approaches improve real-scene generalization but require per-image optimization, hindering weight reuse, and the above methods are usually limited to a fixed scale. To address this issue, we propose G-ZAP, a generalizable zero-shot framework for arbitrary-scale pansharpening, designed to handle cross-resolution, cross-scene, and cross-sensor generalization. G-ZAP adopts a feature-based implicit neural representation (INR) fusion network as the backbone and introduces a multi-scale, semi-supervised training scheme to enable robust generalization. Extensive experiments on multiple real-world datasets show that G-ZAP achieves state-of-the-art results under PAN-scale fusion in both visual quality and quantitative metrics. Notably, G-ZAP supports weight reuse across image pairs while maintaining competitiveness with per-pair retraining, demonstrating strong potential for efficient real-world deployment.

Abstract

Explore similar work

G-ZAP: A Generalizable Zero-Shot Framework for Arbitrary-Scale Pansharpening

GSPan: A Continuous Gaussian Primitive Representation for Arbitrary-Scale Pansharpening

Universal Pansharpening Model