Universal Pansharpening Model

Hebaixu Wang; Jing Zhang; Haonan Guo; Di Wang; Jiayi Ma; Bo Du; Liangpei Zhang

cs.CVMar 4, 2026

Universal Pansharpening Model

Authors: Hebaixu Wang, Jing Zhang, Haonan Guo, Di Wang, Jiayi Ma, Bo Du, Liangpei Zhang

Organizations: School of Electronic Information, Wuhan University, Hubei, 430072, Wuhan, China. · Zhongguancun Academy, Beijing, 100094,China. · School of Computer Science, Wuhan University, Hubei, 430072, Wuhan, China. · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Hubei, 430072, Wuhan, China. · School of Robotics, Wuhan University, Hubei, 430072, Wuhan, China.

Abstract

Pansharpening generates the high-resolution multi-spectral (MS) image by integrating spatial details from a texture-rich panchromatic (PAN) image and spectral attributes from a low-resolution MS image. Existing methods are predominantly satellite-specific and scene-dependent, which severely limits their generalization across heterogeneous sensors and varied scenes, thereby reducing their real-world practicality. To address these challenges, we present UniPS, a universal pansharpening model for satellite-agnostic and scene-robust fusion. Specifically, we introduce a modality-interleaved transformer that learns band-wise modal specializations to form reversible spectral affine bases, mapping arbitrary-band MS into a unified latent space via tensor multiplication. Building upon this, we construct a latent diffusion bridge model to progressively evolve latent representations, and incorporate bridge posterior sampling to couple latent diffusion with pixel-space observations, enabling stable and controllable fusion. Furthermore, we devise infinite-dimensional pixel-to-latent interaction mechanisms to comprehensively capture the cross-domain dependencies between PAN observations and MS representations, thereby facilitating complementary information fusion. In addition, to support large-scale training and evaluation, we construct a comprehensive pansharpening benchmark, termed PSBench, consisting of worldwide MS and PAN image pairs from multiple satellites across diverse scenes. Extensive experiments verify that UniPS consistently outperforms state-of-the-art methods, exhibiting superior generalization and robustness across a wide range of tasks.

Figures & tables

Figure 1: Pansharpening performance comparisons on PSBench across representative spectral band configurations (4-, 7-, 8-, and 10-band). The upper and lower panels show the reduced- and full-scale evaluations using PSNR and the non-reference QNR metric, respectively. UniPS consistently outperforms the advanced task-specific models, offering a universal solution for pansharpening tasks.

Figure 2: Mainstream pansharpening paradigms: (a) Satellite-specific methods employ independent encoder-decoder architectures per spectral configuration, resulting in parameter redundancy and poor cross-satellite scalability. (b) Band-truncated methods unify data format by selecting a fixed band subset, enabling joint training but discarding spectral information and losing the ability to process excluded bands. (c) Our universal method performs band-agnostic fusion by projecting arbitrary-band MS images into a shared latent space, achieving unified modeling without band truncation or parameter duplication, while preserving full cross-band and cross-modal information.

Figure 3: Satellite observation locations and distribution of PSBench. It contains four representative spectral configurations for pansharpening tasks with over seventeen landcover categories.

Satellite/Sensor	Launch	Launch	MS Spatial	PAN Spatial	Multi-Spectral	Panchromatic	Number of	Order of MS Bands	MS Pixel	PAN Pixel
Satellite/Sensor	Date	Country	Resolution	Resolution	Range	Range	MS Bands	Order of MS Bands	Size	Size
GaoFen-1 1 1 1 https://data.cresda.cn	2013	China	8.0 m	2.0 m	450-890 nm	450-900 nm	4	B/G/R/NIR	256	1024
GaoFen-2 1 1 1 https://data.cresda.cn	2014	China	3.2 m	0.8 m	450-890 nm	450-900 nm	4	B/G/R/NIR	256	1024
GaoFen-6 1 1 1 https://data.cresda.cn	2018	China	8.0 m	2.0 m	450-900 nm	450-900 nm	4	B/G/R/NIR	256	1024
GaoFen-7 1 1 1 https://data.cresda.cn	2020	China	2.6 m	0.65 m	450-900 nm	450-900 nm	4	B/G/R/NIR	256	1024
GeoEye-1 2 2 2 https://maxar-opendata.s3.amazonaws.com/	2008	USA	1.64 m	0.41m	450-920 nm	450-800 nm	4	B/G/R/NIR	256	1024

Table 1: Details of the collected multi-spectral and panchromatic image pairs from different satellites.

Figure 4: The overall framework of our method. Multiple satellite observations with arbitrary spectral bands are first unified into a latent space through the modality-interleaved transformer. A latent diffusion bridge model is established for progressive latent refinement through a sequence of iterative updates, whose backbone is a tailored Infinite U-Net empowered by an infinite-dimensional interaction mechanism. This design seamlessly injects complementary spatial cues extracted from PAN images into the latent representations. These enhanced latent representations are projected back to the image domain to reconstruct the high-resolution MS images with flexible spectral dimensions.

Figure 5: The network architecture of Infinite-UNet in latent diffusion bridge model.

Figure 6: Visual comparisons of reduced scale on 4-band PSBench.

Figure 7: Visual comparisons of reduced scale on 8-band PSBench.

Traditional Method
		4 Bands				7 Bands				8 Bands				10 Bands				Average
Method	Year	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$
SFIM Liu (2000)	2000	26.901	0.796	19.712	5.697	29.643	0.899	8.327	2.227	26.770	0.793	35.380	6.176	33.330	0.958	4.220	2.756	29.475	0.867	14.917	4.186
IHS Tu et al. (2001)	2001	22.947	0.712	10.916	7.715	24.400	0.857	4.166	3.026	25.835	0.708	11.245	8.607	33.422	0.955	3.199	2.450	27.060	0.817	7.234	5.286
AWLP Otazu et al. (2005)	2005	28.919	0.799	6.916	6.998	28.923	0.744	2.586	2.160	24.520	0.782	10.888	10.929	34.180	0.867	2.509	1.960	29.994	0.809	5.286	5.065
MTF_GLP Aiazzi et al. (2006)	2006	24.682	0.771	16.405	6.401	20.445	0.866	4.878	1.842	26.609	0.772	10.462	6.104	32.696	0.955	3.091	1.237	26.833	0.848	9.115	3.854

Table 2: Quantitative comparisons on reduced scale over the PSBench. The best and second best results are shown in bold and underline respectively. The up or down arrows indicate higher or lower values correspond to better results.

Figure 8: Visual comparisons of full scale on 4-band PSBench.

Figure 9: Visual comparisons of full scale on 8-band PSBench.

Traditional Method
		4 Bands			7 Bands			8 Bands			10 Bands			Average
Method	Year	QNR $\uparrow$	$D_{\lambda}\downarrow$	$D_{s}\downarrow$	QNR $\uparrow$	$D_{\lambda}\downarrow$	$D_{s}\downarrow$	QNR $\uparrow$	$D_{\lambda}\downarrow$	$D_{s}\downarrow$	QNR $\uparrow$	$D_{\lambda}\downarrow$	$D_{s}\downarrow$	QNR $\uparrow$	$D_{\lambda}\downarrow$	$D_{s}\downarrow$
SFIM Liu (2000)	2000	0.7849	0.0919	0.1372	0.8115	0.0768	0.1227	0.8437	0.0719	0.0918	0.8158	0.0580	0.1343	0.8084	0.0752	0.1270
IHS Tu et al. (2001)	2001	0.5925	0.1055	0.3393	0.6957	0.1360	0.2002	0.8112	0.0875	0.1128	0.7444	0.1408	0.1356	0.6928	0.1198	0.2145
AWLP Otazu et al. (2005)	2005	0.7733	0.0839	0.1619	0.8032	0.0984	0.1112	0.8533	0.0486	0.1042	0.7354	0.1550	0.1305	0.7780	0.1046	0.1341
MTF_GLP Aiazzi et al. (2006)	2006	0.7054	0.1175	0.2051	0.8200	0.0880	0.1035	0.8064	0.0855	0.1196	0.7725	0.0520	0.1857	0.7627	0.0861	0.1680

Table 3: Quantitative comparisons on full scale over the PSBench. The best and second best results are shown in bold and underline respectively. The up or down arrows indicate higher or lower values correspond to better results.

Panel of Figure 10 — Figure 10: Distribution of data and parameters by t-SNE visualization. (a) Expert parameters and their activation frequency. Larger circles represent higher activation frequencies. (b) MS images on the reduced scale. (c) MS latents generated by MiT on the reduced scale. (d) MS images on the full scale. (e) MS latents generated by MiT on the full scale.

NFEs	4 Bands				7 Bands				8 Bands				10 Bands				Average
NFEs	PSNR $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	QNR $\uparrow$	PSNR $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	QNR $\uparrow$	PSNR $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	QNR $\uparrow$	PSNR $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	QNR $\uparrow$	PSNR $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	QNR $\uparrow$
1	33.232	6.219	3.220	0.9240	37.949	1.144	1.195	0.9262	28.610	11.338	4.880	0.8906	45.127	0.849	0.973	0.9042	37.290	4.306	2.366	0.9130
3	34.991	4.839	2.845	0.9434	39.698	0.939	1.004	0.9217	30.736	7.981	4.362	0.9197	46.759	0.700	0.852	0.8934	39.035	3.257	2.090	0.9197
5	36.795	3.872	2.708	0.9392	40.973	0.825	0.912	0.9097	33.196	6.057	4.255	0.9167	47.869	0.616	0.822	0.8760	40.634	2.582	1.996	0.9100
10	37.370	3.637	2.826	0.9327	41.307	0.804	0.904	0.9009	34.063	5.531	4.402	0.9135	47.614	0.640	0.887	0.8679	40.937	2.427	2.079	0.9030
20	37.047	3.782	3.003	0.9294	41.126	0.825	0.927	0.8992	33.807	5.658	4.577	0.9134	47.216	0.671	0.932	0.8664	40.625	2.510	2.185	0.9010

Table 4: Pansharpening performance of reduced scale with different sampling steps.

Configurations				4 Bands				7 Bands				8 Bands				10 Bands				Average
	MiT	$\mathcal{K}_{exp}$	$\mathcal{K}_{geo}$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$
I	✓	✓	✓	37.370	0.929	3.636	2.825	41.307	0.972	0.804	0.904	34.063	0.913	5.531	4.402	47.614	0.991	0.640	0.887	40.937	0.955	2.427	2.079
II	✕	✓	✓	33.497	0.857	5.482	3.937	37.909	0.953	1.141	1.329	29.067	0.778	8.615	5.171	42.349	0.982	1.198	1.494	36.529	0.903	3.767	2.854
III	✓	✓	✕	35.919	0.917	4.205	3.253	38.119	0.957	1.125	1.246	32.382	0.891	7.924	5.680	43.731	0.984	1.021	1.452	38.346	0.942	3.158	2.660
IV	✓	✕	✓	36.255	0.920	4.541	3.066	39.199	0.959	1.090	1.108	32.785	0.898	8.125	5.099	44.793	0.987	0.881	1.139	39.063	0.946	3.252	2.383
V	✓	Add.		33.776	0.865	5.645	3.382	37.698	0.953	1.220	1.251	29.360	0.803	9.375	5.099	44.324	0.983	1.125	1.332	37.278	0.910	3.925	2.584

Table 6: Fusion performance of UniPS variants at reduced scale.

Figure 11: Feature interaction blocks of UniPS variants.

Panel of Figure 12 — Figure 12: Visual comparisons of reduced scale on SegGF.

Method	QuickBird Ma et al. (2020)					SegGF Wang and Ma (2025)					Jiling Cao et al. (2026)					skysat Cao et al. (2026)
Method	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	QNR $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	QNR $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	QNR $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	QNR $\uparrow$
PNN Masi et al. (2016)	24.9903	0.8396	7.5687	11.6871	0.4390	24.1732	0.6991	3.9805	5.0416	0.3723	24.8081	0.8318	6.2130	5.2326	0.8231	26.0699	0.8495	6.4022	5.3754	0.8319
PanNet Yang et al. (2017)	21.6332	0.7963	12.6515	14.6404	0.4285	20.8404	0.5865	6.1647	10.2467	0.5113	22.8129	0.7539	7.9233	10.1700	0.7261	24.0236	0.7826	7.8340	10.1602	0.7052
GPPNN Xu et al. (2021)	26.3027	0.8616	6.9445	11.7491	0.7272	24.0514	0.6990	4.2215	5.6822	0.6025	25.2377	0.7580	4.9689	4.7102	0.7855	26.9535	0.7947	4.7313	4.8363	0.7915
U2Net Peng et al. (2023)	27.4097	0.7240	5.5303	8.7204	0.7961	22.4246	0.7077	5.7532	6.6718	0.8453	26.6666	0.8405	4.8783	5.2118	0.7506	27.6754	0.8642	4.7137	5.7169	0.7776
PanFlowNet Yang et al. (2023)	21.4585	0.7554	17.6751	13.2153	0.6191	12.7717	0.4703	30.0502	14.0260	0.3733	26.3085	0.7826	4.6496	3.5070	0.7918	28.7214	0.8209	4.2169	3.2943	0.8109

Table 9: Zero-shot pansharpening performance on unseen scenes and satellite datasets.

	4 Bands				Truncated 7 Bands				Truncated 8 Bands				Truncated 10 Bands				Average
Method	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$
$\ast$ PNN Masi et al. (2016)	31.547	0.861	10.794	5.986	26.731	0.893	4.745	4.767	30.510	0.837	12.725	6.909	34.104	0.955	3.790	3.736	31.364	0.894	7.703	5.168
$\ast$ PanNet Yang et al. (2017)	32.325	0.875	6.368	7.615	30.029	0.933	2.837	5.576	31.058	0.869	7.385	8.240	37.797	0.972	2.022	4.157	33.513	0.916	4.463	6.211
$\ast$ GPPNN Xu et al. (2021)	32.943	0.878	5.101	5.212	31.818	0.937	2.014	3.872	31.857	0.883	6.404	5.872	38.459	0.970	1.527	3.077	34.388	0.920	3.570	4.370
$\ast$ U2Net Peng et al. (2023)	33.081	0.849	6.156	4.608	33.197	0.923	1.881	3.338	29.632	0.785	8.527	5.632	41.810	0.970	1.253	1.940	35.451	0.892	4.138	3.659
$\ast$ PanFlowNet Yang et al. (2023)	34.306	0.886	5.526	4.696	36.174	0.945	1.310	1.698	31.952	0.867	6.914	6.316	41.120	0.971	1.396	2.169	36.523	0.922	3.626	3.573

Table 10: Quantitative comparisons on reduced scale over the truncated 4-band PSBench. All of deep methods are retrained on the mixed truncated datasets, denoted by

\ast

Mask	4 Bands				7 Bands				8 Bands				10 Bands				Average
Ratio	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	ERGAS $\downarrow$	SAM $\downarrow$
0%	37.370	0.929	3.636	2.825	41.307	0.972	0.804	0.904	34.063	0.913	5.531	4.402	47.614	0.991	0.640	0.887	40.937	0.955	2.427	2.079
10%	36.918	0.927	3.661	2.998	39.174	0.960	1.078	1.181	33.665	0.910	5.760	4.768	45.527	0.986	0.873	1.156	39.657	0.950	2.594	2.330
20%	35.874	0.919	4.453	3.795	36.002	0.950	2.204	2.182	32.889	0.902	6.982	5.719	43.049	0.982	1.465	1.907	37.802	0.943	3.444	3.171
40%	33.879	0.895	8.601	5.797	29.589	0.911	27.523	6.199	30.716	0.862	18.928	8.953	36.674	0.965	4.370	4.961	33.543	0.915	12.166	6.066
60%	31.583	0.851	22.574	8.842	22.313	0.831	165.034	14.053	27.772	0.788	58.058	14.934	28.278	0.906	15.746	13.660	28.281	0.856	51.053	12.249

Table 12: Pansharpening performance at reduced scale under different mask ratio.

Figure 14: Hyperspectral generalization results.

Figure 15: Reduced-scale segmentation comparisons on SegGF.

Figure 16: Full-scale segmentation comparisons on SegGF.

Method	Reduced-scale												Full-scale
	Accuracy						IoU						Accuracy						IoU
	Avg.	Land.	Build.	Road.	Vega.	Water.	Avg	Land.	build.	Road.	Vega.	water.	Avg.	Land.	Build.	Road.	Vega.	Water.	Avg.	Land.	build.	Road.	Vega.	water.
PAN	46.29	37.17	60.37	36.82	67.57	75.82	31.38	31.31	38.72	25.75	42.35	50.18	50.09	50.08	67.24	58.20	58.32	66.72	35.48	38.58	45.80	35.49	41.89	51.12
LRMS	54.84	71.18	59.27	47.72	83.13	67.76	42.52	57.76	47.60	33.16	62.05	42.35	57.15	73.46	71.90	48.77	82.28	66.46	44.83	59.05	52.94	37.38	63.97	55.64
SFIM Liu (2000)	56.42	71.97	69.05	46.11	79.23	72.16	43.14	58.54	49.63	33.35	62.09	55.23	60.06	78.78	69.00	59.32	78.57	74.68	47.59	61.17	56.28	43.19	64.20	60.70
IHS Tu et al. (2001)	56.05	72.16	67.40	42.84	79.40	74.51	42.49	57.64	49.87	31.62	61.31	54.49	57.40	76.09	74.79	54.30	80.62	58.63	45.53	59.45	55.92	40.68	64.61	52.52

Table 14: Segmentation performance on the SegGF dataset. Bold means the best average segmentation performance.

Figure 17: Remote sensing application of NDVI.

Figure 18: Remote sensing application of NDBI.

	4 Bands (NDVI)			7 Bands (NDWI)			8 Bands (NDRE)			10 Bands (NDBI)			Average
Method	RMSE $\downarrow$	MAE $\downarrow$	CC $\uparrow$	RMSE $\downarrow$	MAE $\downarrow$	CC $\uparrow$	RMSE $\downarrow$	MAE $\downarrow$	CC $\uparrow$	RMSE $\downarrow$	MAE $\downarrow$	CC $\uparrow$	RMSE $\downarrow$	MAE $\downarrow$	CC $\uparrow$
SFIM Liu (2000)	0.1202	0.0719	0.7965	0.0326	0.0204	0.9094	0.0966	0.0536	0.6701	0.0254	0.0175	0.9250	0.0701	0.0422	0.8401
IHS Tu et al. (2001)	0.1114	0.0748	0.8077	0.0373	0.0252	0.8802	0.0846	0.0526	0.6965	0.0257	0.0178	0.9233	0.0662	0.0440	0.8421
AWLP Otazu et al. (2005)	0.2294	0.1227	0.6739	0.0298	0.0205	0.9136	0.2563	0.1369	0.3355	0.0221	0.0150	0.9398	0.1299	0.0713	0.7539
MTF_GLP Aiazzi et al. (2006)	0.1353	0.0808	0.7747	0.0317	0.0189	0.9223	0.1398	0.0698	0.5704	0.0217	0.0140	0.9434	0.0803	0.0463	0.8262
GS Aiazzi et al. (2007)	0.1088	0.0722	0.8077	0.0432	0.0286	0.8597	0.0835	0.0503	0.6994	0.0276	0.0193	0.9209	0.0668	0.0439	0.8381

Table 15: Quantitative evaluation of remote sensing index over four representative applications. The best results are shown in bold . The up or down arrows indicate higher or lower values correspond to better results.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 28 — Figure S1: 4-band visual comparisons of reduced scale on PSBench. Zoom in for the best.

Panel of Figure 29 — Figure S3: 8-band visual comparisons of reduced scale on PSBench. Zoom in for the best.

Panel of Figure 30 — Figure S6: 7-band visual comparisons of full scale on PSBench. Zoom in for the best.

Panel of Figure 31 — Figure S9: Visual comparisons of reduced scale on Quickbird.

Panel of Figure 32 — Figure S13: Visual comparisons of full scale on Jinlin.

Explore similar work

Cross-Scale Pansharpening via ScaleFormer and the PanScale Benchmark
Feb 28, 2026Ke Cao, Xuanhua He, Xueheng Li +7Multi-Source Image Fusion Remote Sensing
HFIPS, Chinese Academy of Sciences · University of Science and Technology of China · The Hong Kong University of Science and Technology +3
Fast Model-guided Instance-wise Adaptation Framework for Real-world Pansharpening with Fidelity Constraints
Apr 10, 2026Zhiqi Yang, Jin-Liang Xiao, Shan Yin +2Multi-Source Image Fusion Remote Sensing
University of Electronic Science and Technology of China, Chengdu, China · National Research Council, Institute of Methodologies for Environmental Analysis (CNR-IMAA), Tito, Italy
G-ZAP: A Generalizable Zero-Shot Framework for Arbitrary-Scale Pansharpening
Mar 15, 2026Zhiqi Yang, Shan Yin, Jingze Liang +1Zero-Shot Learning Remote Sensing
University of Electronic Science and Technology of China

Feb 28, 2026J/K move · Enter open · S save

Cross-Scale Pansharpening via ScaleFormer and the PanScale Benchmark

Ke Cao, Xuanhua He, Xueheng Li, Lingting Zhu+6

HFIPS, Chinese Academy of Sciences · University of Science and Technology of China · The Hong Kong University of Science and Technology+3

Multi-Source Image Fusion Remote Sensing Remote Sensing Benchmarks Vision Transformer

Open paper AskPDF arXiv

Pansharpening aims to generate high-resolution multi-spectral images by fusing the spatial detail of panchromatic images with the spectral richness of low-resolution MS data. However, most existing methods are evaluated under limited, low-resolution settings, limiting their generalization to real-world, high-resolution scenarios. To bridge this gap, we systematically investigate the data, algorithmic, and computational challenges of cross-scale pansharpening. We first introduce PanScale, the first large-scale, cross-scale pansharpening dataset, accompanied by PanScale-Bench, a comprehensive benchmark for evaluating generalization across varying resolutions and scales. To realize scale generalization, we propose ScaleFormer, a novel architecture designed for multi-scale pansharpening. ScaleFormer reframes generalization across image resolutions as generalization across sequence lengths: it tokenizes images into patch sequences of the same resolution but variable length proportional to image scale. A Scale-Aware Patchify module enables training for such variations from fixed-size crops. ScaleFormer then decouples intra-patch spatial feature learning from inter-patch sequential dependency modeling, incorporating Rotary Positional Encoding to enhance extrapolation to unseen scales. Extensive experiments show that our approach outperforms SOTA methods in fusion quality and cross-scale generalization. The datasets and source code are available at https://github.com/caoke-963/ScaleFormer.

Abstract

Explore similar work

Cross-Scale Pansharpening via ScaleFormer and the PanScale Benchmark

Fast Model-guided Instance-wise Adaptation Framework for Real-world Pansharpening with Fidelity Constraints

G-ZAP: A Generalizable Zero-Shot Framework for Arbitrary-Scale Pansharpening