ReTailor: A Residual Feature Tailoring Framework for Real-Time Cross-Sensor Pansharpening Adaptation

Jing-Ze Liang; Wei-Jie Xiao; Zi-Han Cao; Yang-Dong Li; Tian-Yi Hu; Liang-Jian Deng

cs.CVNov 4, 2026

ReTailor: A Residual Feature Tailoring Framework for Real-Time Cross-Sensor Pansharpening Adaptation

Authors: Jing-Ze Liang, Wei-Jie Xiao, Zi-Han Cao, Yang-Dong Li, Tian-Yi Hu, Liang-Jian Deng

Organizations: University of Electronic Science and Technology of China, Chengdu, 611731, Sichuan, China · Multi-Hazard Early Warning Key Laboratory of Sichuan Province, Chengdu, 611731, Sichuan, China

Abstract

Recent pansharpening models achieve strong fusion performance on trained sensors, yet often generalize poorly to unseen sensors. Existing cross-sensor adaptation is limited by target-sensor retraining, which requires target-sensor training pairs and a dedicated training stage, and by per-image zero-shot adaptation, which is computationally expensive due to whole-network optimization for each test image. To address these problems, we propose ReTailor, a real-time sequential cross-sensor adaptation framework. Only a lightweight residual Feature Tailor is rapidly optimized under self-supervised, physics-aware constraints to correct cross-sensor shifts without modifying the well-learned fusion prior. For efficient and effective deployment, ReTailor combines content-aware patch selection, which identifies a few informative patches per image for optimization at negligible cost to improve both quality and efficiency, with rolling residual bootstrap, which inherits sensor-related knowledge across the image stream to further improve quality. Extensive experiments on cross-sensor adaptation settings show that ReTailor mitigates cross-sensor degradation both effectively and efficiently. Moreover, we compare ReTailor with Low-Rank Adaptation (LoRA), a widely used method for fine-tuning, to demonstrate the advantages of our approach. We further evaluate the core principle of ReTailor on two additional remote sensing tasks: super-resolution pansharpening and image segmentation, where it consistently improves performance. In general, ReTailor provides a highly effective and efficient strategy for deploying pansharpening models under real-world cross-sensor degradation, while also offering useful insights into managing cross-sensor degradation in other remote sensing tasks. Code will be released at https://github.com/JingZe-Liang/ReTailor .

Figures & tables

Figure 1: Comparison of cross-sensor adaptation paradigms. Existing methods (top) trade fusion quality against efficiency and ground-truth (GT) requirements, whereas ReTailor (bottom) achieves high-quality, real-time adaptation without GT while preserving the pretrained fusion prior.

Figure 2: Empirical overview of ReTailor. (a) Pairwise cosine similarity between residual embeddings of independently adapted FusionNet Feature Tailors on 20 target images per transfer (mean $\pm$ std over 190 pairs). (b) Fusion quality (HQNR, $1-D_{\lambda}$ , $1-D_{s}$ ; higher is better) on QB $\rightarrow$ GF2 (upper axes) and WV3 $\rightarrow$ WV2 (lower axes); the ReTailor-family curve takes the best mean of FT-only and ReTailor over both backbones (the HQNR and $1-D_{\lambda}$ values are those of ReTailor FusionNet alone, the best method in Table 1 ). (c) End-to-end per-image duration of ZS-Pan [ 9 ] , LoRA [ 25 ] , and ReTailor (FusionNet); smaller is faster.

Figure 3: Deployment pipeline of ReTailor. Content-aware selection picks informative PAN/LRMS patches. The frozen Feature Extractor produces latent features $\mathbf{Z}$ , the trainable Feature Tailor adds a residual correction to give $\mathbf{Z}^{*}$ , and the frozen Channel Mapper outputs the source and tailored predictions. Together with the observations and target-sensor modulation transfer function (MTF) priors, these form the spectral, spatial, and frozen-source losses that update only the Tailor. The lower panel shows the three-layer Tailor.

Figure 4: Content-aware patch selection. PAN spatial variation and LRMS inter-band variation of the $N_{c}$ candidates are combined into a PAN-anchored score, discounted by regional occupancy, and greedily admitted until relative stopping ( $\tau$ ) yields $M^{t}$ patches for adaptation; inference still covers all $N_{c}$ candidates.

Figure 5: Rolling residual bootstrap. The seed image adapts $\phi_{\mathrm{init}}$ on all candidates to give $\phi^{0}$ . Each later image inherits $\phi^{t-1}$ , refines it on its selected patches, and passes $\phi^{t}$ on. Only Tailor parameters are transferred, and Adam is reinitialized for each image.

Figure 6: Qualitative comparison. For each transfer: fused image with two regions of interest (ROIs), magnified ROIs, and HQNR map (0–1, warmer is higher). Columns: the seven baselines with the highest HQNR (order of Table 1 ) and the FT-only and ReTailor variants.

Figure 7: End-to-end duration of ReTailor and LoRA. Mean per-image latency in the four backbone–transfer settings (smaller radius is faster); ReTailor’s latency includes patch selection, adaptation, and full-patch inference.

Method	aAcc $\uparrow$	mIoU $\uparrow$	mAcc $\uparrow$	Bldg. IoU $\uparrow$	Bldg. Acc. $\uparrow$
Source-only	99.10	71.67	78.20	44.25	56.76
Head-only	99.23	74.74	80.64	50.26	61.56
Feature Tailor	99.57	85.09	90.97	70.61	82.14

Table 4: CrossEarth Aerial-to-Satellite segmentation [ 21 , 27 ] (%). Best in bold, second-best underlined.

Figure 8: Qualitative CrossEarth segmentation [ 21 , 27 ] . First eight Satellite test samples; columns: input, ground truth, Source-only, Head-only, Feature Tailor (white: building).

Figure 9: Adaptation time of the three selectors. Bars: mean per-image time (patch selection plus Tailor optimization); numbers inside: mean selected patches (Table 5 ). The vertical axis starts at 0.12 s.

Variant	HQNR $\uparrow$	$D_{\lambda}\downarrow$	$D_{s}\downarrow$
PAN-only-8	$\underline{0.8794\pm 0.0502}$	$\underline{0.0824\pm 0.0409}$	$\bm{0.0422\pm 0.0158}$
LRMS-only-8	$0.8675\pm 0.0629$	$0.0910\pm 0.0504$	$0.0466\pm 0.0201$
LRMS-anchored-8	$0.8688\pm 0.0590$	$0.0896\pm 0.0469$	$0.0466\pm 0.0191$
Content-aware-8	$\bm{0.8977\pm 0.0411}$	$\bm{0.0612\pm 0.0408}$	$\underline{0.0438\pm 0.0138}$

Table 6: Ablation of evidence composition. QB

\rightarrow

GF2, FusionNet, eight patches per variant, per-image cold start; best in bold, second-best underlined.

Variant	HQNR $\uparrow$	$D_{\lambda}\downarrow$	$D_{s}\downarrow$
WV3 $\rightarrow$ WV2
Random-8	$0.9075\pm 0.0259$	$0.0463\pm 0.0217$	$\underline{0.0485\pm 0.0150}$
No-discount-8	$\underline{0.9083\pm 0.0236}$	$\underline{0.0428\pm 0.0235}$	$0.0512\pm 0.0140$
Content-aware-8	$\bm{0.9186\pm 0.0261}$	$\bm{0.0385\pm 0.0241}$	$\bm{0.0446\pm 0.0128}$
QB $\rightarrow$ GF2
Random-8	$0.8871\pm 0.0574$	$0.0763\pm 0.0458$	$\bm{0.0404\pm 0.0179}$

Table 7: Ablation of regional discounting. FusionNet, eight patches per variant, per-image cold start; best in bold, second-best underlined within each transfer block. Random-8 and Content-aware-8 replicate Table 5 .

Backbone	Transfer	Selection	HQNR $\uparrow$	$D_{\lambda}\downarrow$	$D_{s}\downarrow$
FusionNet	WV3 $\rightarrow$ WV2	Random-8	0.9266	0.0579	0.0165
	WV3 $\rightarrow$ WV2	Content-aware-8	0.9513	0.0392	0.0098
	QB $\rightarrow$ GF2	Random-8	0.8211	0.1378	0.0476
	QB $\rightarrow$ GF2	Content-aware-8	0.8751	0.1012	0.0263
U2Net	WV3 $\rightarrow$ WV2	Random-8	0.8358	0.1049	0.0663
	WV3 $\rightarrow$ WV2	Content-aware-8	0.8808	0.0887	0.0334

Table 8: Patch selection on large images. One

4{,}000\times 4{,}000

image each from WV2 and GF2, eight patches per method, per-image cold start; better in bold.

Backbone	Transfer	Initialization	HQNR $\uparrow$	$D_{\lambda}\downarrow$	$D_{s}\downarrow$	Adaptation time (s) $\downarrow$
FusionNet	WV3 $\rightarrow$ WV2	Cold-start	0.9075 $\pm$ 0.0259	0.0463 $\pm$ 0.0217	0.0485 $\pm$ 0.0150	0.2131 $\pm$ 0.0824
	WV3 $\rightarrow$ WV2	Rolling	0.9249 $\pm$ 0.0417	0.0317 $\pm$ 0.0192	0.0452 $\pm$ 0.0266	0.1974 $\pm$ 0.0581
	QB $\rightarrow$ GF2	Cold-start	0.8871 $\pm$ 0.0574	0.0763 $\pm$ 0.0458	0.0404 $\pm$ 0.0179	0.1488 $\pm$ 0.0363
	QB $\rightarrow$ GF2	Rolling	0.9074 $\pm$ 0.0212	0.0260 $\pm$ 0.0212	0.0683 $\pm$ 0.0186	0.1315 $\pm$ 0.0447
U2Net	WV3 $\rightarrow$ WV2	Cold-start	0.9200 $\pm$ 0.0460	0.0583 $\pm$ 0.0443	0.0232 $\pm$ 0.0051	0.2408 $\pm$ 0.0381
	WV3 $\rightarrow$ WV2	Rolling	0.9249 $\pm$ 0.0408	0.0542 $\pm$ 0.0415	0.0221 $\pm$ 0.0045	0.2327 $\pm$ 0.0195

Table 9: Ablation of rolling residual bootstrap. Both arms use Random-8; better in bold. Time covers patch selection and Tailor optimization.

Transfer	Latent similarity	Residual similarity
WV3 $\rightarrow$ WV2	0.6782 $\pm$ 0.2995	0.8537 $\pm$ 0.1107
QB $\rightarrow$ GF2	0.8684 $\pm$ 0.1089	0.9383 $\pm$ 0.0427
Aggregated	0.7733 $\pm$ 0.2446	0.8960 $\pm$ 0.0940

Table 10: Cosine similarity between target images. Mean

\pm

std over 190 image pairs per transfer (380 pooled) with independently cold-started Tailors; higher in bold.

Transfer	Mean HQNR $\uparrow$	Var. ( $10^{-6}$ )	Std	Min–Max
QB $\rightarrow$ GF2	0.8999	7.50	0.0027	0.8959–0.9072
WV3 $\rightarrow$ WV2	0.9278	2.60	0.0016	0.9255–0.9312

Table 11: Seed-image resampling. Twenty sequences per transfer, each with a different test image as seed and the others randomly permuted; the backbone is reloaded for each sequence.

Transfer	Length	HQNR $\uparrow$	$D_{\lambda}\downarrow$	$D_{s}\downarrow$
QB $\rightarrow$ GF2	20	$0.8976\pm 0.0264$	$0.0339\pm 0.0292$	$0.0706\pm 0.0218$
QB $\rightarrow$ GF2	100	$0.8978\pm 0.0253$	$0.0305\pm 0.0279$	$0.0736\pm 0.0226$
WV3 $\rightarrow$ WV2	20	$0.9245\pm 0.0394$	$0.0335\pm 0.0198$	$0.0438\pm 0.0233$
WV3 $\rightarrow$ WV2	100	$0.9238\pm 0.0391$	$0.0328\pm 0.0215$	$0.0452\pm 0.0222$

Table 12: Long-sequence stability (FusionNet). The standard 20-image stream versus 100-image sequences resampled from the same images.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Study	Schedule	Optimizer	LR	WD
FusionNet/U2Net	30 epochs (seed image)	Adam	$10^{-4}$	$10^{-5}$
FusionNet/U2Net	10 epochs (later images)	Adam	$10^{-4}$	$10^{-5}$
LoRA	10 epochs	Adam	$10^{-4}$	$10^{-5}$
G-ZAP	500 epochs	Adam	$5\times 10^{-4}$	$10^{-5}$
CrossEarth A2S	5,000 iterations	AdamW	$10^{-4}$	$0.05$

Appendix

Table 13: Optimization settings. Epochs are per target image; each epoch takes

\lceil M^{t}/8\rceil

steps (later images), 8 steps (seed image), or 1 step (LoRA). LR: learning rate; WD: weight decay.

Model/task	Insertion point	Width $d$	Params
FusionNet	res4 $\rightarrow$ conv3	$32$	$221{,}472$
U2Net	s2block4 $\rightarrow$ to_hrms	$32$	$221{,}472$
G-ZAP	INR query $\rightarrow$ decoder	$48$	$498{,}096$
CrossEarth A2S	pixel decoder $\rightarrow$ Mask2Former head [ 10 ]	$256$	$14{,}158{,}080$

Appendix

Table 14: Feature Tailor configurations. Insertion point, feature width

d

, and trainable parameters for each model.

Figure 10: HQNR for Tailor placements on WV3. Baseline: no Tailor; pre-CM/post-CM: before/after the Channel Mapper.

Figure 11: HQNR versus number of selected patches (WV3

\rightarrow

WV2).

(a)	Whole image	Fixed ( $M$ )	ReTailor ( $M^{t}$ )
CNN	$\mathcal{O}(EHW)$	$\mathcal{O}(EMhw)$	$\mathcal{O}(EM^{t}hw)$
Attention	$\mathcal{O}\!\left(E(HW)^{2}\right)$	$\mathcal{O}\!\left(EM(hw)^{2}\right)$	$\mathcal{O}\!\left(EM^{t}(hw)^{2}\right)$
(b)		Whole/ReTailor	Fixed/ReTailor
CNN		$\dfrac{N_{c}}{M^{t}}$	$\dfrac{M}{M^{t}}$
Attention		$\dfrac{N_{c}^{2}}{M^{t}}$	$\dfrac{M}{M^{t}}$

Appendix

Table 15: Complexity of Tailor updates. (a) Update complexity; (b) compression ratios of ReTailor (attention assumes global self-attention).

Candidate evaluation	Latency (ms) $\downarrow$	Speedup $\uparrow$
Sequential	$5.28\pm 0.84$	$1.00\times$
Parallel	$\mathbf{1.99\pm 0.28}$	$\mathbf{2.65\times}$

Appendix

Table 16: Selector latency on QB

\rightarrow

GF2. Time of a single selection, mean

\pm

std over images.

Pansharpening

See all 14 papers

Apr 10, 2026

Fast Model-guided Instance-wise Adaptation Framework for Real-world Pansharpening with Fidelity Constraints

Pansharpening aims to generate high-resolution multispectral (HRMS) images by fusing low-resolution multispectral (LRMS) and high-resolution panchromatic (PAN) images while preserving both spectral and spatial information. Although deep learning (DL)-based pansharpening methods achieve impressive performance, they require high training cost and large datasets, and often degrade when the test distribution differs from training, limiting generalization. Recent zero-shot methods, trained on a single PAN/LRMS pair, offer strong generalization but suffer from limited fusion quality, high computational overhead, and slow convergence. To address these issues, we propose FMG-Pan, a fast and generalizable model-guided instance-wise adaptation framework for real-world pansharpening, achieving both cross-sensor generality and rapid training-inference. The framework leverages a pretrained model to guide a lightweight adaptive network through joint optimization with spectral and physical fidelity constraints. We further design a novel physical fidelity term to enhance spatial detail preservation. Extensive experiments on real-world datasets under both intra- and cross-sensor settings demonstrate state-of-the-art performance. On the WorldView-3 dataset, FMG-Pan completes training and inference for a 512x512x8 image within 3 seconds on an RTX 3090 GPU, significantly faster than existing zero-shot methods, making it suitable for practical deployment.

Zhiqi Yang, Jin-Liang Xiao, Shan Yin +2

University of Electronic Science and Technology of China, Chengdu, China · National Research Council, Institute of Methodologies for Environmental Analysis (CNR-IMAA), Tito, Italy

Pansharpening Multi-Source Image Fusion Remote Sensing Image Reconstruction Test-Time Adaptation Remote Sensing Image Super-Resolution

Ask PDF

Feb 28, 2026

Cross-Scale Pansharpening via ScaleFormer and the PanScale Benchmark

Pansharpening aims to generate high-resolution multi-spectral images by fusing the spatial detail of panchromatic images with the spectral richness of low-resolution MS data. However, most existing methods are evaluated under limited, low-resolution settings, limiting their generalization to real-world, high-resolution scenarios. To bridge this gap, we systematically investigate the data, algorithmic, and computational challenges of cross-scale pansharpening. We first introduce PanScale, the first large-scale, cross-scale pansharpening dataset, accompanied by PanScale-Bench, a comprehensive benchmark for evaluating generalization across varying resolutions and scales. To realize scale generalization, we propose ScaleFormer, a novel architecture designed for multi-scale pansharpening. ScaleFormer reframes generalization across image resolutions as generalization across sequence lengths: it tokenizes images into patch sequences of the same resolution but variable length proportional to image scale. A Scale-Aware Patchify module enables training for such variations from fixed-size crops. ScaleFormer then decouples intra-patch spatial feature learning from inter-patch sequential dependency modeling, incorporating Rotary Positional Encoding to enhance extrapolation to unseen scales. Extensive experiments show that our approach outperforms SOTA methods in fusion quality and cross-scale generalization. The datasets and source code are available at https://github.com/caoke-963/ScaleFormer.

Ke Cao, Xuanhua He, Xueheng Li +7

HFIPS, Chinese Academy of Sciences · University of Science and Technology of China · The Hong Kong University of Science and Technology +3

Pansharpening Multi-Source Image Fusion Remote Sensing Remote Sensing Benchmarks Vision Transformer

Ask PDF

Mar 4, 2026

Universal Pansharpening Model

Pansharpening generates the high-resolution multi-spectral (MS) image by integrating spatial details from a texture-rich panchromatic (PAN) image and spectral attributes from a low-resolution MS image. Existing methods are predominantly satellite-specific and scene-dependent, which severely limits their generalization across heterogeneous sensors and varied scenes, thereby reducing their real-world practicality. To address these challenges, we present UniPS, a universal pansharpening model for satellite-agnostic and scene-robust fusion. Specifically, we introduce a modality-interleaved transformer that learns band-wise modal specializations to form reversible spectral affine bases, mapping arbitrary-band MS into a unified latent space via tensor multiplication. Building upon this, we construct a latent diffusion bridge model to progressively evolve latent representations, and incorporate bridge posterior sampling to couple latent diffusion with pixel-space observations, enabling stable and controllable fusion. Furthermore, we devise infinite-dimensional pixel-to-latent interaction mechanisms to comprehensively capture the cross-domain dependencies between PAN observations and MS representations, thereby facilitating complementary information fusion. In addition, to support large-scale training and evaluation, we construct a comprehensive pansharpening benchmark, termed PSBench, consisting of worldwide MS and PAN image pairs from multiple satellites across diverse scenes. Extensive experiments verify that UniPS consistently outperforms state-of-the-art methods, exhibiting superior generalization and robustness across a wide range of tasks.

Hebaixu Wang, Jing Zhang, Haonan Guo +4

School of Electronic Information, Wuhan University, Hubei, 430072, Wuhan, China. · Zhongguancun Academy, Beijing, 100094,China. · School of Computer Science, Wuhan University, Hubei, 430072, Wuhan, China. +2

Pansharpening Multi-Source Image Fusion Remote Sensing

Ask PDF

Abstract

Pansharpening

Explore similar work

Fast Model-guided Instance-wise Adaptation Framework for Real-world Pansharpening with Fidelity Constraints

Cross-Scale Pansharpening via ScaleFormer and the PanScale Benchmark

Universal Pansharpening Model