A Unified Foundation Model for All-in-One Multi-Modal Remote Sensing Image Restoration and Fusion with Language Prompting

Yongchuan Cui; Peng Liu

cs.CVApr 7, 2026

A Unified Foundation Model for All-in-One Multi-Modal Remote Sensing Image Restoration and Fusion with Language Prompting

Authors: Yongchuan Cui, Peng Liu

Organizations: Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, China · School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing, China

Abstract

Remote sensing imagery suffers from clouds, haze, noise, resolution limits, and sensor heterogeneity. Existing restoration and fusion approaches train separate models per degradation type. In this work, we present Language-conditioned Large-scale Remote Sensing restoration model (LLaRS), the first unified foundation model for multi-modal and multi-task remote sensing low-level vision. LLaRS employs Sinkhorn-Knopp optimal transport to align heterogeneous bands into semantically matched slots, routes features through three complementary mixture-of-experts layers (convolutional experts for spatial patterns, channel-mixing experts for spectral fidelity, and attention experts with low-rank adapters for global context), and stabilizes joint training via step-level dynamic weight adjustment. To train LLaRS, we construct LLaRS1M, a million-scale multi-task dataset spanning eleven restoration and enhancement tasks, integrating real paired observations and controlled synthetic degradations with diverse natural language prompts. Experiments show LLaRS consistently outperforms seven competitive models, and parameter-efficient finetuning experiments demonstrate strong transfer capability and adaptation efficiency on unseen data. Repo: https://github.com/yc-cui/LLaRS

Figures & tables

Figure 1: LLaRS takes a degraded remote sensing image and a natural-language prompt as input, and produces the restored image.

Figure 2: Overall architecture of LLaRS.

Figure 3: Entropy-regularized channel-to-slot matching.

Figure 4: Geographic distribution of LLaRS1M sampling locations.

Figure 5: ( Left ) Per-task sample counts and number of prompts per task in LLaRS1M. ( Right ) T-SNE of embedded prompts.

Figure 6: Word cloud of all prompts in LLaRS1M.

Method	CR		PS		SR		HR		STF		DN
Method	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM
MPRNet [ 40 ]	29.06	0.8303	32.31	0.8928	43.89	0.9835	19.57	0.7032	32.13	0.8962	42.38	0.9609
Restormer [ 66 ]	27.24	0.7601	34.32	0.9346	43.96	0.9840	22.58	0.7269	32.52	0.8969	43.66	0.9688
GridFormer [ 57 ]	26.60	0.6762	33.26	0.9207	43.63	0.9831	22.07	0.7220	32.49	0.8973	43.05	0.9656
PromptIR [ 45 ]	26.88	0.7636	34.46	0.9367	43.91	0.9840	22.34	0.7218	32.21	0.8973	43.82	0.9695
AMIR [ 62 ]	28.63	0.7628	32.90	0.9119	43.60	0.9822	21.56	0.7154	31.91	0.8945	43.28	0.9665

Table 1: Experimental results: pixel fidelity and structural similarity. Best result. Second-best result.

Method	CR		PS		SR		HR		STF		DN
Method	SAM	ERGAS	SAM	ERGAS	SAM	ERGAS	SAM	ERGAS	SAM	ERGAS	SAM	ERGAS
MPRNet [ 40 ]	0.1635	45.4368	0.0902	5.9270	0.0186	1.4666	0.0493	6.6063	0.0747	4.4056	0.0394	1.9584
Restormer [ 66 ]	0.1943	87.9132	0.0742	4.5972	0.0183	1.4408	0.0452	5.0565	0.0719	4.2993	0.0343	1.6787
GridFormer [ 57 ]	0.2519	44.1388	0.0894	5.1833	0.0211	1.5481	0.0458	5.3097	0.0725	4.2417	0.0362	1.8049
PromptIR [ 45 ]	0.2000	26.2611	0.0754	4.4878	0.0192	1.5149	0.0457	5.1578	0.0705	4.3680	0.0336	1.6458
AMIR [ 62 ]	0.1833	98.1772	0.0937	5.3990	0.0198	1.5169	0.0498	5.4899	0.0752	4.4343	0.0367	1.7509

Table 2: Experimental results: spectral and radiometric fidelity. Best result. Second-best result.

Figure 7: Model predictions and error maps for denoising.

Figure 8: Model predictions and error maps for destriping.

Figure 9: Model predictions and error maps for spatiotemporal fusion.

Figure 10: Comparison of t-SNE task feature separability across different models.

Figure 11: Relationship between trainable parameter ratio and average task performance for different fine-tuning methods.

Config	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$	Params(M)	FLOPs(G)
Baseline	32.76	0.8371	0.1399	14.6733	11.85	96.1
+ ConvMoE	34.92	0.8900	0.0882	9.4069	75.43	96.3
+ ConvMoE + MoCE	35.08	0.8942	0.0770	5.7072	77.15	96.8
+ ConvMoE + MoCE + MoT	37.60	0.9172	0.0644	4.4069	83.64	109.6

Table 3: Ablation study on mixture-of-experts modules.

Figure 12: Evolution of channel-to-slot transport for eleven tasks. ( Left ) Top-5 selected slots with the largest channel-to-slot transport mass. ( Right ) The mass flow across channels to slots (the red , green, and blue bands are encoded with colors).

Figure 13: Evolution of per-task weight changing.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Dataset	Source	Input	GSD	Size	Bands	Samples
SR	OLI2MSI [ 56 ]	Landsat-8 / Sentinel-2	MS	30 m / 10 m	$160\times 160$	3	5,325
SR	SEN2VEN $\mu$ S [ 42 ]	Sentinel-2 / VEN $\mu$ S	MS	10–20 m / 5 m	$128\times 128$ / $256\times 256$	4	132,955
Pan	PansharpRSData [ 41 ]	GF-1 / IKONOS / QB / WV-2/3/4	MS	1–4 m	$256\times 256$	4/8	2,270
	PansharpRSData [ 41 ]	GF-1 / IKONOS / QB / WV-2/3/4	PAN	0.3–1 m	$1024\times 1024$	1	2,270
	PanCollection [ 14 ]	GF-2 / QB / WV-2/3	PAN	0.3–0.8 m	$64\times 64$ / $256\times 256$	1	51,928
	PanCollection [ 14 ]	GF-2 / QB / WV-2/3	MS	1.2–3.2 m	$64\times 64$ / $256\times 256$	4/8	51,928

Appendix

Table 4: Basic information of LLaRS1M sources.

Task	Dataset	Modality	Format	dtype	Mean	Std	Min	Max
SR	OLI2MSI [ 56 ]	MS	TIFF	float32	0.0736	0.0150	0.0336	0.1492
SR	SEN2VEN $\mu$ S [ 42 ]	MS	PT	int16	1049.70	398.02	91.57	4405.95
Pan	PansharpRSData [ 41 ]	MS	MAT	uint16	346.48	163.35	106.80	1127.70
	PansharpRSData [ 41 ]	PAN	MAT	uint16	335.57	159.21	145.75	1099.51
	PanCollection [ 14 ]	PAN	H5	float64	295.78	149.00	124.87	951.15
	PanCollection [ 14 ]	MS	H5	float64	308.43	129.09	67.73	935.95

Appendix

Table 5: Pixel statistics of LLaRS1M sources.

Task	Prompt
CR	Clear the clouds from this satellite image to reveal the ground.
SR	Increase the image’s resolution for a more detailed view.
PS	Combine panchromatic and multispectral images to produce a higher-resolution color image.
HR	Dehaze this image to reveal the underlying surface features.
DN	Enhance the image clarity by reducing the noise level.
DB	Deblur the image to recover fine structure and detail.

Appendix

Table 6: Fixed text prompts used at test time for each task.

Figure 15: Contribution analysis of text prompt and OT-based channel alignment.

$E$	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$	Params(M)	FLOPs(G)
$0$	32.76	0.8371	0.1399	14.6733	11.85	96.1
$2$	34.04	0.8894	0.0853	6.2959	42.31	109.2
$4$	35.00	0.8991	0.0763	4.9168	56.08	109.3
$8$	37.60	0.9172	0.0644	4.4069	83.64	109.6
$16$	35.75	0.9001	0.0752	7.2115	138.74	110.1

Appendix

Table 7: Effect of the number of experts

E

on model performance.

Strategy	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$	Time/step (ms)	Extra mem
Equal wgt	34.31	0.8696	0.1000	13.0818	542	–
PCGrad [ 63 ]	–	–	–	–	2884	+18.7 GB
MGDA [ 49 ]	–	–	–	–	2661	+21.4 GB
CAGrad [ 34 ]	–	–	–	–	2773	+23.8 GB
NashMTL [ 43 ]	–	–	–	–	2798	+13.9 GB
GradNorm [ 6 ]	34.56	0.8338	0.0997	12.0695	2841	+29.0 GB

Appendix

Table 8: Comparison of multi-task optimization strategies.

Method	Params(M)	FLOPs(G)	Time (ms)	Avg.PSNR $\uparrow$
MPRNet [ 40 ]	3.67	285.9	45	32.49
Restormer [ 66 ]	26.15	312.7	73	36.07
GridFormer [ 57 ]	34.18	742.1	390	35.24
PromptIR [ 45 ]	35.61	348.3	77	36.25
AMIR [ 62 ]	23.57	281.5	96	34.15
HOGformer [ 58 ]	30.00	426.9	308	35.33

Appendix

Table 9: Comparison of model efficiency.

Model	FT method	Total params	Train params	Train.Ratio
PromptIR	Full	35.61M	35.61M	100.00%
	LoRA	36.88M	1.27M	3.44%
	Adapter	56.90M	21.28M	37.41%
	DoRA	36.97M	1.36M	3.68%
	SSF	35.94M	329.7K	0.92%
	BitFit	35.61M	17.4K	0.05%

Appendix

Table 10: Parameter statistics of different fine-tuning methods for each model.

Figure 16: Model predictions and error maps for deblurring.

Figure 17: Model predictions and error maps for cloud removal.

Figure 18: Model predictions and error maps for pansharpening.

Figure 19: Model predictions and error maps for histogram equalization.

Figure 20: Model predictions and error maps for brightness enhancement.

Figure 21: Fine-tuning qualitative comparison for dehazing.

Figure 22: Fine-tuning qualitative comparison for super-resolution.

Figure 23: Fine-tuning qualitative comparison for SAR despeckling.

All-in-One Image Restoration

All-in-one image restoration trains unified models to remove diverse and co-occurring degradations while preserving scene content.

See all 87 papers

Task-Guided Prompting for Unified Remote Sensing Image Restoration
Apr 3, 2026Wenli Huang, Yang Wu, Xiaomeng Xin +3All-in-One Image Restoration Remote Sensing
School of Electronic and Information Engineering, Ningbo University of Technology, Ningbo, Zhejiang 315211, China · Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, China · University of Exeter, Exeter EX4 4PY, UK +1
Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks
Jun 9, 2026Miaoxin Cai, Guanqun Wang, Wei Zhang +6Multimodal Understanding Multimodal Grounding
National Key Laboratory of Science and Technology on Space-Born Intelligent Information Processing (SBIIP), Beijing Institute of Technology, Beijing 100081, China · Advanced Research Institute of Multidisciplinary Sciences, Beijing Institute of Technology, Beijing 100081, China · School of Mechatronical Engineering, Beijing Institute of Technology, Beijing 100081, China +5
OmniRSI: Taming Latent Diffusion for Unified Multimodal Remote Sensing Image Generation
Oct 11, 2026Hebaixu Wang, Jing Zhang, Haonan Guo +3Multimodal Generation Unified Multimodal Models
Wuhan University & Zhongguancun Academy

Apr 3, 2026J/K move · Enter open · S save

Task-Guided Prompting for Unified Remote Sensing Image Restoration

Wenli Huang, Yang Wu, Xiaomeng Xin, Zhihong Liu+2

School of Electronic and Information Engineering, Ningbo University of Technology, Ningbo, Zhejiang 315211, China · Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, China · University of Exeter, Exeter EX4 4PY, UK+1

All-in-One Image Restoration Remote Sensing Multi-Task Learning Remote Sensing Benchmarks

Open paper AskPDF arXiv

Remote sensing image restoration (RSIR) is essential for recovering high-fidelity imagery from degraded observations, enabling accurate downstream analysis. However, most existing methods focus on single degradation types within homogeneous data, restricting their practicality in real-world scenarios where multiple degradations often across diverse spectral bands or sensor modalities, creating a significant operational bottleneck. To address this fundamental gap, we propose TGPNet, a unified framework capable of handling denoising, cloud removal, shadow removal, deblurring, and SAR despeckling within a single, unified architecture. The core of our framework is a novel Task-Guided Prompting (TGP) strategy. TGP leverages learnable, task-specific embeddings to generate degradation-aware cues, which then hierarchically modulate features throughout the decoder. This task-adaptive mechanism allows the network to precisely tailor its restoration process for distinct degradation patterns while maintaining a single set of shared weights. To validate our framework, we construct a unified RSIR benchmark covering RGB, multispectral, SAR, and thermal infrared modalities for five aforementioned restoration tasks. Experimental results demonstrate that TGPNet achieves state-of-the-art performance on both unified multi-task scenarios and unseen composite degradations, surpassing even specialized models in individual domains such as cloud removal. By successfully unifying heterogeneous degradation removal within a single adaptive framework, this work presents a significant advancement for multi-task RSIR, offering a practical and scalable solution for operational pipelines. The code and benchmark will be released at https://github.com/huangwenwenlili/TGPNet.

Abstract

All-in-One Image Restoration

Explore similar work

Task-Guided Prompting for Unified Remote Sensing Image Restoration

Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks

OmniRSI: Taming Latent Diffusion for Unified Multimodal Remote Sensing Image Generation