cs.AINov 2, 2026

Bridging Low-Level Image Fusion and High-Level Semantic Understanding

Authors: Guangyuan Dong, Lingdong Shen, Xudong Zhang, Haodong Jing, Haoyu Wang, Zihao Li, Zonghao Ying, Ziyu Song, 

Organizations: NUS · Peking University · Xi’an Jiaotong University · Nankai University · 

Abstract

Remote sensing image fusion and semantic interpretation are usually optimized separately, so reconstruction quality need not preserve downstream retrieval utility. We propose LANCE, which uses structured language as an explicit, indexable semantic state. A frozen vision-language model maps the input to a closed-vocabulary text record; a language-conditioned generator fuses the imagery; and a dense verifier with a differentiable continuous summarizer re-describes the output. Text-text consistency and anti-collapse contrastive text-mask losses with negative queries close the loop without dense annotations or VLM fine-tuning. A variational EM formulation yields a per-step contraction result under regularity assumptions, distinguishing semantic faithfulness from verifier overfitting. Across seven test sets on GF2, QuickBird, WorldView-III, and WorldView-II, LANCE improves reduced-scale reference metrics by up to 11.7% (median 2.1%) over the strongest learning baseline. Independent evaluation using a held-out VLM and a supervised aerial detector yields gains of +6.4 mAP and +0.072 open-vocabulary F1 over the closest competitor. Text perturbations, cross-verifier transfer, and pseudo-RGB ablations examine the mechanisms underlying these gains. The framework connects image fusion to verifiable semantic representations for multimodal retrieval and knowledge management.

Figures & tables

Explore similar work

CardsList
  1. A Unified Foundation Model for All-in-One Multi-Modal Remote Sensing Image Restoration and Fusion with Language Prompting

    Apr 7, 2026Yongchuan Cui, Peng LiuAll-in-One Image RestorationImage Restoration

  2. Win-Win Fusion: Multi-Modal Image Fusion with Visual-Semantic Collaboration

    Oct 9, 2026Hao Zhang, Xuhui Zuo, Jiayi MaMulti-Source Image Fusion

  3. VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing

    Dec 12, 2025Emanuel Sánchez Aimar, Gulnaz Zhambulova, Fahad Shahbaz Khan +2Multimodal UnderstandingCross-Modal Learning