Remote sensing image fusion and semantic interpretation are usually optimized separately, so reconstruction quality need not preserve downstream retrieval utility. We propose LANCE, which uses structured language as an explicit, indexable semantic state. A frozen vision-language model maps the input to a closed-vocabulary text record; a language-conditioned generator fuses the imagery; and a dense verifier with a differentiable continuous summarizer re-describes the output. Text-text consistency and anti-collapse contrastive text-mask losses with negative queries close the loop without dense annotations or VLM fine-tuning. A variational EM formulation yields a per-step contraction result under regularity assumptions, distinguishing semantic faithfulness from verifier overfitting. Across seven test sets on GF2, QuickBird, WorldView-III, and WorldView-II, LANCE improves reduced-scale reference metrics by up to 11.7% (median 2.1%) over the strongest learning baseline. Independent evaluation using a held-out VLM and a supervised aerial detector yields gains of +6.4 mAP and +0.072 open-vocabulary F1 over the closest competitor. Text perturbations, cross-verifier transfer, and pseudo-RGB ablations examine the mechanisms underlying these gains. The framework connects image fusion to verifiable semantic representations for multimodal retrieval and knowledge management.
Figures & tables
Figure 1. Overview of Lance as a three-stage semantic closed loop. Stage I (semantic anchoring, A ) maps the pseudo-RGB projection of the low-quality input to a structured record T retrieved from a closed prompt bank and to its frozen text embedding eT . Stage II (language-guided fusion, G ) generates the fused output Y^ from MS and PAN, injecting eT at decoder levels 2–4 through joint attention and FiLM modulation. Stage III (semantic closed-loop feedback) re-interprets Y^ with the shared frozen SigLIP 2 encoder into dense query responses M^ ( D ), re-describes them into T^ and eT^ with the differentiable summarizer ( R ), and returns the gradients of the anti-collapse consistencies to G . Independent evaluators (RemoteCLIP, supervised detector) sit outside the loop to prevent verifier overfitting. A three-stage closed loop links semantic anchoring, language-guided image fusion, and semantic closed-loop feedback composed of dense responses, a differentiable re-description and anti-collapse consistency losses. The feedback stage sends gradients back to the fusion generator, while independent evaluators remain outside the training loop.
Vocabulary
∣V∣
Examples
Vs (scenes)
12
urban, port, farmland, coastal, water, …
Vo (objects)
32
road, building, bridge, ship, dock, vegetation, …
Vℓ (layouts)
8
linear, clustered, parallel_strips, …
Table 1. Closed vocabularies shared across all benchmarks, compiled before training from public RS label taxonomies (DOTA, AID). ∣V∣ values are kept dataset-agnostic to avoid leakage.
Hyperparameter
Value
Hyperparameter
Value
λTT/λTM/β
0.2 / 0.3 / 0.01
η (SSIM)
0.5
α (neg. weight)
0.5
K− neg. /img
8
τ (mask temp.)
0.07
μt percentile
80%
prompt bank NB
4096
∣Vs,o,ℓ∣
12, 32, 8
U-Net levels / ch.
5 / 32
res. blocks / level
3, 1, 1, 1, 1
warm-up / ramp ep.
50 / 150
random seeds
{0,1,2}
Table 2. All hyperparameters used by Lance , fixed across datasets.
Data
Metric
Traditional
Learning-based fusion
Ours
FS
BDSD-PC
ADKNet
SSAFF
SSUN-Net
FAME
RAMSF
PanComplex
Lance
GF2
RST
PSNR ( ↑ )
34.40
34.16
39.80
39.86
39.02
39.79
40.23
39.85
41.17
ERGAS ( ↓ )
1.6228
1.7000
0.8268
0.8180
0.8983
0.8189
0.7457
0.8194
0.6617
SAM ( ↓ )
1.6891
1.7230
0.8866
0.8796
1.1336
0.9012
0.8089
0.8830
0.7146
Q2n ( ↑ )
0.8914
0.8856
0.9731
0.9744
0.9703
0.9739
0.9777
0.9744
0.9830
FST
Ds ( ↓ )
0.0523
0.0560
0.0252
0.0379
0.0233
0.0493
0.0320
0.0374
0.0246
Table 3. Quantitative fusion results on four sensors / seven test sets (mean over the test set). ↑ / ↓ indicate higher/lower-is-better. Bold : best; underline : second-best. The Lance column is highlighted in light blue. WV-II reports reference-style metrics only because no FST protocol is defined. Compute is measured at the actual test resolution (256 × 256).
Figure 2. Qualitative comparison on (a) a 4-band GF2 scene (river, highway bridge, farmland) and (b) an 8-band WV-III coastal town (built-up ridge, shoreline road, beaches, forested slopes). Odd rows: fused outputs of nine methods plus reference; even rows: per-pixel absolute residuals against the reference. The structured text retrieved by A is printed between the rows of each scene, making the language state of each output explicit and inspectable. Two qualitative comparison grids show fused outputs, reference images, absolute residuals, and structured text records for a GF2 river-and-farmland scene and a WorldView-III coastal-town scene across nine methods.
Table 4. Independent downstream semantic evaluation on fused WV-III outputs. (a) Detection [email protected] with a YOLOv8-DOTA model trained on disjoint aerial imagery. (b) Open-vocabulary localization F1 with a held-out RemoteCLIP verifier ( Liu et al., 2024 ) . (c) Scene retrieval R@1 on a 4096-image AID index ( Xia et al., 2017 ) . No evaluator is used during training.
Method
SigLIP-2
RemoteCLIP
Det. mAP
Gap (%)
Lance, no LTM
0.961
0.892
46.7
7.18
Lance, no negatives
0.972
0.913
48.4
6.07
Lance, full
0.986
0.972
50.5
1.42
Table 5. Held-out verifier transfer on WV-III. Models are trained against SigLIP 2 and evaluated with three independent verifiers. “Gap” is the relative drop from in-loop SigLIP 2 to RemoteCLIP; lower means less verifier-specific overfitting.
Text condition (example query)
Q2n ( ↑ )
OV-F1 ( ↑ )
Correct text ( urban, road, building )
0.9166
0.654
Paraphrase ( city, streets, rooftops )
0.9160
0.651
Synonym-replaced ( transport corridor, asphalt )
0.9152
0.647
Object dropped (remove building )
0.9141
0.622
Wrong objects ( water, ship, dock )
0.9082
0.481
Shuffled across batch
0.9047
0.444
Table 6. Text perturbation controls on WV-III. Q2n is fusion fidelity; OV-F1 is held-out RemoteCLIP localization (§ 5.3 ). Correct text helps both; random / shuffled / contradicting text hurts the semantic metric while leaving fidelity nearly intact.
Setting
TC
D
R
LTT
LTM
Neg
Q2n
HQNR
Lrec only
×
×
×
×
×
×
0.9041
0.8856
+ TC
✓
×
×
×
×
×
0.9089
0.9034
+ dense D
✓
✓
×
×
×
×
0.9101
0.9172
+ continuous R
✓
✓
✓
×
×
×
0.9112
0.9218
+ LTT
✓
✓
✓
✓
×
×
0.9126
0.9301
+ LTM (pos. only)
✓
✓
✓
✓
✓
×
0.9148
0.9388
Table 7. Component-orthogonal ablation on WV-III. Each row toggles one factor relative to the row above. TC: text conditioning; D : dense interpreter; R : continuous summarizer; LTT / LTM : semantic losses; Neg: hard-negative term ( 17 ).
Mechanism
Attn
Mod
Q2n
HQNR
OV-F1
none
×
×
0.9101
0.9240
0.575
Attn only
✓
×
0.8892
0.9221
0.588
Mod only
×
✓
0.8928
0.9238
0.595
Attn + Mod (Ours)
✓
✓
0.9166
0.9439
0.654
Table 8. Ablation on language injection mechanisms (WV-III).
Projection
grad. W
Q2n
HQNR
OV-F1
Fixed sensor RGB
–
0.9134
0.9388
0.628
Learned W (fusion only)
×
0.9151
0.9420
0.642
Learned W (Ours)
✓
0.9166
0.9439
0.654
Random init., no clip
✓
0.9088
0.9311
0.591
Table 9. Effect of the pseudo-RGB projection W . “grad. W ” indicates whether semantic losses update W .
λTT\λTM
0.1
0.2
0.3
0.4
0.6
0.1
0.9136
0.9148
0.9152
0.9143
0.9128
0.2
0.9151
0.9162
0.9166 ⋆
0.9159
0.9144
0.4
0.9148
0.9156
0.9161
0.9152
0.9133
Table 10. Sensitivity of Q2n to loss weights (λTT,λTM) on WV-III. The landscape is flat within a ±50% neighborhood of the default (0.2,0.3)⋆ .
Figure 3. Efficiency vs. fusion performance on WV-III (values of Table 3 , 256×256 ). Left: Q2n vs. FLOPs (G); right: HQNR vs. inference time (ms). Marker area ∝ parameter count. Lance (top left) attains the best quality at near-best efficiency, including the 14 ms SigLIP 2/summarizer overhead. Two scatter plots compare fusion quality against computational cost. Lance lies on the quality-efficiency Pareto frontier in both Q2n versus FLOPs and HQNR versus inference time.
Figure 4. Mask diagnostics on WV-III. (a) Positive query masks ( water , vegetation , building , ship ; amber overlay, p^ = soft area) localize to the corresponding content, whereas the same queries on tiles without it yield uniformly low masks. (b) Mask area during training: the positive-only loss inflates towards all-one masks, whereas the hard-negative term in ( 17 ) keeps p^ stable. Positive-query masks highlight water, vegetation, buildings and moored vessels in WorldView-III tiles while negative-query masks remain low. A training curve shows stable mask area with hard negatives and collapse without them.
Figure 5. Failure case on GF2. The scene is an active construction site whose defining content (excavated soil, temporary site buildings, unpaved access road) has no token in Vo ; Lance falls back to the closest in-vocabulary record ( urban ; building , road ), producing a physically plausible but semantically incomplete reconstruction. Right: manual annotation of the true content (limitation discussed in § 6 ). A GF2 construction-site example shows the interpolated input, the fused output with its incomplete structured record, and a manually annotated overlay marking excavated soil, temporary site buildings and an unpaved access road that the closed vocabulary cannot name.
Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, China · School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing, China