cs.CVApr 4, 2026

When Does Multimodal AI Help? Diagnostic Complementarity of Vision-Language Models and CNNs for Spectrum Management in Satellite-Terrestrial Networks

Authors: Yuanhang Li

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The adoption of vision-language models (VLMs) for wireless network management is accelerating, yet no systematic understanding exists of where these large foundation models outperform lightweight convolutional neural networks (CNNs) for spectrum-related tasks. This paper presents the first diagnostic comparison of VLMs and CNNs for spectrum heatmap understanding in non-terrestrial network and terrestrial network (NTN-TN) cooperative systems. We introduce SpectrumQA, a benchmark comprising 108K visual question-answer pairs across four granularity levels: scene classification (L1), regional reasoning (L2), spatial localization (L3), and semantic reasoning (L4). Our experiments on three NTN-TN scenarios with a frozen Qwen2-VL-7B and a trained ResNet-18 reveal a clear taskdependent complementarity: CNN achieves 72.9% accuracy at severity classification (L1) and 0.552 IoU at spatial localization (L3), while VLM uniquely enables semantic reasoning (L4) with F1=0.576 using only three in-context examples-a capability fundamentally absent in CNN architectures. Chain-of-thought (CoT) prompting further improves VLM reasoning by 12.6% (F1: 0.209->0.233) while having zero effect on spatial tasks, confirming that the complementarity is rooted in architectural differences rather than prompting limitations. A deterministic task-type router that delegates supervised tasks to CNN and reasoning tasks to VLM achieves a composite score of 0.616, a 39.1% improvement over CNN alone. We further show that VLM representations exhibit stronger cross-scenario robustness, with smaller performance degradation in 5 out of 6 transfer directions. These findings provide actionable guidelines: deploy CNNs for spatial localization and VLMs for semantic spectrum reasoning, rather than treating them as substitutes.

Figures & tables

LevelTaskCNNVLMWinner
L1Severity (Acc.)72.9 ±\pm 2.1%0.6% † (84.6% ∗ )CNN
L2Regional (Acc.)65.7 ±\pm 1.8%33.6%CNN
L3Spatial (IoU)0.552 ±\pm 0.0280.467 ±\pm 0.006CNN
L4Reasoning (F1)N/A ‡0.576VLM
† Strict 3-class match; ∗ binary severity detection (see text).
‡ CNN architectures lack language generation capability.
TABLE I : Four-level diagnostic comparison of CNN (ResNet-18) and VLM (Qwen2-VL-7B) on spectrum heatmap understanding. CNN is trained with supervision; VLM is frozen with zero-shot or 3-shot prompting. Best per level in bold .
Fig. 1 : CNN vs. VLM performance across four granularity levels. CNN dominates supervised tasks (L1–L3) while VLM uniquely enables semantic reasoning (L4). This complementarity motivates the task-type routing strategy.
SettingKeyword F1ROUGE-L
Zero-shot0.1720.192
3-shot0.5760.576
Improvement+235%+200%
TABLE II : Few-shot scaling for VLM reasoning (L4). Three in-context examples produce a 235% improvement in keyword F1.
Fig. 2 : Few-shot scaling for VLM reasoning. Three in-context spectrum analysis examples unlock latent reasoning capability, producing a 235% F1 improvement.
Prompting StrategyL2 Acc.L4 F1
Zero-shot23.5%0.207
Zero-shot + CoT22.0%0.228 (+10.0%)
Few-shot (3 examples)21.0%0.209
Few-shot + CoT22.5%0.233 (+11.6%)
Random baseline25.0%—
TABLE III : Effect of chain-of-thought prompting on VLM performance. CoT selectively improves semantic reasoning (L4) but has no effect on spatial tasks (L2). Random chance for L2 is 25%.
ConfigurationCompositevs. CNN-only
CNN-only0.443—
VLM-only0.381−- 14.0%
Naive router (L3 →\to CNN, else →\to VLM)0.407−- 8.1%
Optimal router (L1–3 →\to CNN, L4 →\to VLM)0.616+39.1%
TABLE IV : Composite performance scores for different routing configurations. Weights: wL1=wL2=0.2w_{L1}=w_{L2}=0.2 , wL3=wL4=0.3w_{L3}=w_{L4}=0.3 .
DirectionVLMCNNVLM dropCNN drop
A →\to A0.6640.745——
A →\to B0.3330.301−- 0.331−- 0.444
A →\to C0.4000.513−- 0.264−- 0.232
B →\to B0.5090.548——
B →\to A0.5360.443−- 0.027++ 0.106
B →\to C0.0590.036−- 0.450−- 0.512
TABLE V : Cross-scenario spatial localization (IoU). VLM shows smaller performance degradation in 5/6 transfer directions.
Fig. 3 : Cross-scenario generalization. Each subplot shows in-distribution and cross-scenario IoU for a different training scenario. Annotations indicate which model degrades less during transfer. VLM shows smaller degradation in 5/6 directions.
Fig. 4 : Spatial probe IoU and Pearson correlation vs. VLM hidden layer index. Earlier layers preserve more spatial information, with Layer 0 achieving 22.5% higher IoU than Layer 28.
ArchitectureParametersIoU
Random baseline—0.200
SimpleConvNet317K0.476
U-Net Mini4.7M0.500
ResNet-18 (scratch)11.2M0.562
ResNet-18 (ImageNet)11.2M0.552
VLM probe (frozen)10M0.467
TABLE VI : CNN architecture comparison for spatial localization (L3). ResNet-18 with ImageNet pretraining achieves the best IoU.
Weight Schemew1:4w_{1:4}CNNVLMRouter
Equal0.25 each0.4740.3730.618
Spatial-heavy0.1, 0.1, 0.5, 0.30.4540.3670.449
Reasoning-heavy0.1, 0.1, 0.2, 0.60.3880.4560.534
Default0.2, 0.2, 0.3, 0.30.4430.3810.616
TABLE VII : Router composite score under different weight schemes. The optimal router consistently outperforms CNN-only regardless of weight choice.
QuestionVLM Response (3-shot)Ground Truth
Why is the Ka band congested?The Ka band shows high interference because multiple satellite beams and terrestrial stations share this frequency. The spatial concentration in the upper region suggests overlapping beam footprints from LEO satellites.Ka band: 3 satellite beams and 4 TN stations sharing the same frequency, creating co-channel interference with 7 total transmitters.
What reallocation would reduce interference?Consider migrating satellite traffic from the congested C band to L band, which has fewer active transmitters. This would reduce co-channel overlap in the high-interference northwest region.Migrating 2 satellite beams from C to L band would reduce C-band co-channel transmitters from 7 to 5.
TABLE VIII : Qualitative examples of VLM reasoning (L4) on SpectrumQA. The VLM produces spatially aware, actionable analysis that is impossible with CNN-only systems.
MetricCNNVLMRatio
Parameters11.2M8.3B (INT4)741 ×\times
GPU Memory0.5 GB8 GB16 ×\times
Inference Latency∼\sim 5 ms∼\sim 150 ms30 ×\times
Training Data5K labeled0 (frozen)—
L4 CapabilityNoYes—
TABLE IX : Computational cost comparison. VLM inference is approximately 30 ×\times slower but enables capabilities CNN cannot provide.
More in this topic

VLM Evaluation

Vision-language model evaluation measures multimodal systems’ perception, reasoning, grounding, robustness, and reliability across controlled and real-world tasks.

See all 989 papers

Explore similar work

CardsList
Jul 17, 2026

More with Less: a Large Scale Remote Sensing VLM with a Simple Recipe

Remote sensing vision-language models are increasingly expected to support open-ended reasoning over Earth Observation data and a variety of tasks. Most recent progress in this area has been driven by remote-sensing-specific architectural designs, often introducing new encoders, alignment modules, or task-specific fusion mechanisms. In this work, we challenge the necessity of such architectural specialization. We show that a generally capable vision-language model can achieve competitive or state-of-the-art performance at challenging remote sensing benchmarks, provided that it is trained at sufficient scale across diverse data and tasks. Our model uses a single language policy that can either answer directly in text or invoke a localization tool for segmentation and grounding. To train this heterogeneous behaviour, we employ a multi-task reinforcement learning framework with adaptive task rewards covering multiple-choice VQA, free-form VQA, captioning, detection, and segmentation across a large variety of input types. Our approach achieves competitive results across a broad set of benchmarks, including high-resolution, multi-temporal, multi-modal and multi-view tasks. Further, as training data scales, our experiments show consistent improvements across most tasks both in and out of distribution, which correlate with per-task data diversity. These findings suggest that, for remote sensing VLMs, data scale is sufficient even without architectural novelty.

Stefan Maria Ailuro, Mario Markov, Mohammad Mahdi +2

INSAIT, Sofia University "St. Kliment Ohridski"

Remote Sensing Image UnderstandingRemote Sensing VQARemote Sensing VLMsLarge Vision-Language Models
AskPDF
Feb 26, 2026

Spectrally Distilled Representations Aligned with Instruction-Augmented LLMs for Satellite Imagery

Vision-language foundation models (VLFMs) promise zero-shot and retrieval understanding for Earth observation. While operational satellite systems often lack full multi-spectral coverage, making RGB-only inference highly desirable for scalable deployment, the adoption of VLFMs for satellite imagery remains hindered by two factors: (1) multi-spectral inputs are informative but difficult to exploit consistently due to band redundancy and misalignment; and (2) CLIP-style text encoders limit semantic expressiveness and weaken fine-grained alignment. We present SATtxt, a spectrum-aware VLFM that operates with RGB inputs only at inference while retaining spectral cues learned during training. Our framework comprises two stages. First, Spectral Representation Distillation transfers spectral priors from a frozen multi-spectral teacher to an RGB student via a lightweight projector. Second, Spectrally Grounded Alignment with Instruction-Augmented LLMs bridges the distilled visual space and an expressive LLM embedding space. Across EuroSAT, BigEarthNet, and ForestNet, SATtxt improves zero-shot classification on average by 4.2%, retrieval by 5.9%, and linear probing by 2.7% over baselines, showing an efficient path toward spectrum-aware vision-language learning for Earth observation. Project page: https://ikhado.github.io/sattxt/

Minh Kha Do, Wei Xiang, Kang Han +5

La Trobe University, Melbourne, VIC 3086, Australia · Cisco Research, San Jose, CA, USA

Remote Sensing VLMsRemote Sensing Image UnderstandingVisual Representation LearningLarge Vision-Language ModelsVision-Language AlignmentKnowledge Distillation
AskPDF
Apr 2, 2026

Are VLMs Lost Between Sky and Space? LinkS2^2Bench for UAV-Satellite Dynamic Cross-View Spatial Intelligence

Synergistic spatial intelligence between UAVs and satellites is indispensable for emergency response and security operations, as it uniquely integrates macro-scale global coverage with dynamic, real-time local perception. However, the capacity of Vision-Language Models (VLMs) to master this complex interplay remains largely unexplored. This gap persists primarily because existing benchmarks are confined to isolated Unmanned Aerial Vehicle (UAV) videos or static satellite imagery, failing to evaluate the dynamic local-to-global spatial mapping essential for comprehensive cross-view reasoning. To bridge this gap, we introduce LinkS2^2Bench, the first comprehensive benchmark designed to evaluate VLMs' wide-area, dynamic cross-view spatial intelligence. LinkS2^2Bench links 1,022 minutes of dynamic UAV footage with high-resolution satellite imagery covering over 200 km2^2. Through an LMM-assisted pipeline and rigorous human annotation, we constructed 17.9k high-quality question-answer pairs comprising 12 fine-grained tasks across four dimensions: perception, localization, relation, and reasoning. Evaluations of 18 representative VLMs reveal a substantial gap compared to human baselines, identifying accurate cross-view dynamic alignment as the critical bottleneck. To alleviate this, we design a Cross-View Alignment Adapter, demonstrating that explicit alignment significantly improves model performance. Furthermore, fine-tuning experiments underscore the potential of LinkS2^2Bench in advancing VLM adaptation for complex spatial reasoning.

Dian Liu, Jie Feng, Di Li +4

Xidian University, China · Qinghai Normal University, China · Sun Yat-sen University, China

Visual Spatial ReasoningVLM EvaluationRemote Sensing VQACross-View Geo-Localization
AskPDF