RSJEV: Discriminative Remote Sensing Scene Classification with Multimodal Large Language Models

Dongchen Si; Di Wang; Mingzhen Xu; Jing Zhang; Bo Du; Liangpei Zhang

cs.CVOct 6, 2026

RSJEV: Discriminative Remote Sensing Scene Classification with Multimodal Large Language Models

Authors: Dongchen Si, Di Wang, Mingzhen Xu, Jing Zhang, Bo Du, Liangpei Zhang

Organizations: School of Computer Science, Wuhan University, Wuhan 430072, China · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China

Abstract

Remote sensing scene classification is a fundamental task in Earth observation and geospatial analysis. Existing approaches mainly follow three paradigms: task-specific visual classification, vision-language similarity matching, and autoregressive multimodal generation. However, visual classifiers rely on predefined label spaces, CLIP-based methods perform recognition through static image-text alignment, and multimodal large language models (MLLMs) introduce unnecessary token-level generation for classification tasks with explicit candidate categories. To address these limitations, we propose RSJEV, a one-pass multimodal decision framework for remote sensing scene classification. Unlike conventional MLLMs that formulate classification as autoregressive text generation, RSJEV reformulates scene classification as a candidate-conditioned multimodal discriminative decision process, where visual representations, task instructions, and candidate category semantics are jointly modeled. Specifically, we introduce a OnePass Decider that extracts multimodal decision states and directly estimates category probabilities within the candidate category space, eliminating autoregressive decoding while preserving vision-language interactions. Extensive experiments on three widely used remote sensing scene classification benchmarks, including UC Merced, AID, and NWPU-RESISC45, demonstrate that RSJEV achieves superior classification performance compared with representative CNN-, Transformer-, Mamba-, CLIP-, and MLLM-based methods. Moreover, RSJEV significantly reduces inference costs and achieves a better accuracy-efficiency trade-off with only a compact 0.8B-parameter model. These results demonstrate the effectiveness of state-conditioned multimodal decision making for efficient remote sensing image understanding. The code will be available at https://github.com/Dongtcs/RSJEV.

Figures & tables

Fig. 1: Conceptual comparison of different classification paradigms: (left) conventional image-only visual classifiers, (middle) similarity-based or generative MLLMs, and (right) the proposed RSJEV. By eliminating iterative autoregressive decoding, RSJEV enables instruction-guided one-pass discriminative decision-making, achieving an effective balance between classification performance and inference efficiency.

Fig. 2: Overview of the proposed RSJEV framework for one-pass remote sensing scene classification.

CNN-based Models
Method	Param	UCM				AID				NWPU
Method	Param	P	R	F1	OA	P	R	F1	OA	P	R	F1	OA
ResNet-101 [ 7 ]	42.6 M	95.91	95.71	95.70	95.71	92.33	91.79	91.89	92.23	90.34	90.34	90.25	90.34
DenseNet-161 [ 8 ]	28.7 M	97.18	97.05	97.02	97.05	93.90	93.48	93.57	93.90	92.77	92.73	92.69	92.73
EfficientNet [ 18 ]	87.4 M	97.12	96.76	96.80	96.76	91.48	91.39	91.33	91.76	92.49	92.45	92.43	92.45
Transformer-based Models

TABLE I: Comparison with state-of-the-art methods on three remote sensing scene classification benchmarks. The bold and underlined values denote the best and second-best results, respectively.

Backbone	Method	UCM	AID	NWPU
Qwen3.5-0.8B	Baseline	87.96	91.29	88.24
Qwen3.5-0.8B	Ours	97.80	96.44	94.61
InternVL3.5-1B	Baseline	93.21	92.26	93.54
InternVL3.5-1B	Ours	96.68	96.27	94.93

TABLE II: Generalizability across MLLM Backbones (P, %).

Explore similar work

Jul 22, 2026cs.CV

Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?

The rapid development of multimodal large language models (MLLMs) has introduced a flexible paradigm for remote sensing image scene understanding (RSISU), enabling natural-language interaction with remote sensing imagery. However, a systematic understanding of the capability boundaries, cross-task generalization, and task-specific limitations of existing remote sensing MLLMs (RS-MLLMs) is still lacking. This paper presents a systematic survey and diagnostic evaluation of MLLMs for RSISU. We review the technical evolution of RS-MLLMs, focusing on model design, multimodal learning, training data, and downstream capabilities. We further compare RS-MLLMs with general-purpose computer vision MLLMs (CV-MLLMs) across diverse RSISU tasks and benchmarks. RS-MLLMs remain competitive in domain-specific settings, particularly remote sensing visual grounding and high-resolution visual question answering. More notably, general-purpose CV-MLLMs can match or even outperform these specialized models on several RSISU tasks without remote sensing-specific fine-tuning. These findings demonstrate the strong transferability of general-purpose CV-MLLMs and show that current RS-MLLMs do not consistently outperform them across diverse RSISU tasks. Current MLLMs also face limitations in spatial and relational reasoning, fine-grained visual understanding, instruction diversity, and generalization across heterogeneous task formats. Based on these findings, we outline future directions toward reliable evaluation, multimodal and high-resolution reasoning, efficient deployment, and tool-augmented remote sensing agents. This survey provides a systematic reference for developing robust, generalizable, and practical MLLMs for RSISU.

Qiwei Ma, Chunping Qiu, Xinjun Cheng +5

School of Artificial Intelligence and Robotics, Hunan University, Changsha, 410082, China · Intelligent Game and Decision Lab (IGDL), Beijing, 100091, China · cYuelushan Center for Industrial Innovation, Changsha, 410082, China

Remote Sensing Multimodal Large Language Models Fine-Grained Perception Domain-Specific Language

Ask PDF

Jun 9, 2026cs.CV

Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks

RS-MLLMs enable natural-language understanding and spatial reasoning over earth observation imagery. However, existing models support only a narrow range of sensor types and tasks, yielding a fragmented view of the earth and leaving cross-modal geoscientific knowledge largely unexploited. This work presents Earth-OneVision, a 2B RS-MLLM that unifies six sensor modalities (i.e., optical, SAR, infrared, multispectral, temporal, and video) and cross-sensor fusion across 9 task categories within a single autoregressive framework. Three dedicated mechanisms address three bottlenecks. Full-Granularity Vision-Language Alignment (FGVLA) aligns multi-level visual features with the multi-dimensional language space. Spatial-Linguistic Isomorphic Serialization (SLIS) unifies heterogeneous spatial outputs as autoregressive tokens. Progressive Cross-Modality Adaptation (PCMA) decomposes the compound domain gap into sequential stages, tackling the viewpoint and imaging physics gaps in turn. To support joint training, MMRS-OneVision is constructed with ~34M QA pairs spanning all six sensor modalities and cross-sensor fusion across 9 task categories, substantially exceeding existing RS multimodal instruction datasets. With only 2B parameters, Earth-OneVision achieves competitive or state-of-the-art results across extensive benchmarks, consistently matching or outperforming 4B-72B RS-MLLMs. It achieves 87.52% P@0.5 on the OPT-RSVG testset for optical visual grounding and 80.68% on the SAR VQA benchmark SARLANG-Bench, exceeding 7B models by over 7%. It further achieves 75.74% recall on the BigEarthNet-MS testset for multispectral classification, and 81.94% MCQ accuracy on EarthMind-Bench for cross-modality reasoning.

Miaoxin Cai, Guanqun Wang, Wei Zhang +6

National Key Laboratory of Science and Technology on Space-Born Intelligent Information Processing (SBIIP), Beijing Institute of Technology, Beijing 100081, China · Advanced Research Institute of Multidisciplinary Sciences, Beijing Institute of Technology, Beijing 100081, China · School of Mechatronical Engineering, Beijing Institute of Technology, Beijing 100081, China +5

Multimodal Large Language Models Recent Vision-Language Models Multi-Sensor Fusion Synthetic Aperture Radar Mllm-Based

Ask PDF

May 11, 2026cs.CV

MPerS: Dynamic MLLM MixExperts Perception-Guided Remote Sensing Scene Segmentation

The multimodal fusion of images and scene captions has been extensively explored and applied in various fields. However, when dealing with complex remote sensing (RS) scenes, existing studies have predominantly concentrated on architectural optimizations for integrating textual semantic information with visual features, while largely neglecting the generation of high-quality RS captions and the investigation of their effectiveness in multimodal semantic fusion.In this context, we propose the Dynamic MLLM Mixture-of-Experts Perception-Guided Remote Sensing Scene Segmentation, referred to as MPerS.We design multiple prompts for MLLMs to generate high-quality RS captions, enabling MLLMs to perceive RS scenes from diverse expert perspectives. DINOv3 is employed to extract dense visual representations of land-covers.We design a Dynamic MixExperts module that adaptively integrates the most effective textual semantics. Linguistic Query Guided Attention is constructed to utilize textual semantic information to guide visual features for precise segmentation. The MLLMs include LLaVA, ChatGPT, and Qwen. Our method achieves superior performance on three public semantic segmentation RS datasets.

Ziyi Wang, Xianping Ma, Ziyao Wang +2

The Chinese University of Hong Kong (Shenzhen) · Southwest Jiaotong University

Remote Sensing Image Segmentation Scene Reasoning Multimodal Fusion Multimodal Large Language Models Scene Understanding Mixture-Of-Experts Large Language Models

Ask PDF