TerraScope: Pixel-Grounded Visual Reasoning for Earth Observation

Yan Shu; Bin Ren; Zhitong Xiong; Xiao Xiang Zhu; Begüm Demir; Nicu Sebe; Paolo Rota

cs.CVMar 19, 2026

TerraScope: Pixel-Grounded Visual Reasoning for Earth Observation

Authors: Yan Shu, Bin Ren, Zhitong Xiong, Xiao Xiang Zhu, Begüm Demir, Nicu Sebe, Paolo Rota

Organizations: University of Trento · MBZUAI · Technical University of Munich · BIFOLD and TU Berlin

Abstract

Vision-language models (VLMs) have shown promise in earth observation (EO), yet they struggle with tasks that require grounding complex spatial reasoning in precise pixel-level visual representations. To address this problem, we introduce TerraScope, a unified VLM that delivers pixel-grounded geospatial reasoning with two key capabilities: (1) modality-flexible reasoning: it handles single-modality inputs (optical or SAR) and adaptively fuses different modalities into the reasoning process when both are available; (2) multi-temporal reasoning: it integrates temporal sequences for change analysis across multiple time points. In addition, we curate Terra-CoT, a large-scale dataset containing 1 million samples with pixel-level masks embedded in reasoning chains across multiple sources. We also propose TerraScope-Bench, the first benchmark for pixel-grounded geospatial reasoning with six sub-tasks that evaluates both answer accuracy and mask quality to ensure authentic pixel-grounded reasoning. Experiments show that TerraScope significantly outperforms existing VLMs on pixel-grounded geospatial reasoning while providing interpretable visual evidence.

Figures & tables

Figure 2: Overview of TerraScope. TerraScope generates textual reasoning tokens and segmentation masks in an interleaved manner, where masked visual features are injected at each reasoning step to ensure faithful pixel-grounded reasoning. TerraScope supports multi-modal and multi-temporal reasoning across EO data.

Figure 3: Terra-CoT curation pipeline. First, we generate Cap-CoT using ground truth masks and class labels to train an initial annotation model. Second, we use the trained model to annotate unlabeled data with pixel-accurate masks and captions. Third, based on the synthetic annotations, we apply hierarchical data synthesis to generate diverse reasoning questions with chain-of-thought traces at two levels: (L1) basic spatial grounding and (L2) complex multi-step reasoning including spatial and semantic tasks.

Model	Size	TerraScope-Bench							Landsat30AU				DisasterM3
		CA	AQ	CR	BRD	DM	BCE	Avg.	APR	NUM	SRI	Avg.	BDC	DRE	Avg.
General VLMs
GPT-4o † [ 34 ]	-	27.6	25.4	54.3	75.3	22.5	27.1	38.7	-	-	-	-	24.2	21.4	22.8
LLaVA-OV [ 18 ]	7B	28.0	21.2	56.6	75.9	19.4	23.7	37.5	39.4	46.6	85.1	57.0	26.4	24.2	25.3
Qwen2.5-VL [ 1 ]	7B	25.3	33.5	55.7	67.7	23.3	25.7	38.5	29.8	53.1	92.8	58.6	34.2	29.3	31.8
InternVL3 [ 59 ]	8B	22.3	26.3	57.2	67.0	18.6	24.3	36.0	31.4	42.4	90.6	54.8	30.3	24.1	27.2

Table 1: Quantitative performance of Terrascope on TerraScope-Bench (optical only), Landsat30AU and DisasterM3. “Avg” is the average performance of multiple-choice tasks. Bold means the best performance. † denotes proprietary models. ‡ denotes reasoning models. Fine-tuned VLMs are fine-tuned on our proposed Terra-CoT dataset.

Figure 5: Grounding IoU performance of different models.

Model	TerraBen.	Landsat.	Disaster.
Original	33.8	45.7	23.6
Textual CoT w/o Seg.	58.7	56.5	32.9
Textual CoT with Seg.	60.6	58.9	35.8
Random-Mask CoT	43.2	53.8	32.6
Box CoT	62.8	70.5	43.9
TerraScope	68.9	73.9	46.5

Table 2: Ablation study on the effect of different CoT strategies for pixel-grounded visual reasoning. “Original” denotes the base TerraScope model after pretraining, upon which we fine-tune with different CoT variations via SFT.

Figure 6: IoU distribution for correct vs. incorrect predictions.

Model	CA	AQ	CR	BRD	DM
No Fusion	73.2	70.2	71.8	80.0	65.9
Concat.	74.5	71.6	73.0	81.2	67.4
Text-guided (test only.)	72.3	69.0	66.7	78.8	63.6
Text-guided (train + test)	74.3	70.9	72.7	80.7	68.2

Table 3: Ablation study of multi-modal reasoning.

Figure 7: Visualizations of multi-modal reasoning.

Methods	TerraBench.	Landsat.
Interleaved Reasoning Models
Mint-CoT (with SFT)	54.6	62.8
Mint-CoT (with SFT + RL)	55.7	63.2
LLM-Agent-Based Methods
ThinkGeo	28.5	–
EarthAgent	37.6	–

Table A : Comparison of TerraScope with interleaved reasoning models and LLM-agent-based methods on TerraScope-Bench.

Figure A : Data distribution of TerraScope-Bench.

Figure B : Examples of human expert verification of mask accuracy. The first row shows original images, the second row shows original masks, and the third row shows masks modified by human annotators.

Figure C : More examples of TerraScope-Bench, including the questions, answers and the masks involved in the CoT.

Figure D : Data distributions of Terra-CoT.

Hyperparameter	Value
Overall batch size	32
Learning rate	4e-5
LR Scheduler	Cosine decay
DeepSpeed ZeRO Stage	ZeRO-2
Optimizer	Adam
Warmup ratio	0.3

Table B : Hyperparameters of TerraScope.

Model	Total Params	Additional Modules
GPT-4o	-	-
Qwen2.5-VL-7B	7.6B	-
InternVL-3-8B	8.1B	-
GLM-4.1V-9B	9.4B	-
LLaVA-OV-7B	7.2B	-
TerraScope-8B	8.3B	SAM-2 (0.228B)

Table C : Model complexity comparison. TerraScope adds a lightweight pixel-level grounding module on top of InternVL-3.

Model	Avg. Time (s)
InternVL-3-8B	0.85
Qwen2.5-VL-7B	0.92
TerraScope-8B	2.48
GLM-4.1V-9B	2.60

Table D : Average inference time per sample (seconds).

Model	1 Mask	2 Masks	3+ Masks
InternVL-3-8B	18.2	18.3	18.2
Qwen2.5-VL-7B	16.8	17.0	17.0
TerraScope-8B	22.4	23.1	24.2

Table E : GPU memory consumption (GB) on NVIDIA A100.

Model	Size	APR	COA	DLC	FOD	MOP	NUM	SRI	USR	Overall
EarthDial	4B	23.49	10.34	75.27	99.00	61.16	43.62	51.24	15.52	48.29
RS-LLaVA	7B	68.57	80.88	71.24	87.00	63.09	49.85	26.17	10.34	57.24
MiMo	7B	40.00	45.77	92.47	93.33	84.30	61.42	94.21	88.97	75.55
GLM-4.1V	9B	45.71	36.36	72.85	62.67	67.49	58.63	69.97	88.28	62.87
Qwen2.5-V	7B	29.84	89.66	94.09	71.67	76.03	53.12	92.84	82.07	74.28
LLaVA-OV	8B	39.37	79.00	83.06	59.00	72.45	46.59	85.12	10.34	60.96

Table F: Performance on the VQA task on Landsat30-AU. Bold indicates the best score.

Model	RSVQA-LR	BigEarthNet
GeoChat	90.7	20.4
LHRS-Bot	89.2	-
EarthDial	92.7	68.8
TerraScope	91.4	69.2

Table G : Results on RSVQA and scene classification.

Model	Optical-SAR
Model	BDC	DRE	Avg
LLaVA-OV	22.2	19.4	20.8
TEOChat	18.4	9.4	13.9
InternVL3-8B	20.7	18.4	19.6
EarthDial	19.5	10.2	14.9
TerraScope	50.4	32.6	41.5

Table H : Optical-SAR results on DisasterM3 benchmark.

Data	TerraBench.	Landsat.	Disaster.
Cap-CoT	42.8	50.1	26.9
Cap-CoT + L1-VQA	66.7	61.0	46.2
Cap-CoT + L1-VQA + L2-VQA	68.9	73.9	46.5

Table I : Ablations of Terra-CoT.

Training Strategy	TerraScope-Bench	Landsat30-AU	DisasterM3
w/o Grounded Pretrain	65.4	71.8	43.0
w/ Grounded Pretrain	68.9	73.9	46.5

Table J : Ablation study on grounded pretraining.

Figure E : More qualitative results of TerraScope.

Multi-modal	Masked Feature	TerraScope-Bench		Efficiency
Encoding	Interleaving	Mean IoU (%)	Accuracy (%)
Optical only	Optical only	53.4	65.0	High
Optical only	Concat Opt+SAR	53.5	67.6	Low
Optical only	Adaptive selection	53.1	67.4	High
Concat Opt+SAR	Optical only	56.8	69.2	High
Concat Opt+SAR	Concat Opt+SAR	57.2	73.0	Low

Table K : Ablation study on multi-modal reasoning. ”Efficiency” indicates inference efficiency: ”High” for methods with shorter context length (single modality or adaptive selection), ”Low” for concatenation methods that double the visual token count.

Remote Sensing VLMs

See all 41 papers

OmniEarth: A Benchmark for Evaluating Vision-Language Models in Geospatial Tasks
Mar 10, 2026Ronghao Fu, Haoran Liu, Weijie Zhang +4VLM Evaluation Remote Sensing Benchmarks
Jilin University, Changchun, China · Chang Guang Satellite Technology, Changchun, China
LongEarth-R1: Benchmarking and Aligning Vision-Language Models for Long-Horizon Earth Observation Reasoning
Aug 13, 2026Yupan Ding, Jing Xiao, Zhenyuan Zhang +4Remote Sensing Remote Sensing VQA
School of Artificial Intelligence, Wuhan University, Wuhan, China · School of Computer Science, Wuhan University, Wuhan, China · Xi’an University of Electronic Science and Technology, Xi’an, China +1
EarthSpatialBench: Benchmarking Spatial Reasoning Capabilities of Multimodal LLMs on Earth Imagery
Feb 17, 2026Zelin Xu, Yupu Zhang, Saugat Adhikari +6Spatial Reasoning Benchmarks Geospatial Reasoning Benchmarks
Department of Computer & Information Science & Engineering, University of Florida, Gainesville, FL, USA · Department of Computer Science, Indiana University Bloomington, Bloomington, IN, USA.

Mar 10, 2026J/K move · Enter open · S save

OmniEarth: A Benchmark for Evaluating Vision-Language Models in Geospatial Tasks

Ronghao Fu, Haoran Liu, Weijie Zhang, Zhiwen Lin+3

Jilin University, Changchun, China · Chang Guang Satellite Technology, Changchun, China

VLM Evaluation Remote Sensing Benchmarks Remote Sensing VLMs Remote Sensing VQA Remote Sensing Visual Grounding

Open paper AskPDF arXiv

Vision-Language Models (VLMs) have demonstrated effective perception and reasoning capabilities on general-domain tasks, leading to growing interest in their application to Earth observation. However, a systematic benchmark for comprehensively evaluating remote sensing vision-language models (RSVLMs) remains lacking. To address this gap, we introduce OmniEarth, a benchmark for evaluating RSVLMs under realistic Earth observation scenarios. OmniEarth organizes tasks along three capability dimensions: perception, reasoning, and robustness. It defines 28 fine-grained tasks covering multi-source sensing data and diverse geospatial contexts. The benchmark supports two task formulations: multiple-choice VQA and open-ended VQA. The latter includes pure text outputs for captioning tasks, bounding box outputs for visual grounding tasks, and mask outputs for segmentation tasks. To reduce linguistic bias and examine whether model predictions rely on visual evidence, OmniEarth adopts a blind test protocol and a quintuple semantic consistency requirement. OmniEarth includes 9,275 carefully quality-controlled images, including proprietary satellite imagery from Jilin-1 (JL-1), along with 44,210 manually verified instructions. We conduct a systematic evaluation of contrastive learning-based models, general closed-source and open-source VLMs, as well as RSVLMs. Results show that existing VLMs still struggle with geospatially complex tasks, revealing clear gaps that need to be addressed for remote sensing applications. OmniEarth is publicly available at https://huggingface.co/datasets/sjeeudd/OmniEarth.

Abstract

Remote Sensing VLMs

Explore similar work

OmniEarth: A Benchmark for Evaluating Vision-Language Models in Geospatial Tasks

LongEarth-R1: Benchmarking and Aligning Vision-Language Models for Long-Horizon Earth Observation Reasoning

EarthSpatialBench: Benchmarking Spatial Reasoning Capabilities of Multimodal LLMs on Earth Imagery