GeBDA: Building Damage Assessment as Text-Based Sequence Prediction

Olivier Dietrich; Krishna Sapkota; Konrad Schindler; Genady Beryozkin

cs.CVAug 28, 2026

GeBDA: Building Damage Assessment as Text-Based Sequence Prediction

Authors: Olivier Dietrich, Krishna Sapkota, Konrad Schindler, Genady Beryozkin

Organizations: ETH Zurich · Google

Abstract

Conventionally, Building Damage Assessment (BDA) is tackled either with dedicated network architectures or by fine-tuning geospatial image foundation models. In this work, we ask whether a general-purpose Vision-Language Model (VLM) can localize buildings and grade their damage through autoregressive sequence generation alone. We cast BDA as predicting a variable-length set of bounding boxes, each specified by its coordinates and a damage label. Our preliminary implementation, based on the open Gemma model, achieves promising damage mapping results from only bi-temporal satellite images and a suitable text prompt.

Figures & tables

Figure 1: Autoregressive sequence prediction formulation for BDA. Building polygons are transformed into bounding boxes, their coordinates are quantized and serialized into strings alongside the damage label, and finally tokenized to form a single target sequence $S$ . The model is fine-tuned end-to-end to predict that sequence from a task-specific prompt $P$ and the two images $I_{pre}$ and $I_{post}$ .

Dataset	Modality (Pre/Post)	Pairs	Polygons	Classes	Splits (Train/Eval)
xBD [ 14 ]	Optical/Optical	11,034	$>$ 425k	4	Train+Tier3/Test+Holdout
Bright [ 5 ]	Optical/SAR	3,029	$\sim$ 245k	3	Subset of official Train/Test

Table 1: Overview of the two datasets used.

		xBD						Bright
Metrics	Model	$\mathrm{F}_{1}^{\mathrm{loc}}$	$\mathrm{F}_{1}^{\mathrm{cls}}$	$\mathrm{F}_{1}^{\mathrm{int.}}$	$\mathrm{F}_{1}^{\mathrm{min.}}$	$\mathrm{F}_{1}^{\mathrm{maj.}}$	$\mathrm{F}_{1}^{\mathrm{des.}}$	$\mathrm{F}_{1}^{\mathrm{loc}}$	$\mathrm{F}_{1}^{\mathrm{cls}}$	$\mathrm{F}_{1}^{\mathrm{int.}}$	$\mathrm{F}_{1}^{\mathrm{dam.}}$	$\mathrm{F}_{1}^{\mathrm{des.}}$
Instance	GeBDA	72.56	76.62	95.66	61.27	72.78	85.55	74.72	48.99	95.08	35.66	44.11
Instance	GeBDA*	70.95	75.93	95.06	59.61	74.39	83.75	73.74	60.05	97.08	39.12	70.92
Pixel	UNet [ 23 ]	83.46	66.50	–	–	–	–	88.68	70.85	–	–	–
	ChangeOS [ 37 ]	86.97	71.34	–	–	–	–	90.14	70.19	–	–	–
	ChangeMamba [ 6 ]	87.94	73.58	–	–	–	–	91.09	73.38	–	–	–

Table 2: Quantitative results on the xBD and Bright test sets. Instance-based: GeBDA is trained independently per dataset; GeBDA* jointly on both. Pixel-based: Rasterized predictions are compared against standard to state-of-the-art baselines (all numbers taken from [ 7 ] ,

\mathrm{F}_{1}^{\mathrm{cls,c}}

not available). Oracle represents the upper-bound performance, obtained by rasterizing ground-truth bounding boxes.

Figure 2: Qualitative results on the test sets, stitched back to $1024\times 1024$ . The top two rows show examples from xBD and the bottom row from Bright . For consistency, the Bright damaged class is mapped to major-damage . GeBDA demonstrates robust building detection across both datasets, but damage classification remains more challenging, particularly on the SAR modality. Left: Pre-event images with ground-truth bounding boxes. Right: Post-event images overlaid with our model’s predictions.

Panel of Figure 3 — Figure 3: Model performance analysis on the xBD test set. Left: Predicted vs. ground-truth building counts. Right: Analysis of prediction entropy in heterogeneous tiles. Grey bars indicate the total number of tiles per ground-truth damage ratio bin. Red bars highlight the subset of those tiles where the model’s prediction collapsed ( $H\approx 0$ ).

Figure 4: Left: Zero-shot output of our model. Right: Official damage assessment from the Federal Emergency Management Agency (FEMA) [ 10 ] . Damage labels have been binarized (intact vs. damaged) to be directly comparable.

Building Damage Assessment

Building damage assessment uses aerial and satellite imagery to locate buildings and estimate disaster-related structural damage for response and recovery.

See all 25 papers

DisasterInsight: A Building-Centric Benchmark for Evaluating Vision--Language Models in Disaster Response
Jan 26, 2026Sara Tehrani, Yonghao Xu, Leif Haglund +3Building Damage Assessment Remote Sensing Image Understanding
Computer Vision and Learning System, Linköping University, Sweden · Vantor, Linköping, Sweden
Assessing the Benefits of Combining Advanced Deep Learning Techniques for Post-Disaster Building Damage Assessment from UAV Imagery
Aug 3, 2026Huy Quang Ung, Guillaume Habault, Roberto Legaspi +3Disaster Damage Assessment Building Damage Assessment
KDDI Research, Inc., Fujimino, Japan
Revisiting the Task of Building Damage Assessment by Satellite Imaging
Oct 9, 2026Anthony Cioppa, Virgile Blua, Sébastien Piérard +2Building Damage Assessment Disaster Damage Assessment

Jan 26, 2026J/K move · Enter open · S save

DisasterInsight: A Building-Centric Benchmark for Evaluating Vision--Language Models in Disaster Response

Sara Tehrani, Yonghao Xu, Leif Haglund, Amanda Berg+2

Computer Vision and Learning System, Linköping University, Sweden · Vantor, Linköping, Sweden

Building Damage Assessment Remote Sensing Image Understanding Disaster Damage Assessment VLM Evaluation Remote Sensing Benchmarks Disaster Response+1

Open paper AskPDF arXiv

Vision--language models (VLMs) show promise for disaster-response remote sensing, but existing benchmarks mainly emphasize scene-level or damage-centric assessment. To study this building-centric gap, we introduce \method{}, a diagnostic benchmark built on xBD, a pre/post-disaster satellite dataset with building-level damage labels. \method{} enriches building instances with OpenStreetMap-derived functional labels and contains 134{,}108 task-specific instruction records across 15 task types, spanning instance-level assessment, scene-level counting, multi-instance reasoning, and structured report generation. The benchmark supports RGB pre/post-disaster imagery, single- and multi-view instance formulations, and scene-level RGB/SAR diagnostic inputs. Experiments with general-domain and remote-sensing VLMs show that models perform better on visible damage cues than on building-function understanding, multi-instance reasoning, counting, and grounded reporting. Instruction tuning improves performance on several tasks but does not close this building-centric gap.

Abstract

Building Damage Assessment

Explore similar work

DisasterInsight: A Building-Centric Benchmark for Evaluating Vision--Language Models in Disaster Response

Assessing the Benefits of Combining Advanced Deep Learning Techniques for Post-Disaster Building Damage Assessment from UAV Imagery

Revisiting the Task of Building Damage Assessment by Satellite Imaging