LoDEOT: Low-Dimensional and Efficient Offset Tokens for Building Footprint Extraction from Off-Nadir Imagery

Kai Li; Zigan Zhou; Zhenyang Li; Hui Shan; Zhe Chen; Yupeng Deng; Zhihao Xi; Yu Meng

cs.CVOct 5, 2026

LoDEOT: Low-Dimensional and Efficient Offset Tokens for Building Footprint Extraction from Off-Nadir Imagery

Authors: Kai Li, Zigan Zhou, Zhenyang Li, Hui Shan, Zhe Chen, Yupeng Deng, Zhihao Xi, Yu Meng, +2 more

Organizations: City University of Hong Kong · University of the Chinese Academy of Sciences · University of Hong Kong · Zhejiang University · University of Southampton

Abstract

Instance-level roof-to-footprint offset (RFO) prediction is central to extracting building footprints from off-nadir imagery. Query-based pipelines commonly use high-dimensional instance tokens to predict signed two-dimensional RFOs. We investigate whether RFO prediction can instead use a compact offset token. Under local pinhole projection and vertical-extrusion assumptions, the idealized RFO map admits a five-parameter sufficient descriptor comprising intrinsic shape, composite amplitude, and relative geometry. This factorization provides a structural prior for a five-dimensional offset token, whose channels learn task-relevant latent representations through end-to-end training. Based on this design, we propose LoDEOT, which retains high-dimensional instance tokens for detection and segmentation but maps instance-token, concentration-gated roof, and box-mask evidence to a five-dimensional offset token followed by an independent two-dimensional readout. Known denoising-query target indices further align each supervised decoder-layer estimate with the same clean instance RFO, organizing successive predictions as target-aligned recovery under perturbed query conditions. Experiments on five real-world building datasets demonstrate the effectiveness of LoDEOT for building footprint extraction. Experiments on real-world building datasets demonstrate that a five-dimensional offset token can support accurate RFO prediction. On BONAI, LoDEOT achieves the best roof-detection bAP and bAP50 and leads all five offset-corrected footprint metrics among the evaluated end-to-end methods, with FAP50 of 54.58 and mEPE of 5.23 pixels. Its FAP50 exceeds those of the evaluated end-to-end baselines by 7.56-16.85 percentage points.

Figures & tables

Fig. 1: RFO prediction architecture after incorporating LoDEOT. Motivated by the projective geometry of 3D imaging, we redesign the offset-token representation. After the backbone encodes the visual features, each original high-dimensional instance token is augmented with a 5-D LoDEOT that carries the task-specific RFO representation, while retaining its roles in classification, roof-mask, and roof-box prediction. The augmented instance tokens are progressively updated through $L$ Transformer decoder layers; the final head predicts roof masks and RFOs to recover footprints. During training, each layer’s RFO prediction is supervised by the same clean instance target.

Externally Guided Methods
Method	Roof Detection and Segmentation				Offset-Corrected Footprint
Method	bAP $\uparrow$	bAP50 $\uparrow$	sAP $\uparrow$	sAP50 $\uparrow$	FAP50 $\uparrow$	F1 $\uparrow$	B-IoU $\uparrow$	mEPE $\downarrow$	PCP@5 $\uparrow$
OBM	–	–	12.59	36.71	22.21	41.72	9.25	5.72	53.68
PolyFootNet	–	–	8.91	33.18	18.76	39.75	7.32	6.54	44.71
End-to-End Methods
LOFT	10.50	42.90	21.00	59.50	37.73	49.64	8.78	5.65	57.74

TABLE I: Performance comparison of externally guided and end-to-end methods on the BONAI test set. The best results among the end-to-end methods are highlighted. All metrics are percentages except mEPE, which is measured in pixels.

Fig. 2: Qualitative comparison of OBM, PolyFootNet, LOFT, Cas.LOFT, MLS-BRN, and LoDEOT (ours) on three representative BONAI scenes. Each row shows the same scene across methods, and each column corresponds to one method. Yellow denotes true-positive matches (IoU $\geq 0.5$ ), cyan unmatched predictions (false positives), and red unmatched ground-truth instances (false negatives). OBM and PolyFootNet use LoDEOT-predicted masks as guidance.

Level	BONAI		IRSA		WHU		UAV		Huizhou		Average
Level	sAP50	FAP50	sAP50	FAP50	sAP50	FAP50	sAP50	FAP50	sAP50	FAP50	sAP50	FAP50
L1	54.94	54.58	9.70	8.77	49.50	48.75	31.64	28.49	35.72	31.17	36.30	34.35
L2	54.68	51.58	51.67	48.76	26.65	26.60	14.03	12.98	8.82	5.56	31.17	29.09
L3	54.21	51.76	49.42	46.30	96.08	96.08	24.25	22.65	31.22	25.01	51.04	48.36
L4	55.22	52.29	49.27	46.34	95.55	95.54	44.99	41.75	39.13	32.28	56.83	53.64

TABLE II: Evaluation of LoDEOT with

d=5

under cumulative multi-source training. The average is the unweighted mean over the five test domains.

$d$	bAP $\uparrow$	bAP50 $\uparrow$	sAP $\uparrow$	sAP50 $\uparrow$	FAP50 $\uparrow$	F1 $\uparrow$
2	$29.66\pm 1.20$	$49.10\pm 1.56$	$28.66\pm 1.20$	$48.96\pm 1.67$	$45.67\pm 1.60$	$51.60\pm 1.20$
3	$29.63\pm 0.82$	$48.96\pm 1.42$	$28.73\pm 0.83$	$49.01\pm 1.48$	$45.76\pm 1.28$	$51.62\pm 1.34$
4	$29.04\pm 1.25$	$48.03\pm 2.04$	$28.16\pm 1.19$	$48.03\pm 2.06$	$45.07\pm 1.94$	$50.97\pm 1.55$
5	$29.42\pm 1.50$	$48.54\pm 2.35$	$28.41\pm 1.47$	$48.38\pm 2.46$	$45.20\pm 2.52$	$50.93\pm 2.16$
6	$30.08\pm 0.97$	$49.68\pm 0.74$	$29.10\pm 0.76$	$49.62\pm 0.72$	$46.32\pm 0.98$	$51.97\pm 0.69$
7	$\mathbf{30.61}\pm 0.51$	$\mathbf{50.31}\pm 0.79$	$\mathbf{29.62}\pm 0.48$	$\mathbf{50.35}\pm 0.89$	$\mathbf{47.08}\pm 0.76$	$\mathbf{52.91}\pm 0.66$

TABLE III: Offset-token width ablation on the IRSA test set. Results are mean

\pm

sample standard deviation over five random seeds.

Configuration	bAP $\uparrow$	bAP50 $\uparrow$	sAP $\uparrow$	sAP50 $\uparrow$	FAP50 $\uparrow$	F1 $\uparrow$	B-IoU $\uparrow$	mEPE $\downarrow$	PCP@5 $\uparrow$
Disabled	24.02	40.59	23.06	40.57	37.21	43.91	21.13	3.29	84.58
Multiplicative	28.33	47.32	27.49	47.66	44.21	50.24	22.29	3.04	85.52
Additive	30.27	49.57	29.17	49.65	46.79	52.54	22.27	2.71	88.09

TABLE IV: Ablation of the target-indexed RFO denoising path on the IRSA test set with

d=5

. Multiplicative and additive denote perturbations applied to the initial offset token.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

$d$	Token scalars $Kd$	Readout parameters $2d+2$
5	1,500	12
7	2,100	16
256	76,800	514

Appendix

TABLE B.1: Offset-representation size and affine-readout parameters, including bias, for

K=300

Explore similar work

Joint Instance Segmentation and Geometric Attribute Regression for Roof Structures in Aerial Imagery
May 25, 2026Luuk Versteeg, Rob G. J. Wijnhoven, Martin R. OswaldAerial Imagery Large-Scale 3D Editing Dataset
University of Amsterdam (UvA), The Netherlands · Spotr.ai, The Hague, The Netherlands
PCFootprint: A Large-Scale Dataset and Benchmark for Vectorized Building Footprint Extraction from Aerial LiDAR Point Clouds
Jun 18, 2026Haoyuan Shen, Kuihao Wang, Ruisheng Wang +1Footprint Point Clouds
School of Architecture and Urban Planning, Shenzhen University, Shenzhen 518060, China
ObliCity: A Benchmark and Baseline for Roof-to-Ground Projection Displacement Correction
Jul 28, 2026Kai Li, Yupeng Deng, Ligao Deng +6Contours Urban Environments
School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, 1 East Yanqi Lake Road, Beijing, 100049, China · cAerospace Information Research Institute, Chinese Academy of Sciences, 9 Dengzhuang South Road, Beijing, 101408, China · dDepartment of Electrical and Computer Engineering, The University of Hong Kong, Pokfulam, Hong Kong, 999077, China +1

May 25, 2026, cs.CVJ/K move · Enter open · S save

Joint Instance Segmentation and Geometric Attribute Regression for Roof Structures in Aerial Imagery

Luuk Versteeg, Rob G. J. Wijnhoven, Martin R. Oswald

University of Amsterdam (UvA), The Netherlands · Spotr.ai, The Hague, The Netherlands

Aerial Imagery Large-Scale 3D Editing Dataset

Open paper AskPDF arXiv

We present a method for jointly predicting instance-level roof segment masks together with three continuous geometric attributes -- building height, roof slope, and roof azimuth -- from a single aerial orthophoto. Our approach extends Mask R-CNN with a dedicated attribute regression branch and introduces two key innovations: a conditional azimuth loss that suppresses supervision for flat roof segments where azimuth labels are inherently noisy, and a log-normalized height representation that addresses the heavily skewed distribution of building heights. We train and evaluate on a large-scale dataset of Dutch aerial images paired with automatically derived ground truth from 3DBAG, a nationwide LiDAR-based 3D building dataset. Using a DINOv3 ConvNeXt-Base backbone, our method achieves a mean absolute error of approximately 4 degrees for roof slope, 7 degrees for azimuth, and 1 meter for building height, with an instance segmentation AP

_{50}

of 0.566. The predicted per-segment masks and attributes are sufficient to reconstruct simplified 3D building models (LoD2) from a single overhead image, requiring expensive 3D reference data only for training.

Abstract

Explore similar work

Joint Instance Segmentation and Geometric Attribute Regression for Roof Structures in Aerial Imagery

PCFootprint: A Large-Scale Dataset and Benchmark for Vectorized Building Footprint Extraction from Aerial LiDAR Point Clouds

ObliCity: A Benchmark and Baseline for Roof-to-Ground Projection Displacement Correction