LoDEOT: Low-Dimensional and Efficient Offset Tokens for Building Footprint Extraction from Off-Nadir Imagery

Kai Li; Zigan Zhou; Zhenyang Li; Hui Shan; Zhe Chen; Yupeng Deng; Zhihao Xi; Yu Meng

cs.CVOct 5, 2026

LoDEOT: Low-Dimensional and Efficient Offset Tokens for Building Footprint Extraction from Off-Nadir Imagery

Authors: Kai Li, Zigan Zhou, Zhenyang Li, Hui Shan, Zhe Chen, Yupeng Deng, Zhihao Xi, Yu Meng, +2 more

Organizations: City University of Hong Kong · University of the Chinese Academy of Sciences · University of Hong Kong · Zhejiang University · University of Southampton

Abstract

Instance-level roof-to-footprint offset (RFO) prediction is central to extracting building footprints from off-nadir imagery. Query-based pipelines commonly use high-dimensional instance tokens to predict signed two-dimensional RFOs. We investigate whether RFO prediction can instead use a compact offset token. Under local pinhole projection and vertical-extrusion assumptions, the idealized RFO map admits a five-parameter sufficient descriptor comprising intrinsic shape, composite amplitude, and relative geometry. This factorization provides a structural prior for a five-dimensional offset token, whose channels learn task-relevant latent representations through end-to-end training. Based on this design, we propose LoDEOT, which retains high-dimensional instance tokens for detection and segmentation but maps instance-token, concentration-gated roof, and box-mask evidence to a five-dimensional offset token followed by an independent two-dimensional readout. Known denoising-query target indices further align each supervised decoder-layer estimate with the same clean instance RFO, organizing successive predictions as target-aligned recovery under perturbed query conditions. Experiments on five real-world building datasets demonstrate the effectiveness of LoDEOT for building footprint extraction. Experiments on real-world building datasets demonstrate that a five-dimensional offset token can support accurate RFO prediction. On BONAI, LoDEOT achieves the best roof-detection bAP and bAP50 and leads all five offset-corrected footprint metrics among the evaluated end-to-end methods, with FAP50 of 54.58 and mEPE of 5.23 pixels. Its FAP50 exceeds those of the evaluated end-to-end baselines by 7.56-16.85 percentage points.

Figures & tables

Fig. 1: RFO prediction architecture after incorporating LoDEOT. Motivated by the projective geometry of 3D imaging, we redesign the offset-token representation. After the backbone encodes the visual features, each original high-dimensional instance token is augmented with a 5-D LoDEOT that carries the task-specific RFO representation, while retaining its roles in classification, roof-mask, and roof-box prediction. The augmented instance tokens are progressively updated through $L$ Transformer decoder layers; the final head predicts roof masks and RFOs to recover footprints. During training, each layer’s RFO prediction is supervised by the same clean instance target.

Externally Guided Methods
Method	Roof Detection and Segmentation				Offset-Corrected Footprint
Method	bAP $\uparrow$	bAP50 $\uparrow$	sAP $\uparrow$	sAP50 $\uparrow$	FAP50 $\uparrow$	F1 $\uparrow$	B-IoU $\uparrow$	mEPE $\downarrow$	PCP@5 $\uparrow$
OBM	–	–	12.59	36.71	22.21	41.72	9.25	5.72	53.68
PolyFootNet	–	–	8.91	33.18	18.76	39.75	7.32	6.54	44.71
End-to-End Methods
LOFT	10.50	42.90	21.00	59.50	37.73	49.64	8.78	5.65	57.74

TABLE I: Performance comparison of externally guided and end-to-end methods on the BONAI test set. The best results among the end-to-end methods are highlighted. All metrics are percentages except mEPE, which is measured in pixels.

Fig. 2: Qualitative comparison of OBM, PolyFootNet, LOFT, Cas.LOFT, MLS-BRN, and LoDEOT (ours) on three representative BONAI scenes. Each row shows the same scene across methods, and each column corresponds to one method. Yellow denotes true-positive matches (IoU $\geq 0.5$ ), cyan unmatched predictions (false positives), and red unmatched ground-truth instances (false negatives). OBM and PolyFootNet use LoDEOT-predicted masks as guidance.

Level	BONAI		IRSA		WHU		UAV		Huizhou		Average
Level	sAP50	FAP50	sAP50	FAP50	sAP50	FAP50	sAP50	FAP50	sAP50	FAP50	sAP50	FAP50
L1	54.94	54.58	9.70	8.77	49.50	48.75	31.64	28.49	35.72	31.17	36.30	34.35
L2	54.68	51.58	51.67	48.76	26.65	26.60	14.03	12.98	8.82	5.56	31.17	29.09
L3	54.21	51.76	49.42	46.30	96.08	96.08	24.25	22.65	31.22	25.01	51.04	48.36
L4	55.22	52.29	49.27	46.34	95.55	95.54	44.99	41.75	39.13	32.28	56.83	53.64

TABLE II: Evaluation of LoDEOT with

d=5

under cumulative multi-source training. The average is the unweighted mean over the five test domains.

$d$	bAP $\uparrow$	bAP50 $\uparrow$	sAP $\uparrow$	sAP50 $\uparrow$	FAP50 $\uparrow$	F1 $\uparrow$
2	$29.66\pm 1.20$	$49.10\pm 1.56$	$28.66\pm 1.20$	$48.96\pm 1.67$	$45.67\pm 1.60$	$51.60\pm 1.20$
3	$29.63\pm 0.82$	$48.96\pm 1.42$	$28.73\pm 0.83$	$49.01\pm 1.48$	$45.76\pm 1.28$	$51.62\pm 1.34$
4	$29.04\pm 1.25$	$48.03\pm 2.04$	$28.16\pm 1.19$	$48.03\pm 2.06$	$45.07\pm 1.94$	$50.97\pm 1.55$
5	$29.42\pm 1.50$	$48.54\pm 2.35$	$28.41\pm 1.47$	$48.38\pm 2.46$	$45.20\pm 2.52$	$50.93\pm 2.16$
6	$30.08\pm 0.97$	$49.68\pm 0.74$	$29.10\pm 0.76$	$49.62\pm 0.72$	$46.32\pm 0.98$	$51.97\pm 0.69$
7	$\mathbf{30.61}\pm 0.51$	$\mathbf{50.31}\pm 0.79$	$\mathbf{29.62}\pm 0.48$	$\mathbf{50.35}\pm 0.89$	$\mathbf{47.08}\pm 0.76$	$\mathbf{52.91}\pm 0.66$

TABLE III: Offset-token width ablation on the IRSA test set. Results are mean

\pm

sample standard deviation over five random seeds.

Configuration	bAP $\uparrow$	bAP50 $\uparrow$	sAP $\uparrow$	sAP50 $\uparrow$	FAP50 $\uparrow$	F1 $\uparrow$	B-IoU $\uparrow$	mEPE $\downarrow$	PCP@5 $\uparrow$
Disabled	24.02	40.59	23.06	40.57	37.21	43.91	21.13	3.29	84.58
Multiplicative	28.33	47.32	27.49	47.66	44.21	50.24	22.29	3.04	85.52
Additive	30.27	49.57	29.17	49.65	46.79	52.54	22.27	2.71	88.09

TABLE IV: Ablation of the target-indexed RFO denoising path on the IRSA test set with

d=5

. Multiplicative and additive denote perturbations applied to the initial offset token.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

$d$	Token scalars $Kd$	Readout parameters $2d+2$
5	1,500	12
7	2,100	16
256	76,800	514

Appendix

TABLE B.1: Offset-representation size and affine-readout parameters, including bias, for

K=300

Explore similar work

May 25, 2026cs.CV

Joint Instance Segmentation and Geometric Attribute Regression for Roof Structures in Aerial Imagery

We present a method for jointly predicting instance-level roof segment masks together with three continuous geometric attributes -- building height, roof slope, and roof azimuth -- from a single aerial orthophoto. Our approach extends Mask R-CNN with a dedicated attribute regression branch and introduces two key innovations: a conditional azimuth loss that suppresses supervision for flat roof segments where azimuth labels are inherently noisy, and a log-normalized height representation that addresses the heavily skewed distribution of building heights. We train and evaluate on a large-scale dataset of Dutch aerial images paired with automatically derived ground truth from 3DBAG, a nationwide LiDAR-based 3D building dataset. Using a DINOv3 ConvNeXt-Base backbone, our method achieves a mean absolute error of approximately 4 degrees for roof slope, 7 degrees for azimuth, and 1 meter for building height, with an instance segmentation AP

_{50}

of 0.566. The predicted per-segment masks and attributes are sufficient to reconstruct simplified 3D building models (LoD2) from a single overhead image, requiring expensive 3D reference data only for training.

Luuk Versteeg, Rob G. J. Wijnhoven, Martin R. Oswald

University of Amsterdam (UvA), The Netherlands · Spotr.ai, The Hague, The Netherlands

Aerial Imagery Large-Scale 3D Editing Dataset

Ask PDF

Jun 18, 2026cs.CV

PCFootprint: A Large-Scale Dataset and Benchmark for Vectorized Building Footprint Extraction from Aerial LiDAR Point Clouds

Building footprint extraction is a fundamental task in photogrammetry, remote sensing, and computer vision. Recent image-based methods have achieved remarkable progress in extracting vectorized footprints from high-resolution optical imagery. However, optical imagery inherently susceptible to occlusions, perspective distortions, and residual relief displacement, yielding incomplete or misaligned footprint extraction. Furthermore, the lack of explicit elevation information limits its direct applicability to Level of Detail building modeling. In this paper, we present PCFootprint, the first large-scale public dataset for footprint extraction from airborne laser scanning point clouds. PCFootprint comprises \num{33000} tiles derived from the Estonian Land and Spatial Development Board, covering diverse urban and rural landscapes. Each tile spans \qtyproduct{128 x 128}{\m} with systematically aligned vectorized footprints aligned to point clouds. The dataset includes a \num{3000} tiles cross-domain test set for evaluating generalization across geographic regions. We establish comprehensive benchmarks by evaluating mainstream methods. Experimental results reveal significant challenges including high intra-class variance, data imbalance, and noise across complex geospatial environments. We believe PCFootprint will advance future research in building modeling, urban scene understanding, and geospatial analysis. The PCFootprint dataset is publicly available at https://huggingface.co/datasets/Haoyuan-Shen/PCFootprint.

Haoyuan Shen, Kuihao Wang, Ruisheng Wang +1

School of Architecture and Urban Planning, Shenzhen University, Shenzhen 518060, China

Footprint Point Clouds Land Cover Building Information Modeling Large-Scale 3D Editing Dataset Contours+2

Ask PDF

Jul 28, 2026cs.CV

ObliCity: A Benchmark and Baseline for Roof-to-Ground Projection Displacement Correction

Oblique-view urban remote sensing imagery inevitably exhibits geometric projection displacements between building roofs and footprints, leading to significant distortions in spatial structure. Existing approaches either ignore these deformations or handle them implicitly within segmentation-based frameworks, where progress is dominated by general segmentation advances rather than improvements in geometric correction. In this work, we explicitly define roof-to-footprint offset vector (RFOV) extraction as an independent learning task that decouples geometric alignment from semantic segmentation. To support this task, we introduce the Oblique City dataset (ObliCity), the first large-scale benchmark that integrates high-resolution UAV imagery and globally distributed satellite data, covering diverse city morphologies and camera perspectives. Methodologically, we reformulate DragOSM into DragRoof, an ODE-based framework inspired by human annotation behavior. By simulating the continuous process of dragging roofs toward their footprints, DragRoof learns deterministic, geometry-consistent offset fields and adaptively determines convergence through an end token. Extensive experiments on ObliCity demonstrate that DragRoof achieves state-of-the-art RFOV extraction performance, requiring fewer inference steps while delivering superior directional and length accuracy. Our dataset and model establish a principled foundation for studying projection displacement correction in oblique remote sensing imagery. The source code and dataset will be avaliable at https://github.com/likaiucas/DragRoof.

Kai Li, Yupeng Deng, Ligao Deng +6

School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, 1 East Yanqi Lake Road, Beijing, 100049, China · cAerospace Information Research Institute, Chinese Academy of Sciences, 9 Dengzhuang South Road, Beijing, 101408, China · dDepartment of Electrical and Computer Engineering, The University of Hong Kong, Pokfulam, Hong Kong, 999077, China +1

Contours Urban Environments Semantic Segmentation

Ask PDF