cs.CVNov 17, 2026

Any2Building: Unifying Point Clouds and Images for Building Reconstruction via Shared Spatial Queries

Authors: Hongye Hou, Zhuoxiao Li, Liu Xiangyu, Tengxi Wang, Jing Ou, Dongli Wu, Shuai Zhang, Tongyan Hua, 

Organizations: The Hong Kong University of Science and Technology, Guangzhou, China.

Abstract

Abstracting structured buildings from limited observations is essential for 3D urban modeling and digital twins. Partial point clouds and overhead images of the same building provide incomplete 3D surface measurements and visual cues to its footprint, roof geometry, and rooftop structures, respectively. This motivates a unified framework that aligns geometric and visual cues in 3D and reconstructs buildings from either or both modalities. Building on this connection, we present Any2Building, a unified model for Level of Detail 2 (LoD2) building reconstruction from partial point clouds, overhead images, or both, with optional coarse geometric priors. At its core is an occupancy-grounded shared spatial queries representation that organizes available observations within a common 3D grid. Supervised by complete building surface voxels, these queries connect observational evidence to target geometry, enabling a shared generator to learn structural priors across tasks and providing spatial guidance for recovering missing structure. A trained voxel conditional surface-generation and low-poly remeshing pipeline then converts the predicted structure into compact LoD2 building models. To support this framework, we introduce BuildingWorld-Image, comprising approximately one million paired samples of partial point clouds, overhead images, and LoD2 building models across 18 cities. Experiments on multiple benchmarks demonstrate that Any2Building outperforms existing methods in geometric reconstruction accuracy across unimodal and multimodal point-cloud completion, single-image building generation, and prior-assisted generation, while maintaining detailed and robust recovery when point cloud input modality is degraded.

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. BuildAnyPoint: 3D Building Structured Abstraction from Diverse Point Clouds

    Feb 27, 2026Tongyan Hua, Haoran Gong, Yuan Liu +33D Mesh GenerationDiffusion Models

  2. AstraLOD3: Zero-shot multimodal agentic reconstruction of LOD3 building models

    Sep 23, 2026Bryan G. Pantoja-Rosero3D ReconstructionMulti-View 3D Reconstruction

  3. The P3^3 Dataset: Pixels, Points and Polygons for Multimodal Building Vectorization

    May 21, 2025Raphael Sulzer, Liuyun Duan, Nicolas Girard +1Remote Sensing Image SegmentationBuilding Footprint Extraction