cs.CVMar 20, 2026

Synergistic Perception and Generative Recomposition: A Multi-Agent Orchestration for Expert-Level Building Inspection

Authors: Hui Zhong, Yichun Gao, Luyan Liu, Xusen Guo, Zhaonian Kuang, Qiming Zhang, Xinhu Zheng

Organizations: The Hong Kong University of Science and Technology (Guangzhou), Systems Hub, China · Guangdong Provincial Key Lab of Integrated Communication, Sensing and Computation for Ubiquitous Internet of Things, Guangdong, China · The Hong Kong University of Science and Technology, Hongkong · College of Artificial Intelligence, Xi’An Jiaotong University, Xi’An, China

Abstract

Building facade defect inspection is fundamental to structural health monitoring and sustainable urban maintenance, yet it remains a formidable challenge due to extreme geometric variability, low contrast against complex backgrounds, and the inherent complexity of composite defects (e.g., cracks co-occurring with spalling). Such characteristics lead to severe pixel imbalance and feature ambiguity, which, coupled with the critical scarcity of high-quality pixel-level annotations, hinder the generalization of existing detection and segmentation models. To address gaps, we propose FacadeFixer, a unified multi-agent framework that treats defect perception as a collaborative reasoning task rather than isolated recognition. Specifically,FacadeFixer orchestrates specialized agents for detection and segmentation to handle multi-type defect interference, working in tandem with a generative agent to enable semantic recomposition. This process decouples intricate defects from noisy backgrounds and realistically synthesizes them onto diverse clean textures, generating high-fidelity augmented data with precise expert-level masks. To support this, we introduce a comprehensive multi-task dataset covering six primary facade categories with pixel-level annotations. Extensive experiments demonstrate that FacadeFixer significantly outperforms state-of-the-art (SOTA) baselines. Specifically, it excels in capturing pixel-level structural anomalies and highlights generative synthesis as a robust solution to data scarcity in infrastructure inspection. Our code and dataset will be made publicly available.

Explore similar work

CardsList
  1. Can Large Multimodal Models Inspect Buildings? A Hierarchical Benchmark for Structural Pathology Reasoning

    Mar 20, 2026Hui Zhong, Yichun Gao, Luyan Liu +4Zero-Shot Referring Image SegmentationMultimodal Model Evaluation

  2. Beyond Segmentation: Structurally Informed Facade Parsing from Imperfect Images

    Apr 10, 2026Maciej Janicki, Aleksander Plocharski, Przemyslaw MusialskiObject Detection

  3. Cracks in the Foundation: A Civil Infrastructure Dataset to Challenge Vision Foundation Models

    May 18, 2026Nicola Farronato, Niccolo Avogaro, Thomas Frick +6Vision Foundation ModelsInstance Segmentation