Visual-to-Executable Procedural Reconstruction of Buildings
Organizations: Shenzhen VisuCA Key Lab, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · The University of Tokyo · Ben-Gurion University of the Negev
Abstract
Procedural modeling offers compact, editable, and reusable representations for architectural content, but building such models still depends heavily on manually authored grammars. We study image-based procedural building reconstruction from in-the-wild images, with the goal of recovering an executable procedural description that can be instantiated as an editable 3D building asset. Because a single image provides only partial and view-dependent evidence, we focus on recovering the dominant visible facade(s) and attach the recovered facade program to a simple mass model for building-level instantiation. Our method decomposes this task into two structured prediction stages. First, a Global Structure Layout Tree recovers facade-scale split-and-repeat organization through a fine-tuned layout model with render-guided visual-symbolic reflection. Second, a Component Parameterization schema maps representative facade tiles to structured procedural attributes for local architectural component synthesis. These two representations are then deterministically compiled into executable Computer Generated Architecture (CGA) programs, avoiding the brittleness of direct low-level program generation. Experiments show that the proposed framework handles both regular facades and challenging in-the-wild images, while maintaining high structural consistency, procedural executability, and fine-grained architectural detail. The recovered programs further enable controllable and reusable 3D building asset generation.
Figures & tables
| Data | W-Arch | W-Sill | W-Row/Col | E-Stair | E-Handle |
| Real-world | 0.98 | 0.92 | 0.89 | 0.95 | 0.96 |
| Synthetic | 0.99 | 0.96 | 0.92 | 0.98 | 0.97 |
| Methods: | FaçAID | Ours | Tie |
| Vote Proportion: | 30% | 59 % | 11% |
| Method | Exec. Succ.↑ | Arch Acc.↑ | Row/Col Acc.↑ | Sill Acc.↑ | Stair Acc.↑ | Handle Acc.↑ |
| Direct Code (Gemini 3.1 Pro Preview) | 0.68 | N/A | N/A | N/A | N/A | N/A |
| Direct Middle Code (Gemini 3.1 Pro Preview) | 0.95 | 0.94 | 0.88 | 0.88 | 0.96 | 0.91 |
| Ours | 0.98 | 0.99 | 0.90 | 0.93 | 0.96 | 0.96 |
| Method | Exec. Succ. | TRC |
| Direct LLM (Gemini 3.1 Pro Preview) | 0.94 | 1.83 |
| Ours | 0.98 | 1.47 |
| Model | Win Cnt. Acc. | Floor Cnt. Acc. | Door Cnt. Acc. | TRC | Exec. Succ. |
| Initial Layout | 0.61 | 0.83 | 0.69 | 4.32 | 0.98 |
| Iteration 1 | 0.77 | 0.85 | 0.78 | 2.01 | 0.98 |
| Iteration 2 | 0.80 | 0.86 | 0.81 | 1.47 | 0.99 |
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Acc (Win) | Acc (Floor) | Acc (Door) | TRC | Exec. Succ. |
| Initial Layout | 0.61 | 0.83 | 0.69 | 4.32 | 0.98 |
| Iteration 1 | 0.77 | 0.85 | 0.78 | 2.01 | 0.98 |
| Iteration 2 | 0.80 | 0.86 | 0.81 | 1.47 | 0.99 |
| Iteration 3 | 0.80 | 0.87 | 0.80 | 1.46 | 0.99 |
| Iteration 4 | 0.78 | 0.87 | 0.79 | 1.48 | 0.99 |
| Split | Tiles | Building groups | Avg. tiles/group |
| Training | 100 | 31 | 3.23 |
| Test | 107 | 36 | 2.97 |
| Total | 207 | 67 | 3.09 |
| Group | Attribute | Synthetic Acc. | Real-world Acc. | Gap |
| Arch structure | Arch style | 0.96 | 0.93 | 0.03 |
| Arch height class | 0.92 | 0.87 | 0.05 | |
| Categorical | Arch | 0.99 | 0.98 | 0.01 |
| Top sill | 0.96 | 0.92 | 0.04 | |
| Bottom sill | 0.95 | 0.93 | 0.02 | |
| Frame attributes | Outer frame thickness | 0.89 | 0.86 | 0.03 |
| Group | Attribute | Synthetic Acc. | Real-world Acc. | Gap |
| Entrance frame | Frame style | 0.97 | 0.93 | 0.04 |
| Frame head height | 0.92 | 0.89 | 0.03 | |
| Frame side width | 0.93 | 0.90 | 0.03 | |
| Entrance structure | Door type | 0.97 | 0.92 | 0.05 |
| Door asymmetric ratio | 0.91 | 0.86 | 0.05 | |
| Door style | 0.88 | 0.88 | 0.00 |
| Model | Has Arch | Arch Style | Arch Height | Has Sill | Sill Height | Sill Protrusion | Frame Thick. | Grid Num. | Grid Ratio |
| CLIP-only | 0.99 | 0.90 | 0.81 | 0.91 | 0.72 | 0.68 | 0.65 | 0.75 | 0.72 |
| DINOv3-only | 0.98 | 0.93 | 0.88 | 0.90 | 0.89 | 0.89 | 0.86 | 0.88 | 0.85 |
| CLIP+DINOv3 | 0.99 | 0.94 | 0.90 | 0.93 | 0.92 | 0.92 | 0.88 | 0.90 | 0.87 |
| Model | Has Stairs | Stair Height | Stair Count | Has Hardware | Hardware Type | Hardware Side | Frame Style | Frame Size | Door Type | Door Asym. Ratio | Door Style | Grid |
| CLIP-only | 0.95 | 0.74 | 0.71 | 0.95 | 0.94 | 0.80 | 0.93 | 0.82 | 0.86 | 0.78 | 0.86 | 0.77 |
| DINOv3-only | 0.93 | 0.83 | 0.85 | 0.93 | 0.91 | 0.84 | 0.89 | 0.88 | 0.90 | 0.85 | 0.87 | 0.84 |
| CLIP+DINOv3 | 0.96 | 0.87 | 0.88 | 0.96 | 0.96 | 0.88 | 0.95 | 0.90 | 0.94 | 0.88 | 0.88 | 0.86 |
| Attribute group | Exact Acc. | MAE |
| Window frame geometry | 0.900 | 0.026 |
| Window grid ratios | 0.880 | 0.031 |
| Window sill geometry | 0.943 | 0.016 |
| Entrance geometry | 0.910 | 0.024 |
| Training data | Overall macro | W-Arch | W-Row/Col | W-Sill | E-Stair | E-Handle |
| Synthetic only | 0.868 | 0.91 | 0.82 | 0.83 | 0.89 | 0.89 |
| Synthetic + real | 0.940 | 0.98 | 0.89 | 0.92 | 0.95 | 0.96 |