ARCH-B: Architectural Representation, Comprehension and Hierarchy Benchmark
Organizations: Harvard University · Northeastern University
Abstract
Multimodal models increasingly interpret visual environments, but their ability to recognize the same building across photographs, floor plans, elevations, sections, and renderings remains poorly characterized. We introduce ARCH-B, a benchmark of 354 four-choice questions across 11 cross-representational archetypes, constructed from a building-linked corpus of 3.9 million architectural images using visually similar distractors, model-guided difficulty screening, and manual validation. We evaluate 25 multimodal models and collect 5,830 responses from non-expert human participants. Model accuracy ranges from 10.45% to 83.90%, compared with a human baseline of 35.35%. Models perform comparatively well on mixed-representation outlier detection and photograph matching, but remain weaker on floorplan-to-photograph correspondence. Human and model difficulty across archetypes is only weakly correlated (Spearman's (ρ=0.33)). Held-out evaluation confirms that the difficulty identified during screening generalizes beyond the curation models. ARCH-B provides a diagnostic evaluation of visual correspondence and representation transfer across architectural media.
Figures & tables
| Rank | Model | Family | Accuracy | Valid Response Rate |
|---|---|---|---|---|
| 1 | gemini-3.1-pro-preview | 83.90% | 97.46% | |
| 2 | gemini-3.5-flash | 83.05% | 98.87% | |
| 3 | claude-fable-5 | Anthropic | 80.23% | 98.87% |
| 4 | gpt-4.1 | OpenAI | 77.97% | 100.00% |
| 5 | gpt-5.5 reasoning | OpenAI | 76.55% | 99.72% |
| 6 | gemini-3-flash-preview | 75.99% | 98.02% |
| Archetype | Task family | Best acc. | Best Model for Archetype | Mean model acc. | Human acc. | Best-human gap |
|---|---|---|---|---|---|---|
| 0 | Match floor plan with images | 95.65 | gemini-3.1-pro-preview | 58.78 | 34.74 | 60.91 |
| 1 | Match image with floor plan | 76.79 | gpt-4.1 | 44.14 | 36.41 | 40.38 |
| 2 | Choose outlying image | 84.78 | gemini-3.1-pro-preview | 41.22 | 21.61 | 63.17 |
| 3 | Choose inlying image | 100.00 | gpt-4.1 | 60.17 | 32.00 | 68.00 |
| 4 | Identify interior from exterior | 96.00 | gemini-3.1-pro-preview | 52.64 | 35.85 | 60.15 |
| 5 | Identify exterior from interior | 85.00 | claude-opus-4-5-20251101-v1.0 | 50.00 | 29.70 | 55.30 |