Spatial Foundry: A Multimodal Dataset for Spatial Intelligence
Organizations: Ranolabs
Abstract
Spatial intelligence requires connecting visual observations to the geometry, objects, and persistent state of an environment. Images alone leave these relationships implicit, while disconnected annotations make it difficult to study appearance, camera motion, and scene changes under a common reference. We introduce Spatial Foundry, a collection of 5,280 furnished residential 3D scene configurations with editable assets and aligned visual, geometric, and temporal data. Its modalities include floor plans, perspective and panoramic RGB, metric depth, point clouds, meshes, instance-level 3D bounding boxes, and camera trajectories paired with video. Shared scene coordinates link observations across views and time; editable source scenes enable additional camera paths and controlled object changes. We describe the data representation and construction workflow, and discuss applications in floor-plan-conditioned 3D generation, geometric reconstruction, instance-level perception, camera-conditioned video generation, shared world modeling, and scene editing. Controlled camera and scene variations support spatial perception and world-model research. The public project page provides videos and an interactive colored point-cloud preview.
Figures & tables
| Layout | Visual observations | Geometry | Control | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Dataset | Plan–3D pairs | Persp. RGB | 360 ∘ RGB | Video / poses | Metric depth | 3D geometry | Point cloud | Boxes / instances | Editable assets | State edits |
| RealEstate10K ( Zhou et al., 2018 ) | – | – | – | – | – | – | – | – | ||
| ScanNet++ ( Yeshwanth et al., 2023 ) | – | – | ||||||||
| Structured3D ( Zheng et al., 2020 ) | – | – | – | |||||||
| 3D-FRONT ( Fu et al., 2021 ) | ||||||||||
| Hypersim ( Roberts et al., 2021 ) | – | |||||||||
| Modality | Spatial or temporal reference |
|---|---|
| Floor plans | Layout, room boundaries, metric geometry |
| Perspective / 360 RGB | Camera model, pose, scene identity |
| Metric depth | Projection, units, depth convention, mask |
| Point clouds / meshes | World coordinates, scale, object association |
| 3D instances | ID, bounds, dimensions, transform, label provenance |
| Trajectories / video | Native frame ID, time, camera pose, intrinsics |