cs.CVNov 5, 2026
SaveBri3D: Bridging the Object-Scene Gap in Single-Image 3D Generation via Scene-Anchor
Organizations: The Hong Kong University of Science and Technology (Guangzhou) · Taobao & Tmall Group of Alibaba · The Hong Kong University of Science and Technology
Abstract
Generating compositional 3D scenes from a single image requires recovering detailed object geometry and coherent spatial arrangements. Existing object-to-scene methods, including SAM3D and MIDI, extend object generation through image-context reasoning or inter-instance modeling. Coordinating objects remains challenging with limited scene-level supervision. Directly extending object-centric generators to entire scenes can degrade geometry and fail to preserve instance scale, orientation, and pose. We propose Bri3D, a framework bridging scene-level structural reasoning and object-level generative priors through instance-aware voxel scene anchors. Bri3D predicts a color-coded scene scaffold encoding coarse object geometry and spatial arrangements. Each instance is extracted as a scene anchor and refined into a pose-preserving, prior-compatible voxel support, enabling a frozen object generator to synthesize detailed textured geometry in the anchor-defined spatial reference. This coarse-to-fine design separates scene organization from object-scale detail synthesis, focusing scene-level training on spatial organization and anchor refinement while reusing pretrained object priors. Experiments demonstrate improvements in scene layout, object geometry, visual quality, and generalization. Bri3D predicts scene layouts efficiently; full scene generation is faster than SceneGen and SAM3D.
Figures & tables
Figure 1: Bridging the object–scene generation gap. Object-to-scene methods, such as MIDI and SAM3D, extend object generation to scene composition, where coordination errors can still produce collisions and implausible arrangements. Object-centric generators produce high-fidelity individual shapes, but their naive extension to full scenes causes structural collapse and distorted geometry. In contrast, Bri3D first generates a coherent instance-aware scene scaffold and then performs scene-anchored prior refinement, producing pose-preserving voxel supports for detail synthesis.
Figure 2: Overview of Bri3D. The Scene Scaffold Generator and Anchored Voxel Refiner replace the voxel-generation stage of a two-stage object generator, while its detail-synthesis stage remains frozen. SC-VAE denotes the Structure-Color VAE, and snowflakes indicate frozen modules. The scaffold is generated in latent space and decoded at ; each object is refined to a voxel support. The anchor initialization follows Eq. 5 , including object-wise latent normalization.
| Method | Geometry | Visual Quality | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| CD-S | CD-O | F-S | F-O | BBox IoU | PSNR | SSIM | LPIPS | FID | CLIP-S | DINO-S | |
| DepR | 9.07 | 18.81 | 78.91 | 56.15 | 0.3943 | — | — | — | — | — | — |
| Gen3DSR | 8.21 | 17.18 | 79.07 | 60.41 | 0.4456 | 16.30 | 0.8639 | 0.1621 | 19.96 | 0.9009 | 0.8139 |
| PartCrafter | 7.05 | — | 85.66 | — | — | — | — | — | — | — | — |
| SceneGen | 6.11 | 7.07 | 87.72 | 87.73 | 0.5382 | 16.50 | 0.8723 | 0.1552 | 19.80 | 0.9005 | 0.8201 |
| MIDI | 2.14 | 2.81 | 92.20 | 89.57 | 0.6238 | 16.97 | 0.8756 | 0.1522 | 19.78 | 0.9315 | 0.8247 |
Table 1: Quantitative comparisons on the 3D-FRONT [ 12 ] test set. Geometric metrics: scene/object-level Chamfer Distance (CD-S/CD-O, ), F-Score (F-S/F-O, %), and BBox IoU ( ). Visual metrics: PSNR, SSIM, LPIPS, FID, CLIP-S, DINO-S.
| Method | CD-S | F-S | CD-O | F-O | BBox IoU | LPIPS | DINO-S |
|---|---|---|---|---|---|---|---|
| DepR | 14.20 | 64.80 | 13.85 | 66.10 | 0.2420 | - | - |
| Gen3DSR | 12.70 | 68.90 | 12.35 | 69.70 | 0.2785 | 0.3328 | 0.7218 |
| PartCrafter | 11.30 | 74.60 | — | — | — | - | - |
| SceneGen | 12.10 | 71.20 | 11.56 | 72.80 | 0.2864 | 0.3187 | 0.7382 |
| MIDI | 6.20 | 80.50 | 5.74 | 82.60 | 0.4315 | 0.2635 | 0.7756 |
| SAM3D | 3.85 | 87.10 | 3.48 | 89.30 | 0.5628 | 0.2218 | 0.8021 |
Table 2: Cross-dataset generalization results. We evaluate scene- and object-level geometry, layout accuracy, and visual similarity on 300 scenes paired with rendered images and ground-truth 3D scenes. CD-S and CD-O are scaled by ; F-S and F-O are F-Score percentages; BBox IoU is on the scale. The best results are in bold; the second-best results (SAM3D in all reported columns) are underlined.
Figure 3: Qualitative results on the 3D-FRONT [ 12 ] test set. Our method generates more plausible layouts and higher-fidelity geometry, consistent with the quantitative improvements. Red boxes indicate inferior baseline results.
Figure 4: Generalization to diverse room scenes. The upper three rows compare reconstructions of a living room, a garage, and a classroom, with the input image followed by the results of DepR, PartCrafter, MIDI, SceneGen, SAM3D, and Bri3D. These examples cover different object categories, scales, and degrees of occlusion. The lower row shows a multi-room overview and reconstructions of its bathroom, bedroom, living room, and kitchen, each generated from a separate room-level input. Together, the examples illustrate object reconstruction across varied scene configurations and room types.
| Model | 1–10 Objects | 11–20 Objects | Collision Rate (%) | ||
|---|---|---|---|---|---|
| BBox IoU | Test Time | BBox IoU | Test Time | ||
| DepR | 0.4032 | 15.3s | 0.3008 | 20.8s | 22 |
| MIDI | 0.6349 | 40.4s | 0.4193 | 72.9s | 14 |
| SceneGen | 0.5892 | 135.0s | 0.4462 | 230.2s | 38 |
| SAM3D | 0.6568 | 64.1s | 0.4609 | 179.6s | 34 |
| Ours ∗ | 0.7980 | 5.1s (42.6s) | 0.6732 | 5.2s (78.4s) | 4 |
Table 3: Comparison of layout accuracy, inference time, and observed object collisions. BBox IoU is in ; collision rate is the percentage of overlapping object pairs. ∗ Ours reports scaffold-stage anchor BBox IoU and collisions after separately voxelizing extracted instances. Times outside parentheses are for scaffold prediction; parentheses and baseline times report full generation.
Figure 5: Comparison of 3D scene layouts and failure cases. (a) Columns show the RGB input, ground-truth layout, and predictions from MIDI [ 18 ] , SceneGen [ 28 ] , SAM3D [ 3 ] , and Bri3D. Red wireframe boxes mark the ground-truth object extents, making displacement, missing instances, and shape deviations visible. Bri3D better follows the reference arrangements in these examples. (b) Red rectangles highlight our failures: nearby objects may merge in the cluttered seating area (Case 1), and some instances may be omitted in the densely populated classroom (Case 2).
| Variant | CD-S/F-S | CD-O/F-O | BBox IoU | |
|---|---|---|---|---|
| Baseline | ||||
| (a) | TRELLIS-FT ( ) | 10.80/52.91 | – | – |
| Scene Scaffold Generator | ||||
| (b) | One-hot representation + CE loss ( ) | 13.00/43.24 | 13.90/41.23 | 0.3144 |
| (c) | Color representation + loss ( ) | 7.24/55.57 | 8.61/53.55 | 0.3902 |
| (d) | + Rendering-based label loss | 5.47/69.88 | 7.32/60.82 | 0.4653 |
Table 4: Ablation studies of Bri3D. Rows (b)–(f) progressively introduce the components of the Scene Scaffold Generator. Rows (g)/(h) compare interpolation and latent refinement from (f); (i)/(j) apply detail synthesis after (h). All retain the anchor boxes and BBox IoU of (f). CD-S/F-S and CD-O/F-O report scene/object metrics. CD is scaled by , F-Score is in percent, and BBox IoU is in . The best results within each component block are highlighted in bold.
Figure 6: Color-coded scene scaffold vs. connected component analysis. From left to right: RGB input, instance label map, geometry-only CCA result, and the predicted color-coded voxel grid. In the upper example, CCA groups the refrigerator, microwave, and table into one connected region; in the lower example, it merges the touching bed components and teddy bear. The predicted instance colors provide additional separation cues at these contacts, allowing joint position–color clustering to recover distinct object anchors. Colors indicate instance identity rather than surface texture.
Figure 7: Preliminary attempt on structural elements. From left to right: RGB input, label map including structural regions, predicted color-coded voxel grid, and reconstructed scene mesh. The dining-area and bedroom examples include room surfaces together with foreground furniture, illustrating how structural labels can extend the scaffold beyond object instances. This extension depends on reliable labels for those regions and remains preliminary; floors, walls, and ceilings are excluded from the main Bri3D experiments and quantitative results.
Figure 8: Additional qualitative comparisons and robustness to imperfect segmentation. (a) Geometry and layout comparisons with MIDI and SAM3D on indoor and outdoor scenes; MIDI is shown with normal shading. Bri3D produces coherent layouts and faithful object geometry, while MIDI suffers from distorted shapes and implausible arrangements. (b) With an erroneous instance label map (Wrong Seg), Bri3D exhibits only local cracks, whereas SAM3D produces incorrect object partitions and collisions, including interpenetration between the teddy bear and the bed.
Figure 9: Robustness to instance segmentation granularity, label colors, and implausible partitions. From left to right: input image, input instance segmentation, predicted semantic voxel grid, and reconstructed textured meshes with bounding boxes. The first two rows show the same tabletop scene containing a bottle, a mug, a large box, a ball, and two small stacks of blocks. The segmentation changes only the grouping of these small stacks: each stack receives one label in the first row, while its individual blocks receive separate labels in the second row. The last two rows show deliberately implausible partitions of a chair, together with changes in instance label colors. Despite these inconsistent instance boundaries, Bri3D preserves the overall object geometry and spatial arrangement.