Coarse-to-fine 3D-native generation relies heavily on the first-stage sparse structure, which determines the global orientation and overall geometry for subsequent refinement. However, existing coarse generators suffer from two key limitations: their orientation is difficult to control due to ambiguous semantic canonicalization, and their structures are often inconsistent with input images due to weak 2D-3D grounding. We propose Coarse3D, a sparse-structure generation model that produces orientation-controllable and structurally faithful voxel geometry from arbitrary unposed images. Coarse3D introduces three designs: a first-frame anchored canonical space that aligns generation with the first input view while preserving object-centric normalization; a dual-tower DiT that couples generation with auxiliary feed-forward reconstruction to learn spatially grounded features; and spatial-prior guided attention that regularizes cross-attention with pixel-to-voxel correspondence. Coarse3D can directly replace the coarse stage of existing pipelines. Experiments show that it substantially improves sparse structure quality and consistently enhances final 3D generation, achieving state-of-the-art performance.
Figures & tables
Figure 1: Coarse3D generates orientation-controllable and structurally faithful sparse structures. Trellis2, ReconViaGen and Pixal3D use their own coarse and refinement stages while Ours+Trellis2 replaces only the coarse stage with Coarse3D and reuses the Trellis2 refinement stage. Each cell shows the coarse voxel structure ( left ) and the final shape it yields ( right ). Coarse3D consistently follows the first-view orientation and preserves structural fidelity across diverse objects, directly translating into higher-quality final shapes through the coarse-to-fine pipeline.
Figure 2: Comparison of three coordinate definitions: canonical space uses a fixed semantic orientation that is ambiguous; pixel-aligned camera space is aligned with the input view but not object-centric; ours is aligned with the first input view and object-centric.
Figure 3: Overview of Coarse3D. (A) First-frame Anchoring rotates the canonical geometry to zero the first view’s azimuth, turning orientation prediction into a well-defined, first-frame-conditioned task. (B) Dual-tower DiT couples an Image-Tower, supervised by auxiliary tasks ( Lrecon ) to encode cross-view geometry, with a Geometry-Tower predicting the velocity field ( Lgeo ) via layer-wise cross-attention, trained end-to-end. (C) Spatial-prior-guided Attention regularizes cross-attention with a rasterized voxel-to-pixel prior ( Lattn ), strengthening 2D-3D correspondence.
GSO
Toys4K
Method
IoU ↑
PA-IoU ↑
CD ↓
IoU ↑
PA-IoU ↑
CD ↓
TRELLIS-MV
0.2243
0.2909
0.0320
0.3516
0.3785
0.0246
TRELLIS2-MV
0.3098
0.3655
0.0274
0.3840
0.4412
0.0221
ReconViaGen
0.2323
0.2984
0.0293
0.2818
0.3228
0.0271
Pixal3D-MV *
0.4419
0.4420
0.0181
0.5079
0.5080
0.0133
Coarse3D (Ours)
0.5101
0.5364
0.0124
0.6315
0.6368
0.0091
Table 1: Coarse structure generation on GSO and Toys4K. IoU is orientation-sensitive, PA-IoU factors out global orientation, and CD is computed after rigid alignment. Note that Pixal3D-MV * relies on ground-truth camera poses to align and fuse features across the multiple input views, whereas the other methods operate on fully unposed images. Best in bold.
Method
GSO
Toys4k
CD ↓
F-Score ↑
ULIP ↑
Uni3D ↑
CD ↓
F-Score ↑
ULIP ↑
Uni3D ↑
TRELLIS-MV
0.0772
0.4021
0.2112
0.2557
0.0397
0.6157
0.2373
0.3610
+ Ours
0.0128
0.8413
0.2255
0.2688
0.0131
0.8600
0.2461
0.3690
TRELLIS2-MV
0.0613
0.5425
0.2073
0.2607
0.0360
0.6522
0.2373
0.3652
+ Ours
0.0127
0.8576
0.2183
0.2650
0.0099
0.9128
0.2391
0.3684
ReconViaGen
0.0785
0.3835
0.2245
0.2707
0.0490
0.5096
0.2519
0.3687
Table 2: Final shape quality when only the coarse stage is replaced by Coarse3D ( + Ours ). CD and F-Score measure geometric accuracy, while ULIP and Uni3D measure semantic similarity to the input images. Bold marks the better result of each pair. Note that Pixal3D-MV * uses GT cameras.
Figure 4: Qualitative comparison across three baselines (TRELLIS, TRELLIS2, and ReconViaGen). For each baseline, we show the original result ( Baseline ) and the result after replacing its first stage with our Coarse3D ( + Ours ). Anchored by the first frame of the multi-view input (leftmost), our method yields (i) more controllable orientations , (ii) higher structural fidelity , and (iii) seamless plug-and-play integration without modifying subsequent stages.
Method
GSO
Toys4k
RRE ↓
RTE ↓
CD ↓
RRE ↓
RTE ↓
CD ↓
VGGT
10.54
0.2010
0.0226
16.11
0.2352
0.0160
ReconViaGen
4.09
0.0934
0.0203
8.15
0.1293
0.0144
RecGen3D *
2.16
0.0338
–
3.67
0.0637
–
Ours
2.02
0.0319
0.0174
3.43
0.0370
0.0128
Table 3: Auxiliary reconstruction accuracy. RRE and RTE are relative camera rotation and translation errors, and CD is measured on the predicted point maps after rigid alignment. * numbers are copied from the original paper. Best in bold.
Setting
Coarse Generation
Reconstruction
IoU ↑
PA-IoU ↑
CD ↓
RRE ↓
RTE ↓
Baseline
0.4041
0.4677
0.0280
–
–
+ Dual-DiT under canonical space
0.4652
0.5329
0.0117
5.88
0.0692
+ First-frame anchored canonical space
0.5586
0.5704
0.0104
3.42
0.0413
+ Spatial-Prior guided Attention
0.6392
0.6550
0.0087
3.10
0.0399
Table 4: Cumulative ablation on the three key components of Coarse3D. Each row adds one component to the row above. The baseline has no image tower and therefore no reconstruction metrics.
Figure 5: Effect of spatial-prior guided attention. For a representative voxel query, we visualize the attention map over pixel tokens of four input views. Top : spatial prior Asp derived from geometric projection. Middle : base model attention Adense , which spreads diffusely across the object. Bottom : our guided model concentrates on the pixels the query voxel actually projects to.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Space Type
View-dependent
Alignment of Orientation Angles
Standard Canonical Space
✗
none
Pixel-aligned Camera Space
✓
all (azimuth, elevation and roll)
First-frame Anchored Canonical Space (ours)
✓
only azimuth
Appendix
Table 5: Summary of three types of 3D generative spaces.
GSO
Toys4K
# Views
IoU ↑
PA-IoU ↑
CD ↓
IoU ↑
PA-IoU ↑
CD ↓
1
0.3129
0.3495
0.0285
0.3876
0.4148
0.0236
2
0.4200
0.4534
0.0187
0.5234
0.5375
0.0163
3
0.4834
0.5075
0.0168
0.6045
0.6149
0.0131
4
0.5101
0.5364
0.0124
0.6315
0.6368
0.0091
5
0.5244
0.5449
0.0115
0.6419
0.6465
0.0087
Appendix
Table 6: Ablation on the number of input views. We vary the number of input images from 1 to 6 and report the coarse structure reconstruction quality of Coarse3D on GSO and Toys4K.