3D learning systems implicitly assume that objects occupy a coherent reference frame. Nonetheless, in practice, every asset arrives with an arbitrary global rotation, and models are left to resolve directional ambiguity on their own. This persistent misalignment suppresses pose-consistent generation, and blocks the emergence of stable directional semantics. To address this issue, we construct \methodName{}, a massive canonical 3D dataset of 320K objects over 1,156 categories -- an order-of-magnitude increase over prior work. At this scale, directional semantics become statistically learnable: Canoverse improves 3D generation stability, enables precise cross-modal 3D shape retrieval, and unlocks zero-shot point-cloud orientation estimation even for out-of-distribution data. This is achieved by a new canonicalization framework that reduces alignment from minutes to seconds per object via compact hypothesis generation and lightweight human discrimination, transforming canonicalization from manual curation into a high-throughput data generation pipeline. The Canoverse dataset will be publicly released upon acceptance. Project page: https://github.com/123321456-gif/Canoverse
Figures & tables
Figure 1: We present CanoVerse, a large-scale canonical dataset standardizing object orientation, size, and position with intra- and inter-category alignment. It contains 1,156 categories and 320k objects, making it the largest canonical dataset to date, an order of magnitude larger than existing ones.
Figure 2: Scale and category coverage of different canonical datasets. Our dataset boasts the largest number of categories and objects among existing canonical datasets—far surpassing each individual one and even the total of all aforementioned datasets.
Figure 4: For symmetric categories, we annotate the symmetry axis and angle (left). Due to diverse object shapes, unified annotation is impractical. We thus design separate standards based on object characteristics and generate candidate poses in the vertical (middle) and horizontal (right) directions.
Figure 5: Annotation statistics. 750K annotated; 51 % discarded for quality, 6 % for pose errors, 43 % retained (middle). Main discard types including thin-shell meshes, meaningless structures (left). Selection distribution over the 5 candidates (right).
Figure 6: Category coverage (left) and long-tailed distribution with more shapes per category than COD (right). Vertical axis: linear below 0.1, exponential above.
Figure 7: Qualitative comparison of the performance between our model and Orient Anything [ 30 ] on the OmniObject3D [ 34 ] dataset shows that the model trained on our dataset has better generalization ability.
Method
Acc@ 10∘↑
Acc@ 30∘↑
Abs ↓
PCA
22.98
34.83
91.49
CleanPose(CanoVerse) [ 15 ]
35.02
57.36
60.72
AGPose(CanoVerse) [ 16 ]
23.80
48.88
70.39
SecondPose(CanoVerse) [ 4 ]
49.15
58.39
60.11
VI-Net(Objaverse-OA) [ 14 ]
21.77
25.86
97.69
VI-Net(CanoVerse) [ 14 ]
55.31
64.14
51.47
Table 1: In-distribution comparisons for orientation estimation. Best results in bold .
Figure 8: Scalability Analysis on the in-distribution dataset. Left: COD vs. Ours at 32K samples. Right: Ours performance versus dataset size (32K → 310K). For metrics: higher Acc@10° and Acc@30° are better; lower Abs error is better.
Method
Acc@ 10∘↑
Acc@ 30∘↑
Abs ↓
GenPose++ [ 40 ]
4.33
6.95
105.17
Orient Anything [ 30 ]
4.77
7.32
103.11
VI-Net(Nocs) [ 14 ]
3.92
6.64
108.51
VI-Net(Objaverse-OA) [ 14 ]
6.56
9.60
105.90
VI-Net(CanoVerse) [ 14 ]
20.18
24.83
94.60
Table 2: Out-of-distribution orientation estimation results on the real-world scanned dataset OmniObject3D [ 34 ] .
Figure 9: Qualitative analysis of 3D object generation. Fine-tuning with canonical representation outperforms fine-tuning without it in semantics, aspect ratio, and structure.
Toys4K
GSO
Method
canon.
CD ↓
LPIPS ↓
CLIP ↑
IQR( ∘ ) ↓
CD ↓
LPIPS ↓
CLIP ↑
IQR( ∘ ) ↓
Trellis [ 37 ]
w/o
0.227
0.436
89.7
48.5
0.272
0.468
88.0
74.0
w
0.095
0.254
93.9
7.7
0.072
0.244
92.5
8.1
Hun. 2.1 [ 28 ]
w/o
0.258
0.413
90.8
75.59
0.244
0.471
88.1
74.02
w
0.085
0.260
93.7
8.1
0.090
0.288
90.6
18.6
Table 3: 3D Object Generation comparison results. Lower IQR(degree) indicates better pose stability. All variants are fine-tuned for the same total training time.
Canoverse
OmniObject3D
Method
Canon.
Modality
Re.@10 ↑
Re.@30 ↑
Re.@10 ↑
Re.@30 ↑
ULIP [ 38 ]
w/o
Text
21.33
35.17
8.19
15.98
Uni3d [ 44 ]
w/o
Text
51.29
62.18
34.21
48.38
ULIP [ 38 ]
w
Text
25.05
39.43
9.95
19.89
Uni3d [ 44 ]
w
Text
53.88
64.31
35.88
49.02
ULIP [ 38 ]
w/o
Image
19.03
32.39
8.46
16.55
Table 4: Quantitative Comparison of Retrieval Performance. Higher Recall@10 and Recall@30 scores indicate better retrieval performance.
Method
Speed (s/object)
Precision (°)
Low
Medium
High
Low
Medium
High
Blender (Manual)
125.3
101.4
94.5
8.2
4.5
3.7
COD [ 10 ]
4.6
3.4
3.0
9.4
8.5
8.1
Ours
4.0
2.7
2.6
6.7
6.3
5.9
Table 5: Annotation Speed and Precision Comparison. Smaller is better for both speed (s/object) and precision (°). Best results are in Bold , and second best is underlined .
Method
Aeroplane
Car
Bowl
Camera
Bottle
Can
Mug
Avg
IC
GEC
IC
GEC
IC
GEC
IC
GEC
IC
GEC
IC
GEC
IC
GEC
IC
GEC
CaCa [ 26 ]
0.785
0.869
1.985
1.881
1.569
1.895
1.294
1.451
2.632
3.080
1.797
2.095
0.661
0.655
1.532
1.704
ConDor [ 23 ]
0.823
1.391
0.605
0.782
0.898
1.202
1.035
1.529
1.108
1.261
1.048
1.753
0.126
0.189
0.806
1.158
ShapeMat. [ 6 ]
0.659
2.337
2.409
3.596
0.368
0.452
0.377
1.491
1.279
3.791
1.369
2.211
0.399
0.629
0.980
2.072
COD [ 10 ]
0.051
0.058
0.103
0.118
0.011
0.012
1.116
1.177
0.031
0.034
0.030
0.037
0.018
0.019
0.194
0.208
Ours
0.129
0.176
0.007
0.012
0.018
0.032
0.033
0.584
0.009
0.012
0.005
0.017
0.022
0.026
0.032
0.123
Table 6: Comparison of 3D object canonicalization on the DREDS [ 5 ] dataset.
Canonicalizing 3D object orientation is fundamental to 3D understanding and analysis. Existing approaches often rely on geometric cues, although 3D canonicalization ultimately requires a semantically meaningful orientation. To address this gap, we propose CANIS, a category-agnostic, generation-assisted framework that introduces the semantic orientation prior of a frozen image-to-3D generative model into 3D canonicalization, without canonicalization-specific training or category-specific templates. Specifically, CANIS first renders the input object from candidate viewpoints, selects an informative view, and generates a proxy in a canonical orientation. During generation, a sparse structural latent encoded from the input guides the proxy to preserve the geometry of an object. CANIS then uses the selected image as a semantic bridge between the input and the proxy. Image patches identify semantic regions on the proxy, and depth back-projection locates the corresponding regions on the input. The resulting semantic anchors constrain geometric matching, from which we estimate the rigid transformation that canonicalizes the input. Experiments on synthetic benchmarks validate CANIS and its key components, while qualitative results on partial observations and OmniObject3D suggest its applicability to incomplete and real-world scans. CANIS also improves downstream 3D classification, part segmentation, and dense correspondence under arbitrary rotations. Project page: https://kenkenzaii.github.io/Canis.
Kendong Liu, Yuxin Yao, Junhui Hou
The authors are with the Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China
Generating pose-aligned 3D objects is challenging due to the spatial mismatches and transformation ambiguities inherent in decoupled canonical-then-rotate paradigms. To this end, we introduce Pose-Aware Diffusion (PAD), a novel end-to-end diffusion framework that synthesizes 3D geometry directly within the observation space. By unprojecting monocular depth into a partial point cloud and explicitly injecting it as a 3D geometric anchor, PAD abandons canonical assumptions to enforce rigorous spatial supervision. This native generation intrinsically resolves pose ambiguity, producing high-fidelity pose-aligned assets. Extensive experiments demonstrate that PAD achieves superior geometric alignment and image-to-3D correspondence compared to state-of-the-art methods. Additionally, PAD naturally extends to compositional 3D scene reconstruction via a simple union of independently generated objects, highlighting its robust ability to preserve precise spatial layouts.
Zihan Zhou, Luxi Chen, Jingzhi Zhou +4
Gaoling School of AI, Renmin University of China · VCIP, School of Computer Science, Nankai University · THU-Bosch MLCenter, Tsinghua University +1
Single-image 3D object generation can now produce high-fidelity assets, yet accurately placing them into a coherent scene layout remains an open challenge. A central difficulty lies in how object layout is represented. Holistic methods absorb placement into a scene-level generation process, sacrificing object-level detail. Compositional methods preserve object fidelity by decoupling geometry from layout, but typically parameterize layout as sparse, unbounded pose variables that are difficult to learn and generalize poorly under scarce scene-level supervision. We present Mira-Scene, a compositional 3D scene reconstruction framework that replaces sparse pose regression with dense, bounded correspondence recovery. At its core is the Canonical Coordinate Map (CCM), a pixel-aligned field that maps each visible object pixel to a surface coordinate in the object's bounded canonical space. When paired with a scene-space Point Cloud Map (PCM) from monocular geometry estimation, CCM induces dense canonical-to-scene correspondences from which object transformations are recovered through robust geometric alignment. Because CCM operates in bounded canonical space, it provides a stable prediction target that can be trained from scalable object-level 3D data without requiring scene-level layout annotations. Mira-Scene further introduces a multimodal diffusion transformer that jointly generates object geometry and CCMs, using modality-specific expert streams with shared attention and positional encoding to promote geometry-layout consistency. Experiments on indoor, outdoor, synthetic, and in-the-wild scenes show that Mira-Scene substantially outperforms strong baselines in layout accuracy, achieving relative gains of 39.8% in 3D-IoU and 16.5% in 2D-IoU over SAM3D, using limited open-source training data.