Organizations: School of Computer Science and Technology, Zhejiang Gongshang University, Hangzhou 310018, China · Shining3D Tech Co., Ltd., Hangzhou 311200, China · School of Artificial Intelligence and Computer Science, Nantong University, Nantong 226019, China · Department of Computer Science, College of Computer Engineering and Sciences, Prince Sattam Bin Abdulaziz University, AlKharj, 16278, Saudi Arabia · Department of Computer Science, Qena University, Qena 83523, Egypt · Department of Computing Sciences, Tampere University, Tampere 33720, Finland · Department of Computer Science, Universidade Federal Fluminense, Niteroi 24210-346, Brazil · Department of Electrical and Electronic Engineering, Faculty of Technology, Firat University, 23000 Elazig, Turkey · Systems Research Institute of the Polish Academy of Sciences, 01-447 Warsaw, Poland · AGH University of Krakow, 30-059 Krakow · SAN University, 90-113 Lodz, Poland
The automatic design of a 3D tooth model plays a crucial role in dental digitization. However, current approaches face challenges in compositional 3D tooth generation because both the layouts and shapes of missing teeth need to be optimized.In addition, collision conflicts are often omitted in 3D Gaussian-based compositional 3D generation, where objects may intersect with each other due to the absence of explicit geometric information on the object surfaces. Motivated by graph generation through diffusion models and collision detection using 3D Gaussians, we propose an approach named DM-CFO for compositional tooth generation, where the layout of missing teeth is progressively restored during the denoising phase under both text and graph constraints. Then, the Gaussian parameters of each layout-guided tooth and the entire jaw are alternately updated using score distillation sampling (SDS). Furthermore, a regularization term based on the distances between the 3D Gaussians of neighboring teeth and the anchor tooth is introduced to penalize tooth intersections. Experimental results on three tooth-design datasets demonstrate that our approach significantly improves the multiview consistency and realism of the generated teeth compared with existing methods. Project page: https://amateurc.github.io/CF-3DTeeth/.
Figures & tables
Fig. 1: Illustration of approaches using the pairwise relations and the proposed approach. (a) GALA3D uses only the text description to optimize the layout (teeth positions), receiving limited results. (b) Our DM-CFO constructs a graph to represent the jaw with multiple missing teeth, then the target graph is incrementally restored during the denoising process through a graph diffusion model.
Fig. 2: An illustration of the proposed DM-CFO. Given 3D Gaussian representing jaw with missing teeth, the graph diffusion generates layout of missing teeth by progressively denoising graph with both text and graph constraints. Then, the Gaussian parameters of each layout-guided tooth and the whole jaw are alternately updated using SDS. A regularization term based on 3D Gaussians of neighboring teeth is explored to penalize the tooth intersection.
Fig. 3: An illustration of the layout editing module. Given the original jaw graph, noise is progressively added in the forward diffusion process, and then graph is progressively updated in the reverse denoising process to obtain the augmented jaw graph. The yellow arrow represents the iteration step.
Fig. 4: An illustration of the compositional optimization module. Instance-level diffusion and scene-level diffusion are jointly optimized.
Fig. 5: An illustration of collision loss. Our approach employs an intravariance Ri rather than a fixed threshold to avoid collision conflict.
Fig. 6: Data distributions in multiple datasets of (a) age, (b) pathology, and (c) types of missing teeth.
Approach
Shining3D
Aoralscan3
DeepBlue
FID ↓
LPIPS ↓
PSNR ↑
FID ↓
LPIPS ↓
PSNR ↑
FID ↓
LPIPS ↓
PSNR ↑
DGE [ 53 ]
223.15
0.70
12.38
234.61
0.75
10.42
228.73
0.72
11.11
VcEdit [ 54 ]
221.44
0.69
12.84
233.67
0.74
11.02
227.05
0.71
11.85
Gaussctrl [ 55 ]
220.90
0.69
13.09
233.22
0.74
11.15
226.10
0.70
12.20
CAT3D [ 34 ]
218.50
0.67
14.45
231.12
0.73
11.57
224.24
0.69
13.08
CompGS [ 4 ]
208.82
0.65
16.23
216.75
0.69
13.10
212.45
0.67
14.83
TABLE I: Effectiveness comparison of the Shining3D, Aoralscan3, and DeepBlue dataset. ’ ↓ ’ means lower is better while ’ ↑ ’ means upper is better.
Fig. 7: Illustration of variance on (a) tooth number and (b) training loss.
Fig. 8: Illustration of the impact of different group of tooth categories. Each column represents a specific group of tooth categories.
Fig. 9: Evaluation of the proposed modules on the Shining3D tooth dataset. Two samples are separated by the black dotted line. The baseline (GALA3D) refines layout without handling collisions. Integrating Graph Diffusion Model (GDM) improves tooth layout in a jaw, while adding Geometry Collision Loss (GCL) effectively resolves collisions, improving geometry quality.
GDM
GCL
FID ↓
LPIPS ↓
PSNR ↑
196.62
0.60
19.24
✓
194.25
0.58
21.85
✓
194.49
0.59
20.71
✓
✓
193.29
0.57
22.55
TABLE II: Effectiveness comparison of improved modules on the Shining3D dataset. GDM represents the graph diffusion model, and GCL represents Gaussian collision loss in compositional optimization.
Approach
Shining3D
Aoralscan3
DeepBlue
CD ↓
F-Score ↑
PD ↓
CD ↓
F-Score ↑
PD ↓
CD ↓
F-Score ↑
PD ↓
TranSDFNet [ 16 ]
0.33
0.80
0.16
0.37
0.77
0.19
0.36
0.78
0.18
Point-to-mesh [ 17 ] *
0.28
0.82
0.14
0.34
0.79
0.18
0.30
0.81
0.16
SSEN [ 18 ] *
0.25
0.83
0.14
0.30
0.81
0.17
0.27
0.82
0.16
VBCD [ 19 ]
0.24
0.83
0.12
0.27
0.81
0.15
0.26
0.82
0.14
DPD [ 20 ]
0.34
0.79
0.17
0.38
0.77
0.21
0.36
0.78
0.19
TABLE III: 3D metric comparison of the Shining3D, Aoralscan3, and DeepBlue dataset. ’ ↓ ’ and ’ ↑ ’ means lower and upper is better, respectively. ’*’ means the approach is implemented by ourselves.
Fig. 10: Qualitative comparisons of rendered images between MVDC, VBCD, ComboVerse, GALA3D, and our approach on the Shining3D tooth design dataset are illustrated. The results of the two samples are divided by the black dotted line. Colorful bounding boxes are used for comparison of multiview consistency.
Fig. 11: Qualitative comparisons of meshes between MVDC, VBCD, ComboVerse, GALA3D, and our approach on the Shining3D tooth design dataset are illustrated. Each row represents a specific sample. GT means the ground truth.
Fig. 12: User Study. In a comprehensive user study encompassing three evaluation criteria that are 3D consistency, collision avoidance, and text fidelity, our approach achieves the highest scores of all the criteria.
Approach
MVDream
GALA3D
ComboVerse
Ours
Time (minute)
2.5
4.2
4.4
4.7
TABLE IV: Efficiency comparison of different approaches in the Shining3D dataset.
Fig. 13: Illustration of the drawbacks in our approach. Multiple generated teeth may adhere to neighboring teeth.
Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Department of Computer Science and Technology, Tsinghua University +2