Automatic dental superimposition of 3D intraorals and 2D photographs for human identification
Authors: Antonio D. Villegas-Yeguas, Xavier Abreau-Freire, Guillermo R-García, Andrea Valsecchi, Teresa Pinho, Daniel Pérez-Mongiovi, Oscar Ibáñez, Oscar Cordón
Organizations: Department of Computer Science and Artificial Intelligence of the University of Granada (DECSAI), Spain · Panacea Cooperative Research S. Coop., Ponferrada, Spain · Associate Laboratory i4HB - Institute for Health and Bioeconomy, University Institute of Health Sciences (IUCS), CESPU, 4585-116 Gandra, Portugal · UCIBIO-Research Unit on Applied Molecular Biosciences, Forensic Science Research Laboratory, IUCS-CESPU (1H-TOXRUN), 4585-116 Gandra, Portugal · Institute for Mummy Studies, Eurac Research, Viale Druso 1, 39100, Bolzano, Italy · UNIPRO-Oral Pathology and Rehabilitation Research Unit, IUCS-CESPU, 4585-116 Gandra, Portugal · UMIB-Multidisciplinary Biomedical Research Unit, Abel Salazar Institute of Biomedical Sciences (ICBAS), University of Porto, 4050-313 Porto, Portugal · Department of Computer Science and Information Technologies, University of A Coru, Spain · DECSAI, Spain · Andalusian Research Institute in Data Science and Computational Intelligence
Dental comparison is considered a primary identification method, at the level of fingerprints and DNA profiling. One crucial but time-consuming step of this method is the morphological comparison. One of the main challenges to apply this method is the lack of ante-mortem medical records, specially on scenarios such as migrant death at the border and/or in countries where there is no universal healthcare. The availability of photos on social media where teeth are visible has led many odontologists to consider morphological comparison using them. However, state-of-the-art proposals have significant limitations, including the lack of proper modeling of perspective distortion and the absence of objective approaches that quantify morphological differences. Our proposal involves a 3D (post-mortem scan) - 2D (ante-mortem photos) approach. Using computer vision and optimization techniques, we replicate the ante-mortem image with the 3D model to perform the morphological comparison. Two automatic approaches have been developed: i) using paired landmarks and ii) using a segmentation of the teeth region to estimate camera parameters. Both are capable of obtaining very promising results over 20,164 cross comparisons from 142 samples, obtaining mean ranking values of 1.6 and 1.5, respectively. These results clearly outperform filtering capabilities of automatic dental chart comparison approaches, while providing an automatic, objective and quantitative score of the morphological correspondence, easily to interpret and analyze by visualizing superimposed images.
Figures & tables
Fig. 1: The three landmark sets proposed.
Fig. 2: Examples of occlusion of the upper part (top image) and lower part (bottom images) of teeth in the 2D photos due to the lips.
Fig. 3: Example of the segmentation for one case. In white the region of interest and in red the occlusion mask.
Fig. 4: Example of a 3D mesh segmentation. Above the original 3D mesh with the region of interest colored in red and below the final 3D mesh used for the comparison.
Landmark Set
Dataset
# Cases
AVG
MIN
Q1
Q2
Q3
P95
P99
MAX
Set 1
A
50
1.12
1
1.0
1.0
1.0
2
2
2
Set 2
A
50
1.16
1
1.0
1.0
2.0
2
3
4
Set 3
A
50
1.24
1
1.0
1.0
2.0
2
5.51
6
Set 1
B
50
1.18
1
1.0
1.0
1.0
2.10
4.53
6
Set 2
B
50
1.18
1
1.0
1.0
1.0
2.10
4.53
6
Set 3
B
50
1.58
1
1.0
1.0
1.0
5.55
9.55
12
TABLE I: Ranking statistics of all the experiments using the landmark-based approach.
Fig. 5: Comparison of the superimposition of the PT0230 photograph with the PT0230 3D IOS (above), and the PT0040 3D IOS (below). The first one with a RMSE of 7.49 pixels and the second one with a RMSE of 6.87 .
Dataset
# Cases
AVG
MIN
Q1
Q2
Q3
P95
P99
MAX
A
50
1
1
1
1
1
1
1
1
B
50
1.24
1
1
1
1
2
6.1
7
C
42
1
1
1
1
1
1
1
1
All
142
1.5
1
1
1
1
1
15
26
TABLE II: Ranking results of all the experiments using the region-based approach.
Fig. 6: Comparison of the superimposition of the PT0123 photograph with the PT0123 3D IOS (above), and the PT0040 IOS (below). The first one with a masked DICE of 0.051 and the second one with a masked DICE of 0.031 . In red the silhouette of the 2D teeth, in blue the silhouette of the projected 3D teeth and in green the overlap between the silhouettes.
Fig. 7: Comparison of the superimposition of the PT0046 photograph with the PT0046 3D IOS (above), and the PT0237 3D IOS (below). The first one with a masked DICE of 0.045 and the second one with a masked DICE of 0.034 . In red the silhouette of the 2D teeth, in blue the silhouette of the projected 3D teeth and in green the overlap between the silhouettes.
Fig. 8: Example of the PDF for the results of the regions approach using the 142 available subjects.
Landmark Set 1
Landmark Set 2
Landmark Set 3
Regions
Dataset A (50 cases)
0.122
0.219
0.231
0.166
Dataset B (50 cases)
0.266
0.266
0.315
0.113
Dataset C (42 cases)
0.307
0.248
0.310
0.236
All (142 cases)
0.290
0.286
0.281
0.2316
TABLE III: Cllr results of all the experiments. The lower, the better.
Fig. 9: CMC curves for the complete dataset. Note that the Y-axis starts at 70% of cases for a clearer visualization.
The comparison of dental records is a standardized technique in forensic dentistry used to speed up the identification of individuals in multiple-comparison scenarios. Specifically, the odontogram comparison is a procedure to compute criteria that will be used to perform a ranking. State-of-the-art automatic methods either make use of simple techniques, without utilizing the full potential of the information obtained from a comparison, or their internal behavior is not known due to the lack of peer-reviewed publications. This work aims to design aggregation mechanisms to automatically compare pairs of dental records that can be understood and validated by experts, improving the current methods. To do so, we introduce different aggregation approaches using the state-of-the-art codification, based on seven different criteria. In particular, we study the performance of i) data-driven lexicographical order-based aggregations, ii) well-known fuzzy logic aggregation methods and iii) machine learning techniques as aggregation mechanisms. To validate our proposals, 215 forensic cases from two different populations have been used. The results obtained show how the use of white-box machine learning techniques as aggregation models (average ranking from 2.02 to 2.21) are able to improve the state-of-the-art (average ranking of 3.91) without compromising the explainability and interpretability of the method.
Antonio D. Villegas-Yeguas, Guillermo R-García, Tzipi Kahana +4
Department of Computer Science and Artificial Intelligence, University of Granada, Granada, Spain · Panacea Cooperative Research S. Coop, Ponferrada, Spain · Institute of Criminology, The Faculty of Law, Hebrew University of Jerusalem, Jerusalem, Israel +4
Oral 3D modelling is one of the most essential stages in dentistry, and many different approaches, such as impression taking and intraoral scanning, are commonly used for this phase, each with notable limitations. Impression taking, which involves placing alginate or silicone material in a tray and inserting it into the patient's oral cavity to form a negative mold, suffers from significant patient discomfort, material deformation errors, and difficulties in storage and transportation. Intraoral scanners, which directly scan oral structures in real time using structured light or laser technology, produce state-of-the-art results but are associated with substantially high equipment costs. To address these limitations, this paper proposes a software-based approach that reconstructs a 3D oral model using only ten 2D intraoral images captured from different angles, requiring no dedicated hardware devices. The proposed method reduces cost, eliminates the need for physical scanning equipment, minimises patient discomfort, and enables automated 3D reconstruction. The model is trained on the publicly available Teeth3DS dataset, comprising 950 upper jaw samples, and employs MobileNetV2 as the image encoder combined with Multi-head Attention for multi-view feature fusion. The proposed model achieves an accuracy of 77.49%, measured by nearest-neighbor matching with a distance threshold of 0.035. However, predicted vertices tend to concentrate in high-density regions of the ground truth, resulting in uneven point distribution across the reconstructed model.
Jihun Cho, Soo-Yeon Jeong, Eun-Jeong Bae +1
Dept. of Computer Engineering Pai Chai University Daejeon, Korea · Division of Software Engineering Pai Chai University Daejeon, Korea · Dept. of Dental Technology Bucheon University Bucheon, Korea
CBCT (Cone Beam Computed Tomography) scans provide detailed three-dimensional images, widely used in dentistry for diagnostic and treatment planning tasks. While invaluable, analyzing and documenting these scans is labor-intensive, prompting efforts to automate key steps like the classification and segmentation of anatomical structures to identify tooth types and associated pathologies. In this article, we propose an approach to automation that leverages persistent homology, a framework from topological data analysis that studies the shape of data by identifying features like connected components, holes, and voids across multiple scales. Persistent homology, together with a support vector machine, allows us to classify teeth in a CBCT scan and to perform diagnostics. Our method advances the state of the art, reaching average accuracy scores of 97.67% for tooth-labeling and 96.77% for diagnostic tasks, outperforming a CNN trained on the same data with accuracy of 70.27% and 86.67%, respectively.
Leon Dahlmeier, Sara Kališnik, Albert Mehl +1
Department of Mathematics, ETH Zürich, Zürich, Switzerland · Department of Mathematics, PSU, Pennsylvania, United States · Department of Computerized Dentistry, Clinic of Conservative and Preventive Dentistry, University of Zürich, Zürich, Switzerland +1