Automatic dental superimposition of 3D intraorals and 2D photographs for human identification
Authors: Antonio D. Villegas-Yeguas, Xavier Abreau-Freire, Guillermo R-García, Andrea Valsecchi, Teresa Pinho, Daniel Pérez-Mongiovi, Oscar Ibáñez, Oscar Cordón
Organizations: Department of Computer Science and Artificial Intelligence of the University of Granada (DECSAI), Spain · Panacea Cooperative Research S. Coop., Ponferrada, Spain · Associate Laboratory i4HB - Institute for Health and Bioeconomy, University Institute of Health Sciences (IUCS), CESPU, 4585-116 Gandra, Portugal · UCIBIO-Research Unit on Applied Molecular Biosciences, Forensic Science Research Laboratory, IUCS-CESPU (1H-TOXRUN), 4585-116 Gandra, Portugal · Institute for Mummy Studies, Eurac Research, Viale Druso 1, 39100, Bolzano, Italy · UNIPRO-Oral Pathology and Rehabilitation Research Unit, IUCS-CESPU, 4585-116 Gandra, Portugal · UMIB-Multidisciplinary Biomedical Research Unit, Abel Salazar Institute of Biomedical Sciences (ICBAS), University of Porto, 4050-313 Porto, Portugal · Department of Computer Science and Information Technologies, University of A Coru, Spain · DECSAI, Spain · Andalusian Research Institute in Data Science and Computational Intelligence
Dental comparison is considered a primary identification method, at the level of fingerprints and DNA profiling. One crucial but time-consuming step of this method is the morphological comparison. One of the main challenges to apply this method is the lack of ante-mortem medical records, specially on scenarios such as migrant death at the border and/or in countries where there is no universal healthcare. The availability of photos on social media where teeth are visible has led many odontologists to consider morphological comparison using them. However, state-of-the-art proposals have significant limitations, including the lack of proper modeling of perspective distortion and the absence of objective approaches that quantify morphological differences. Our proposal involves a 3D (post-mortem scan) - 2D (ante-mortem photos) approach. Using computer vision and optimization techniques, we replicate the ante-mortem image with the 3D model to perform the morphological comparison. Two automatic approaches have been developed: i) using paired landmarks and ii) using a segmentation of the teeth region to estimate camera parameters. Both are capable of obtaining very promising results over 20,164 cross comparisons from 142 samples, obtaining mean ranking values of 1.6 and 1.5, respectively. These results clearly outperform filtering capabilities of automatic dental chart comparison approaches, while providing an automatic, objective and quantitative score of the morphological correspondence, easily to interpret and analyze by visualizing superimposed images.
Figures & tables
Fig. 1: The three landmark sets proposed.
Fig. 2: Examples of occlusion of the upper part (top image) and lower part (bottom images) of teeth in the 2D photos due to the lips.
Fig. 3: Example of the segmentation for one case. In white the region of interest and in red the occlusion mask.
Fig. 4: Example of a 3D mesh segmentation. Above the original 3D mesh with the region of interest colored in red and below the final 3D mesh used for the comparison.
Landmark Set
Dataset
# Cases
AVG
MIN
Q1
Q2
Q3
P95
P99
MAX
Set 1
A
50
1.12
1
1.0
1.0
1.0
2
2
2
Set 2
A
50
1.16
1
1.0
1.0
2.0
2
3
4
Set 3
A
50
1.24
1
1.0
1.0
2.0
2
5.51
6
Set 1
B
50
1.18
1
1.0
1.0
1.0
2.10
4.53
6
Set 2
B
50
1.18
1
1.0
1.0
1.0
2.10
4.53
6
Set 3
B
50
1.58
1
1.0
1.0
1.0
5.55
9.55
12
TABLE I: Ranking statistics of all the experiments using the landmark-based approach.
Fig. 5: Comparison of the superimposition of the PT0230 photograph with the PT0230 3D IOS (above), and the PT0040 3D IOS (below). The first one with a RMSE of 7.49 pixels and the second one with a RMSE of 6.87 .
Dataset
# Cases
AVG
MIN
Q1
Q2
Q3
P95
P99
MAX
A
50
1
1
1
1
1
1
1
1
B
50
1.24
1
1
1
1
2
6.1
7
C
42
1
1
1
1
1
1
1
1
All
142
1.5
1
1
1
1
1
15
26
TABLE II: Ranking results of all the experiments using the region-based approach.
Fig. 6: Comparison of the superimposition of the PT0123 photograph with the PT0123 3D IOS (above), and the PT0040 IOS (below). The first one with a masked DICE of 0.051 and the second one with a masked DICE of 0.031 . In red the silhouette of the 2D teeth, in blue the silhouette of the projected 3D teeth and in green the overlap between the silhouettes.
Fig. 7: Comparison of the superimposition of the PT0046 photograph with the PT0046 3D IOS (above), and the PT0237 3D IOS (below). The first one with a masked DICE of 0.045 and the second one with a masked DICE of 0.034 . In red the silhouette of the 2D teeth, in blue the silhouette of the projected 3D teeth and in green the overlap between the silhouettes.
Fig. 8: Example of the PDF for the results of the regions approach using the 142 available subjects.
Landmark Set 1
Landmark Set 2
Landmark Set 3
Regions
Dataset A (50 cases)
0.122
0.219
0.231
0.166
Dataset B (50 cases)
0.266
0.266
0.315
0.113
Dataset C (42 cases)
0.307
0.248
0.310
0.236
All (142 cases)
0.290
0.286
0.281
0.2316
TABLE III: Cllr results of all the experiments. The lower, the better.
Fig. 9: CMC curves for the complete dataset. Note that the Y-axis starts at 70% of cases for a clearer visualization.
Department of Computer Science and Artificial Intelligence, University of Granada, Granada, Spain · Panacea Cooperative Research S. Coop, Ponferrada, Spain · Institute of Criminology, The Faculty of Law, Hebrew University of Jerusalem, Jerusalem, Israel +4
Dept. of Computer Engineering Pai Chai University Daejeon, Korea · Division of Software Engineering Pai Chai University Daejeon, Korea · Dept. of Dental Technology Bucheon University Bucheon, Korea
Department of Mathematics, ETH Zürich, Zürich, Switzerland · Department of Mathematics, PSU, Pennsylvania, United States · Department of Computerized Dentistry, Clinic of Conservative and Preventive Dentistry, University of Zürich, Zürich, Switzerland +1