On the use of Aggregation Operators to improve Human Identification using Dental Records
Authors: Antonio D. Villegas-Yeguas, Guillermo R-García, Tzipi Kahana, Jorge Pinares Toledo, Esi Sharon, Oscar Ibañez, Oscar Cordón
Organizations: Department of Computer Science and Artificial Intelligence, University of Granada, Granada, Spain · Panacea Cooperative Research S. Coop, Ponferrada, Spain · Institute of Criminology, The Faculty of Law, Hebrew University of Jerusalem, Jerusalem, Israel · Forensic Odontology Unit, Division of Identification and Forensic Science, Israel Police, Jerusalem, Israel · Faculty of Dentistry, University of Chile, Santiago de Chile, Chile · Department of Prosthodontics, Faculty of Dental Medicine, Hebrew University of Jerusalem, Jerusalem, Israel · Department of Computer Science and Information Technologies, University of Coruña, A Coruña, Spain
The comparison of dental records is a standardized technique in forensic dentistry used to speed up the identification of individuals in multiple-comparison scenarios. Specifically, the odontogram comparison is a procedure to compute criteria that will be used to perform a ranking. State-of-the-art automatic methods either make use of simple techniques, without utilizing the full potential of the information obtained from a comparison, or their internal behavior is not known due to the lack of peer-reviewed publications. This work aims to design aggregation mechanisms to automatically compare pairs of dental records that can be understood and validated by experts, improving the current methods. To do so, we introduce different aggregation approaches using the state-of-the-art codification, based on seven different criteria. In particular, we study the performance of i) data-driven lexicographical order-based aggregations, ii) well-known fuzzy logic aggregation methods and iii) machine learning techniques as aggregation mechanisms. To validate our proposals, 215 forensic cases from two different populations have been used. The results obtained show how the use of white-box machine learning techniques as aggregation models (average ranking from 2.02 to 2.21) are able to improve the state-of-the-art (average ranking of 3.91) without compromising the explainability and interpretability of the method.
Figures & tables
Comparison result by AM code
PM Code
Match
Mismatch
Possible
V
V
F, S, X, I
P
F
F
S, X, I
V, P
S
S
X, I
F, V, P
X
X
V, F, S, I, P
I
I
V, F, S, X, P
Table 1: Classification matrix for each possible PM-AM comparison in SCS.
Comparison result by PM code
AM Code
Match
Mismatch
Possible
V
V
F, S, X, I, P
F
F
V
S, X, I, P
S
S
V, F
X, I, P
X
X
V, F, S, P
I
I
I
V, F, S, P
X
Table 2: Classification matrix for each possible AM-PM comparison in SCS.
Israel
Chile
Total
Training
83
56
139
Validation
20
14
34
Test
25
17
42
Total
128
87
215
Table 3: Data split by number of cases. Training and validation refers to the numbers of cases in one fold.
Average
Q1
Q2
Q3
P95
P99
Max
AA
3.91
1
1
1
30
34
36
LO
2.14
1
1
1
6
16
34
Table 4: Comparison of ranking performance between Adams and Aschheim’s proposal (AA) and our LO aggregation mechanism.
Figure 1: CMC plot comparing the results of our LO aggregation mechanism and Adams and Aschheim’s proposal in the test dataset.
Average
Q1
Q2
Q3
P95
P99
Max
AA
3.91
1
1
1
30
34
36
OWA-Maximum
5.9
1
4
7
18
25
35
OWA-Minimum
4.99
1
1
3
27
37
39
OWA-Average
3.27
1
1
1
15
26
36
OWA-Linear
2.86
1
1
1
11
22
36
Table 5: Comparison of ranking performance between Adams and Aschheim’s proposal (AA) and our approach using the OWA operator with different weights.
Figure 2: CMC plot comparing the results of our proposal of the OWA operator with Adams and Aschheim’s proposal in the test dataset.
Average
Q1
Q2
Q3
P95
P99
Max
AA
3.91
1
1
1
30
34
36
OWHM-Average
6.51
1
2
6
28
36
39
OWHM-Linear
6.02
1
2
5
28
36
38
Table 6: Comparison of ranking performance between Adams and Aschheim’s proposal (AA) and our approach using the OWHM aggregator with different weights.
Figure 3: CMC plot comparing the results of our proposal of the OWHM aggregator with Adams and Aschheim’s proposal in the test dataset. Notice that differently to previous CMC plots, the Y axis range is 0-100%.
Average
Q1
Q2
Q3
P95
P99
Max
AA
3.91
1
1
1
30
34
36
Choquet integral: μ : Cardinality
3.26
1
1
1
15
26
36
Choquet integral: μ : λ
3.14
1
1
1
16
33
35
Table 7: Comparison of ranking performance between Adams and Aschheim’s proposal (AA) and the Choquet integral.
Figure 4: CMC plot comparing the results of our proposal of the Choquet integral aggregator with Adams and Aschheim’s proposal in the test dataset.
Average
Q1
Q2
Q3
P95
P99
Max
AA
3.91
1
1
1
30
34
36
Sugeno integral: μ : Cardinality
6.06
1
1
6
27
34
40
Sugeno integral: μ : λ
5.28
1
3
5
18
25
35
Table 8: Comparison of ranking performance between Adams and Aschheim’s proposal (AA) and the Sugeno integral.
Figure 5: CMC plot comparing the results of our proposal of the Sugeno integral aggregator with Adams and Aschheim’s proposal in the test dataset.
Fold
Training
Validation
Average
Maximum
Average
Maximum
1
0,0061
0,3478
0,0041
0,0857
2
0,0058
0,3043
0,0055
0,1429
3
0,0041
0,1449
0,0139
0,3714
4
0,0065
0,3381
0,0009
0,0294
5
0,0074
0,3165
0,0013
0,0294
Table 9: Normalized results of the average and maximum ranking obtained using a genetic algorithm to learn the weights of a linear regression. In bold the best validation results.
Average
Q1
Q2
Q3
P95
P99
Max
AA
3.91
1
1
1
30
34
36
LR
2.21
1
1
1
6
23
24
SR
2.02
1
1
1
4
23
30
MLP
1.94
1
1
1
4
18
28
Table 10: Comparison of ranking performance between Adams and Aschheim’s proposal (AA) and our machine learning-based aggregation mechanisms.
Figure 6: CMC plot comparing the results of our proposal of machine learning-based aggregation mechanisms as score generator with Adams and Aschheim’s proposal in the test dataset.
Fold
Training
Validation
Average
Maximum
Average
Maximum
1
0,0059
0,3188
0,0069
0,1429
2
0,0047
0,2754
0,0101
0,3429
3
0,0039
0,1594
0,0184
0,9714
4
0,0048
0,2662
0,0009
0,0294
5
0,0049
0,2374
0,0013
0,0294
Table 11: Normalized results of the average and maximum ranking obtained using a genetic algorithm to learn a symbolic regression-based aggregation mechanism. In bold the best validation results.
Fold
Training
Validation
Average
Maximum
Average
Maximum
1
0,0030
0,1159
0,0061
0,1429
2
0,0030
0,1014
0,0105
0,3714
3
0,0022
0,0725
0,0155
0,6286
4
0,0034
0,0791
0,0004
0,0294
5
0,0039
0,0935
0,0013
0,0294
Table 12: Normalized results of the average and maximum ranking obtained using a genetic algorithm to train a multilayer perceptron-based aggregation mechanism. In bold the best validation results.
Average
Q1
Q2
Q3
P95
P99
Max
AA
3.91
1
1
1
30
34
36
LO
2.14
1
1
1
6
16
34
OWA-Linear
2.86
1
1
1
11
22
36
OWHM-Linear
6.02
1
2
5
28
36
38
Choquet integral: μ : λ
3.14
1
1
1
16
33
35
Sugeno integral: μ : λ
5.28
1
3
5
18
25
35
Table 13: Comparison of ranking performance between all the proposals.
Figure 7: CMC plot comparing the results of the best fuzzy aggregators and Adams and Aschheim’s proposal in the test dataset.
Figure 8: CMC plot comparing the results of the machine learning aggregators, our LO aggregation mechanism proposal and the Adams and Aschheim’s proposal in the test dataset.
Department of Computer Science and Artificial Intelligence of the University of Granada (DECSAI), Spain · Panacea Cooperative Research S. Coop., Ponferrada, Spain · Associate Laboratory i4HB - Institute for Health and Bioeconomy, University Institute of Health Sciences (IUCS), CESPU, 4585-116 Gandra, Portugal +7
AIM for Health Lab, Faculty of Information Technology, Monash University, Melbourne, Australia · Faculty of Information Technology, Monash University, Melbourne, Australia · Curae Health, Melbourne, Australia +4
Faculty of Dentistry, the University of Hongkong, Hong Kong SAR, China · Department of Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA, USA · Shenzhen University, China +6