Credibility Clustering for Experience Rating: An Automated Approach to Risk Group Allocation in Group Life Insurance
Authors: Thom van Rijn, Frank Weber, Michael Merz
Organizations: University of Hamburg Business School, University of Hamburg, Hamburg, Germany · Department of Mathematics and Computer Science, University of Basel, Basel-Stadt, Switzerland
In insurance, experience rating plays a crucial role in mitigating adverse selection by adjusting premiums according to historical claims experience. A key step in this process is the classification of insured collectives into homogeneous risk groups. Within group life insurance, this classification traditionally relies on manual decisions and arbitrary thresholds, which may lead to inefficiencies and inconsistent outcomes. This paper proposes a data-driven methodology for automating risk group allocation based on a combination of the multidimensional Bühlmann-Straub model and k-means clustering -- referred to as credibility clustering. By incorporating credibility weights, the method accounts for the varying statistical significance of experience data and ensuresthat risk classifications are both actuarially sound and practically implementable. Empirical results based on real-world group life insurance data from a major European insurer demonstrate that the proposed framework outperforms the two k-means benchmark variants considered in the empirical application in terms of viability, temporal stability and accounting for statistical reliability.
Figures & tables
Sub-collectives i
Years t
1
2
⋯
I
1
(x11,w11)
(x12,w12)
⋯
(x1I,w1I)
2
(x21,w21)
(x22,w22)
⋯
(x2I,w2I)
⋮
⋮
⋮
⋱
⋮
T
(xT1,wT1)
(xT2,wT2)
⋯
(xTI,wTI)
Table 1: Overview of the available statistical information in the risk classification and experience rating process.
Figure 1: Schematic overview of the credibility clustering framework. Squares denote input and output stages; rectangles represent the methodological components (credibility and clustering). In our application, we use the multidimensional Bühlmann–Straub model and k-means. Gray and blue cells specify relevant variables and granularity levels.
Figure 2: Overview of the k-means method, with and without outlier removal, used as a benchmark for the CCF. In contrast to credibility clustering, the input data is directly aggregated using Equation ( 5 ). Depending on the variant, outliers are either retained or removed from the data. Clustering is then applied to this aggregated data. If outliers were removed beforehand, they are subsequently reassigned to the nearest cluster center. Finally, the credibility model is applied to the resulting clusters (i.e. risk groups).
Years
Contracts
Insured
Economic
Risk
t
Sectors i
Measures m
2014– 2022
36 669– 42 624
327 478– 397 216
84
{Claims Frequency, Claims Severity}
Table 2: Overview of the available data. The numbers of contracts and insured individuals are presented as ranges across the observation period. The lower value corresponds to the minimum annual count, while the upper value corresponds to the maximum annual count.
Statistic
No. Insured
No. Expected Cases
Insured Amount
Expected Claim
Min
1
approx. 1/1000
6 449 CHF
6 CHF
Q25
602
0.61
6.7 Mio. CHF
7 109 CHF
Median
1 743
1.79
24.6 Mio. CHF
26 102 CHF
Q75
4 833
5.32
60.9 Mio. CHF
71 115 CHF
Max
37 648
40.64
803.1 Mio. CHF
935 704 CHF
Table 3: Summary statistics of the data at the year-economic sector (t,i) level.
Figure 3: Comparison of the unadjusted (blue, left) and the credibilized claims experience (red, right). The size of each point reflects the proxy for statistical reliability, defined as the normalized credibility weight. Larger points correspond to higher statistical reliability. The intersection of the vertical and horizontal black lines indicates the overall portfolio experience at (1,1).
Claims Experience
Frequency
Severity
Credibilized
38.8%
49.4%
Unadjusted
-44.4%
-47.0%
Table 4: Empirical Pearson correlation between the credibilized and unadjusted claims experience and the credibility weights of the NOGA-codes.
Figure 4: Figure showing the distribution of small (green), medium (blue) and large (red) economic sectors within the risk groups depending on the type of clustering used. Small, medium and large economic sectors correspond to the bottom, middle and top 33.3% of economic sectors, as measured by their summed normalized volumes.
Figure 5: Boxplots of the risk group factors for risk groups one to five, comparing the CCF (red) with k-means clustering, both excluding (blue) and including (green) outliers. The boxes represent the interquartile range (IQR), with whiskers extending up to 1.5 times the IQR. Dots indicate outliers beyond this range. The dashed horizontal line indicates the global risk experience (value of 1). The y-axis is omitted for confidentiality reasons.
Method
Average Absolute Change
Proportion of Changes ≥ 10%
CCF
7.21%
23.57%
K-Means (without Outliers)
16.60%
60.71%
K-Means (with Outliers)
14.46%
52.62%
Table 5: Average absolute change and proportion of tariff-factor changes exceeding 10% across all tariff-year changes for the three methods.
Risk Group
Method
1
2
3
4
5
Credibility Clustering
0.017
0.042
0.035
0.034
0.076
K-Means (without Outliers)
0.055
0.049
0.075
0.081
0.158
K-Means (with Outliers)
0.045
0.061
0.088
0.168
0.196
Table 6: Standard deviation of risk group factors across tariff years by risk group and method.
Risk Group
Method
1
2
3
4
5
Credibility Clustering
0.040
0.064
0.055
0.056
0.062
K-Means (without Outliers)
0.102
0.071
0.124
0.082
0.106
K-Means (with Outliers)
0.091
0.092
0.113
0.156
0.230
Table 7: Standard deviation of risk group factors across bootstrap samples by risk group and method.
Figure 6: Graph comparing the standard deviation of the tariff factors across bootstrap samples and tariff year changes per economic sector and method. The data are ordered by the standard deviation of the CCF risk group factors. The x-axis labels are omitted due to confidentiality.
Credibility Clustering
Risk Group (Next Period)
Risk Group (Current)
1
2
3
4
5
1
73.6%
26.4%
0%
0%
0%
2
15.8%
50.6%
32.7%
0.9%
0%
3
0%
14.0%
66.9%
19.1%
0%
4
0%
0%
22.2%
60.7%
17.0%
Table 8: Transition matrix for the credibility clustering framework.
K-Means (without Outliers)
Risk Group (Next Period)
Risk Group (Current)
1
2
3
4
5
1
84.7%
10.9%
4.3%
0%
0%
2
5.2%
67.8%
19.4%
6.6%
1%
3
0%
19.8%
57.8%
17.9%
4.5%
4
0%
0.8%
45.2%
29.1%
24.9%
Table 9: Transition matrix for the k-means method without outliers.
K-Means (with Outliers)
Risk Group (Next Period)
Risk Group (Current)
1
2
3
4
5
1
83.5%
13%
1.7%
0%
1.8%
2
4.9%
80.6%
12%
2.4%
0%
3
0%
51.7%
34.7%
13%
0.7%
4
0%
0%
41%
48.8%
10.1%
Table 10: Transition matrix for the k-means method with outliers.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Method
ρ=0
ρ=0.2
ρ=0.35
ρ=0.5
ρ=0.75
ρ=0.99
CCF
7.61%
7.21%
7.20%
7.02%
7.07%
7.02%
Appendix
Table 11: Average absolute change by correlation parameter ρ .
Method
ρ=0
ρ=0.2
ρ=0.35
ρ=0.5
ρ=0.75
ρ=0.99
CCF
35.95%
30.00%
23.57%
26.19%
23.10%
25.00%
Appendix
Table 12: Proportion of changes exceeding 10% by correlation parameter ρ .
Figure 7: Figure showing the distribution of small (green), medium (blue) and large (red) economic sectors within the risk groups depending on the type of clustering used. Small, medium and large economic sectors correspond to the bottom, middle and top 33.3% of economic sectors, as measured by their summed normalized volumes.
Figure 8: Boxplots of the risk group factors for risk groups one to three, comparing the CCF (red, left) with k-means clustering, both excluding (blue, middle) and including (green, right) outliers. The boxes represent the interquartile range (IQR), with whiskers extending up to 1.5 times the IQR. Dots indicate outliers beyond this range. The dashed horizontal line indicates the global risk experience (value of 1). The y-axis is omitted for confidentiality reasons.
Method
Average Absolute Change
Proportion of Changes ≥ 10%
CCF
4.99%
11.90%
K-Means (without Outliers)
10.71%
27.38%
K-Means (with Outliers)
19.73%
56.43%
Appendix
Table 13: Average absolute change and proportion of tariff-factor changes exceeding 10% across all tariff-year changes for the three methods.
Risk Group
Method
1
2
3
Credibility Clustering
0.024
0.015
0.020
K-Means (without Outliers)
0.032
0.019
0.053
K-Means (with Outliers)
0.124
0.066
0.160
Appendix
Table 14: Standard deviation of risk group factors across tariff years by risk group and method.
Risk Group
Method
1
2
3
Credibility Clustering
0.035
0.027
0.033
K-Means (without Outliers)
0.105
0.066
0.068
K-Means (with Outliers)
0.126
0.109
0.242
Appendix
Table 15: Standard deviation of risk group factors across bootstrap samples by risk group and method.
Figure 9: Graph comparing the standard deviation of the tariff factors across bootstrap samples and tariff year changes per economic sector and method. The data are ordered by the standard deviation of the CCF risk group factors. The x-axis labels are omitted due to confidentiality.
Figure 10: Figure showing the distribution of small (green), medium (blue) and large (red) economic sectors within the risk groups depending on the type of clustering used. Small, medium and large economic sectors correspond to the bottom, middle and top 33.3% of economic sectors, as measured by their summed normalized volumes.
Figure 11: Boxplots of the risk group factors for risk groups one to seven, comparing the CCF (red) with k-means clustering, both excluding (blue) and including (green) outliers. The boxes represent the interquartile range (IQR), with whiskers extending up to 1.5 times the IQR. Dots indicate outliers beyond this range. The dashed horizontal line indicates the global risk experience (value of 1). The y-axis is omitted for confidentiality reasons.
Method
Average Absolute Change
Proportion of Changes ≥ 10%
CCF
6.35%
26.90%
K-Means (without Outliers)
12.24%
40.48%
K-Means (with Outliers)
11.43%
35.48%
Appendix
Table 16: Average absolute change and proportion of tariff-factor changes exceeding 10% across all tariff-year changes for the three methods.
Risk Group
Method
1
2
3
4
5
6
7
Credibility Clustering
0.015
0.052
0.063
0.041
0.038
0.036
0.053
K-Means (without Outliers)
0.042
0.067
0.065
0.027
0.083
0.039
0.065
K-Means (with Outliers)
0.083
0.031
0.036
0.105
0.151
0.082
0.145
Appendix
Table 17: Standard deviation of risk group factors across tariff years by risk group and method.
Risk Group
Method
1
2
3
4
5
6
7
Credibility Clustering
0.036
0.053
0.051
0.060
0.070
0.055
0.077
K-Means (without Outliers)
0.112
0.064
0.105
0.123
0.088
0.076
0.106
K-Means (with Outliers)
0.117
0.074
0.112
0.097
0.129
0.168
0.195
Appendix
Table 18: Standard deviation of risk group factors across bootstrap samples by risk group and method.
Figure 12: Graph comparing the standard deviation of the tariff factors across bootstrap samples and tariff year changes per economic sector and method. The data are ordered by the standard deviation of the CCF risk group factors. The x-axis labels are omitted due to confidentiality.
Fairness in insurance pricing remains a long-standing and deeply debated puzzle. On one hand, insurers, driven by profitability considerations, set premiums that differentiate across individual risks to achieve actuarial fairness. On the other hand, insurance serves a critical societal function by pooling risks across a population, motivating cross-subsidization among groups to promote solidarity fairness. The tension between these two competing notions of fairness makes insurance pricing inherently complex, particularly in modern settings where granular data allow for increasingly fine risk differentiation and regulators face growing pressure to protect vulnerable groups. To address this challenge, we propose an α-Fair Individual Solvent Premium (α-FISP) framework for insurance pricing that explicitly captures the trade-off between actuarial and solidarity fairness while guaranteeing solvency, a fundamental requirement in insurance operations. We formulate the pricing problem as a constrained optimization task, where actuarially fair premiums are adjusted subject to budget constraints on cross-subsidization within each risk class. This formulation naturally yields a family of solutions parameterized by α, tracing a continuum between purely actuarial and purely solidarity-based pricing and enabling decision-makers to select an operating point along this fairness spectrum. We derive theoretical guarantees for the proposed framework. Numerical experiments show that α-FISP is computationally tractable and aligns well with the U.S. regulatory regimes featuring heterogeneous state-level fairness requirements.
Tianhe Zhang, Xiguang Liu, Peng Shi
Department of Risk and Insurance, Wisconsin School of Business, University of Wisconsin–Madison · Department of Information Systems and Operations Management, Warrington College of Business, University of Florida
Automated Driving System deployments create a foundational ratemaking challenge: sparse experience, shifting operational design domains, and non-stationary risk across software releases. We propose a hierarchical Bayesian credibility framework pooling across cities, software versions, and territories via a learned ODD-similarity kernel, nesting Buhlmann-Straub as a limiting case. Demonstrated on 648 verified-engaged Waymo crashes across four U.S. metros from the NHTSA Standing General Order database against 116 million matched miles, city-aggregate credibility weights are moderate (0.12-0.46), partial pooling decisively outperforms no pooling, and a power analysis shows the learned kernel's advantage becomes detectable at approximately twelve deployed cities.
Insurance application processes often rely on lengthy and standardized questionnaires that struggle to capture individual differences. Moreover, insurers must blindly trust users' responses, increasing the chances of fraud. The ARQuest framework introduces a new approach to underwriting by using Large Language Models (LLMs) and alternative data sources to create personalized and adaptive questionnaires. Techniques such as social media image analysis, geographic data categorization, and Retrieval Augmented Generation (RAG) are used to extract meaningful user insights and guide targeted follow-up questions. A life insurance system integrated into an industry partner mobile app was tested in two experiments. While traditional questionnaires yielded slightly higher accuracy in risk assessment, adaptive versions powered by GPT models required fewer questions and were preferred by users for their more fluid and engaging experience. ARQuest shows great potential to improve user satisfaction and streamline insurance processes. With further development, this approach may exceed traditional methods regarding risk accuracy and help drive innovation in the insurance industry.
Diogo Silva, João Teixeira, Bruno Lima
Deloitte and Faculty of Engineering, University of Porto, Porto, Portugal · Deloitte, Porto, Portugal · LIACC, Faculty of Engineering, University of Porto, Porto, Portugal