When working with real-world insurance data, practitioners often encounter challenges during the data preparation stage that can undermine the statistical validity and reliability of downstream modeling. This study illustrates that conventional data preparation procedures such as random train-test partitioning, often yield unreliable and unstable results when confronted with highly imbalanced insurance loss data. To mitigate these limitations, we propose a novel data preparation framework leveraging two recent statistical advancements: support points for representative data splitting to ensure distributional consistency across partitions, and the Chatterjee correlation coefficient for initial, non-parametric feature screening to capture feature relevance and dependence structure. We further integrate these theoretical advances into a unified, efficient framework that also incorporates missing-data handling, and embed this framework within our custom InsurAutoML pipeline. The performance of the proposed approach is evaluated using both simulated datasets and datasets often cited in the academic literature. Our findings definitively demonstrate that incorporating statistically rigorous data preparation methods not only significantly enhances model robustness and interpretability but also substantially reduces computational resource requirements across diverse insurance loss modeling tasks. This work provides a crucial methodological upgrade for achieving reliable results in high stakes insurance applications.
Figures & tables
Figure 1: Comparison between Random Splitting and SPlit partitioning.
Figure 2: An illustration of the Informed Data Preparation Pipeline
Figure 3: An illustration of InformedAutoML workflow
Dataset
Missingness
IDPPcomp
IDPPmiss
ε^mech,MAEcomp
∣Smechcomp∣
∣Smechcomp∩SReal∣
ε^mech,MAEmiss
∣Smechmiss∣
∣Smechmiss∩SReal∣
Data 1
MCAR
0.0081
4
3
0.0031
3
2
MAR
0.0122
3
3
0.0057
1
1
MNAR
0.0153
2
2
0.0114
4
3
COMBO
0.0104
2
2
0.0010
1
1
Data 2
MCAR
0.0145
1
1
0.0084
1
1
Table 1: Summary of coefficient estimation on the training set
G
T/s
Model
runtime/s
Train MSE
Test MSE
8
900
InsurAutoML
952.30
993,285.74
1,220,835.99
InformedAutoML
294.69
1,098,973.61
1,191,695.06
16
1,800
InsurAutoML
1,290.22
1,101,871.13
1,192,769.13
InformedAutoML
606.11
1,092,658.21
1,191,822.05
32
3,600
InsurAutoML
2,181.53
1,099,855.88
1,191,872.59
InformedAutoML
672.83
1,098,643.36
1,191,755.57
Table 2: Performance on ausprivauto claim amount
Figure 4: Train/Test MSE and runtime on ausprivauto claims dataset
G
T/s
Model
runtime/s
Subset
Train RMSE
Test RMSE
8
900
InsurAutoML
1,072.68
Complete
0.1585
0.1545
Missing
0.1895
0.1937
InformedAutoML
69.10
Complete
0.0776
0.0932
Missing
0.1194
0.1501
16
1,800
InsurAutoML
1,626.81
Complete
0.1073
0.1194
Missing
0.1489
0.1594
Table 3: Performance on college Pell Grant rate
Figure 5: Train/Test RMSE and runtime on college Pell Grant dataset
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Notation
Description
D
a dataset
X
fully observed feature matrix
Xi
i th feature vector in Rp
Xij
the j -th covariate of Xi (element of X )
Xi1
the single covariate of Xi when p=1
yi
realized response value for the i th observation
Appendix
Table 4: Notations
Table 5: Configurations and empirical summaries for the three simulated datasets
Dataset
MCAR
MAR
MNAR
COMBO
Data 1
11.30%
11.69%
11.83%
11.69%
Data 2
15.55%
15.62%
15.76%
15.79%
Data 3
74.50%
75.18%
74.11%
75.71%
Appendix
Table 6: Row-wise missingness percentage
Dataset
Feature
MCAR
MAR
MNAR
COMBO
Data 1
RealCon1
3.19%
0.00%
0.00%
1.70%
RealCon2
0.00%
0.00%
2.65%
1.47%
RealCon3
4.69%
7.39%
2.59%
6.23%
FakeCon1
0.00%
5.68%
6.03%
0.00%
FakeCon2
3.79%
0.00%
3.54%
4.39%
y
0.00%
0.00%
0.00%
0.00%
Appendix
Table 7: Column-wise missingness rates percentage
Mechanism
Feature
True β
Fitted β^ on full data
IDPPcompleteβ^
IDPPmissβ^
Data 1
MCAR
RealCon1
0.2865
0.2923
0.2891
–
RealCon2
0.6165
0.6301
0.6321
0.6144
RealCon3
0.9465
0.9327
0.9331
0.9506
FakeCon1
0.0000
-0.0116
–
–
FakeCon2
0.0000
-0.0132
0.0010
–
Appendix
Table 8: Coefficient-level recovery results
Figure 6: Comparison of CCC values on tie breaking when only X has ties
Figure 7: Comparison of CCC values on tie breaking when only y has ties
Figure 8: Comparison of CCC values on tie breaking when both X and y have ties
With the rapid rise of InsurTech, traditional insurance companies are increasingly exploring alternative data sources and advanced technologies to sustain their competitive edge. This paper provides both a conceptual overview and practical case studies of natural language processing (NLP) and its emerging applications within insurance operations, focusing on transforming raw, unstructured text into structured data suitable for actuarial analysis and decision-making. Leveraging real-world alternative data provided by an InsurTech industry partner that enriches traditional insurance data sources, we apply various NLP techniques to demonstrate feature de-biasing, feature compression, and industry classification in the commercial insurance context. These enriched, text-derived insights not only add to and refine traditional rating factors for commercial insurance pricing but also offer novel perspectives for assessing underlying risk by introducing novel industry classification techniques. Through these demonstrations, we show that NLP is not merely a supplementary tool but a foundational element of modern, data-driven insurance analytics.
Panyi Dong, Zhiyu Quan
Actuarial and Risk Management Sciences, University of Illinois at Urbana-Champaign, 1409 W. Green Street (MC-382), Urbana, IL, 61801, USA.
Accurate model evaluation in machine learning depends critically on how datasets are split into training and testing subsets. Standard random splitting assumes that both partitions share the same underlying distribution, an assumption often violated in datasets with class imbalance, natural clustering, or spatial autocorrelation. This paper investigates the role of statistical similarity in train-test splitting and its consequences for AutoML model evaluation. Five established strategies are compared across fifteen UCI benchmark datasets: random splitting, stratified sampling, Kennard-Stone, Duplex, and SPXY. Similarity is assessed using chi-square, Kolmogorov-Smirnov, and Maximum Mean Discrepancy (MMD) tests. Geometry-based methods consistently produce near-zero MMD scores, introducing instability into downstream performance estimates. The proposed Optimised-Distribution method treats similarity as an explicit optimisation objective and achieves the highest mean MMD similarity, 89.0%, across all strategies evaluated.
Yearn Tan Yin Tze, Charles Grellois
School of Computer Science, University of Sheffield, U.K.
Insurance application processes often rely on lengthy and standardized questionnaires that struggle to capture individual differences. Moreover, insurers must blindly trust users' responses, increasing the chances of fraud. The ARQuest framework introduces a new approach to underwriting by using Large Language Models (LLMs) and alternative data sources to create personalized and adaptive questionnaires. Techniques such as social media image analysis, geographic data categorization, and Retrieval Augmented Generation (RAG) are used to extract meaningful user insights and guide targeted follow-up questions. A life insurance system integrated into an industry partner mobile app was tested in two experiments. While traditional questionnaires yielded slightly higher accuracy in risk assessment, adaptive versions powered by GPT models required fewer questions and were preferred by users for their more fluid and engaging experience. ARQuest shows great potential to improve user satisfaction and streamline insurance processes. With further development, this approach may exceed traditional methods regarding risk accuracy and help drive innovation in the insurance industry.
Diogo Silva, João Teixeira, Bruno Lima
Deloitte and Faculty of Engineering, University of Porto, Porto, Portugal · Deloitte, Porto, Portugal · LIACC, Faculty of Engineering, University of Porto, Porto, Portugal