Distributional sentiment modeling and anomaly detection for consumer complaint assessment

Peiheng Gao; Chen Yang; Shimin Zhang

cs.CLSep 14, 2026

Distributional sentiment modeling and anomaly detection for consumer complaint assessment

Authors: Peiheng Gao, Chen Yang, Shimin Zhang

Organizations: School of Mathematical and Statistical Sciences, Western University, London, ON N6A 3K7, Canada · Department of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, New York 10029, USA · Institute for Health Care Delivery Science, Icahn School of Medicine at Mount Sinai, New York, New York 10029, USA · Asian Institute of Digital Finance, National University of Singapore, Singapore, 119077

Abstract

Sentiment analysis is a common tool for converting unstructured text into quantitative signals in finance and risk management. Yet most applications reduce the output to a discrete polarity label or a single predictive feature, overlooking the distributional structure of sentiment intensity in consumer complaint narratives. In this paper we treat negative sentiment in consumer complaints as a bounded continuous variable and study its full distribution rather than a single label. We score each narrative with a transformer classifier, model the scores with Beta distributions, and compare the fitted distributions of meritorious and non-meritorious complaints through the Kullback Leibler divergence and the squared Hellinger distance. The fitted distributions are then linked with dollar amounts and company response outcomes to construct anomaly diagnostics that flag complaints whose textual severity is inconsistent with the recorded relief. We find that the two groups have strongly overlapping distributions, so negative sentiment intensity is not a sharp classifier of outcomes on its own; combined with monetary and categorical attributes, it isolates unusually severe complaints for operational risk monitoring. Treating sentiment analysis as continuous distributional measurement, this study links sentiment extraction, bounded response modeling, and anomaly detection in a unified framework for consumer complaint assessment.

Figures & tables

Step	Records
After date restriction	$2{,}280{,}111$
Credit reporting product category	$1{,}662{,}544$
At least one detected dollar amount	$222{,}261$
Average dollar amount in $1\leq A_{i}<10{,}000$	$181{,}316$
Meritorious	$85{,}501$
Non-meritorious	$95{,}815$

Table 2.1 : Data cleaning steps and record counts.

Figure 2.1 : Distribution of average dollar amounts by complaint outcome.

Require

1. Cleaned complaint narrative corpus

\mathcal{C}_{\mathrm{clean}}

2. Complaint outcome labels

Y_{i}\in\{0,1\}

, where

Y_{i}=1

denotes meritorious.

3. Extracted average dollar amounts

A_{i}

satisfying

1\leq A_{i}<10{,}000

Compute distributional sentiment and anomaly diagnostics

1. Apply a transformer based sentiment classifier to each narrative

c_{i}

2. Extract the negative sentiment probability

s_{i}=p_{i,\mathrm{neg}}

Algorithm 3.1 : Algorithmic summary of the distributional sentiment modeling.

Panel of Figure 4 — Figure 4.1 : Negative sentiment score distributions and fitted Beta distributions by complaint outcome.

Sample	$N$	$\hat{\alpha}$	$\hat{\beta}$	Fitted mean	Fitted var.	Sample mean	Sample var.
Full sample	$181{,}316$	$1.473$	$2.296$	$0.391$	$0.050$	$0.382$	$0.052$
Meritorious	$85{,}501$	$1.593$	$2.648$	$0.376$	$0.045$	$0.367$	$0.046$
Non-meritorious	$95{,}815$	$1.394$	$2.058$	$0.404$	$0.054$	$0.395$	$0.057$

Table 4.1 : Beta fits for negative sentiment scores in the cleaned sample and by outcome group.

Figure 4.2 : QQ diagnostics for the fitted Beta models of negative sentiment scores. Each panel plots empirical against fitted Beta quantiles for the full sample and the two outcome groups; the red line is the

45^{\circ}

reference.

Metric	Value
$D_{\mathrm{KL}}(F_{1}\,\\|\,F_{0})$	$0.0168$
$D_{\mathrm{KL}}(F_{0}\,\\|\,F_{1})$	$0.0202$
Symmetric KL average	$0.0185$
Squared Hellinger distance ( $H^{2}$ )	$0.0046$

Table 4.2 : Distributional separation between the fitted Beta sentiment distributions.

F_{1}

and

F_{0}

denote the meritorious and non-meritorious fits.

Figure 4.3 : Overlap between the fitted Beta densities for meritorious and non-meritorious complaints.

Observed outcome	Sentiment region	Diagnostic interpretation
Meritorious	$s_{i}^{}<0.0666$ or $s_{i}^{}>0.5392$	More consistent with the non-meritorious density
Non-meritorious	$0.0666<s_{i}^{*}<0.5392$	More consistent with the meritorious density

Table 4.3 : Anomaly regions implied by the fitted Beta distributions

Diagnostic	Threshold	Meritorious	Non-meritorious	Total	Total rate
Outcome inconsistency	$\eta=0$	$20{,}572$	$66{,}754$	$87{,}326$	$48.16\%$
Outcome inconsistency	$\eta=0.01$	$19{,}546$	$65{,}422$	$84{,}968$	$46.86\%$
Outcome inconsistency	$\eta=0.05$	$15{,}518$	$58{,}179$	$73{,}697$	$40.65\%$
Outcome inconsistency	$\eta=0.10$	$12{,}081$	$39{,}802$	$51{,}883$	$28.61\%$
Tail anomaly	$\tau=0.01$	$1{,}257$	$795$	$2{,}052$	$1.13\%$
Tail anomaly	$\tau=0.05$	$4{,}412$	$4{,}561$	$8{,}973$	$4.95\%$

Table 4.4 : Anomaly diagnostics on the cleaned sample: baseline and threshold sensitivity.

Sentiment Analysis

Sentiment analysis detects and measures opinions, emotions, and evaluative stance in text and multimodal data.

See all 127 papers

From Complaint Narratives to Monetary Relief: A Hybrid Machine Learning Framework for CFPB Consumer Complaints
Jun 21, 2026Zhuoer Wang, Sizhen Zhu, Xiongyu ChenText Classification
University of Illinois Urbana-Champaign · North Carolina State University · Carnegie Mellon University
What sentiment analysis can't see: Measuring whether customers were helped, and what went wrong, across 70,000 support conversations
Jun 18, 2026Jason PotteigerLLM-Assisted Annotation Sentiment Analysis
Dimension Labs
Extracting Consumer Insight from Text: A Large Language Model Approach to Emotion and Evaluation Measurement
Feb 17, 2026Stephan Ludwig, Peter J. Danaher, Xiaohao Yang +4Sentiment Analysis LLM Information Extraction
Monash University, Australia · University of New South Wales, Australia · Flinders University, Australia +3

Jun 21, 2026J/K move · Enter open · S save

From Complaint Narratives to Monetary Relief: A Hybrid Machine Learning Framework for CFPB Consumer Complaints

Zhuoer Wang, Sizhen Zhu, Xiongyu Chen

University of Illinois Urbana-Champaign · North Carolina State University · Carnegie Mellon University

Text Classification

Open paper AskPDF arXiv

Consumer financial complaints provide a valuable source of information for identifying service failures, dispute frictions, and operational deficiencies in consumer-facing financial institutions. This paper proposes a hybrid machine learning framework for predicting monetary relief outcomes using Consumer Financial Protection Bureau complaint data. We formulate the task as an imbalanced binary classification problem, where complaints closed with monetary relief are treated as compensable outcomes. The proposed framework integrates multiple sources of predictive information, including complaint narrative text, LDA-based topic representations, interpretable text-engineered features, and structured categorical attributes such as company and state. An XGBoost classifier is trained using a temporal train-test split, with earlier complaints used for model development and more recent complaints reserved for out-of-sample evaluation. Compared with a TF-IDF baseline, the proposed framework substantially improves predictive performance, increasing AUC-ROC from 0.69 to 0.78 and improving PR-AUC under class imbalance. Feature importance analysis shows that textual signals, latent complaint topics, and company identity all contribute meaningful predictive information. In particular, company-level effects reveal systematic variation in complaint resolution patterns across financial institutions. These findings suggest that consumer complaint narratives can serve as alternative data for monitoring consumer harm, identifying firm-level operational weaknesses, and supporting early-stage risk surveillance in consumer finance.

Abstract

Sentiment Analysis

Explore similar work

From Complaint Narratives to Monetary Relief: A Hybrid Machine Learning Framework for CFPB Consumer Complaints

What sentiment analysis can't see: Measuring whether customers were helped, and what went wrong, across 70,000 support conversations

Extracting Consumer Insight from Text: A Large Language Model Approach to Emotion and Evaluation Measurement