Distributional sentiment modeling and anomaly detection for consumer complaint assessment

Peiheng Gao; Chen Yang; Shimin Zhang

cs.CLSep 14, 2026

Distributional sentiment modeling and anomaly detection for consumer complaint assessment

Authors: Peiheng Gao, Chen Yang, Shimin Zhang

Organizations: School of Mathematical and Statistical Sciences, Western University, London, ON N6A 3K7, Canada · Department of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, New York 10029, USA · Institute for Health Care Delivery Science, Icahn School of Medicine at Mount Sinai, New York, New York 10029, USA · Asian Institute of Digital Finance, National University of Singapore, Singapore, 119077

Abstract

Sentiment analysis is a common tool for converting unstructured text into quantitative signals in finance and risk management. Yet most applications reduce the output to a discrete polarity label or a single predictive feature, overlooking the distributional structure of sentiment intensity in consumer complaint narratives. In this paper we treat negative sentiment in consumer complaints as a bounded continuous variable and study its full distribution rather than a single label. We score each narrative with a transformer classifier, model the scores with Beta distributions, and compare the fitted distributions of meritorious and non-meritorious complaints through the Kullback Leibler divergence and the squared Hellinger distance. The fitted distributions are then linked with dollar amounts and company response outcomes to construct anomaly diagnostics that flag complaints whose textual severity is inconsistent with the recorded relief. We find that the two groups have strongly overlapping distributions, so negative sentiment intensity is not a sharp classifier of outcomes on its own; combined with monetary and categorical attributes, it isolates unusually severe complaints for operational risk monitoring. Treating sentiment analysis as continuous distributional measurement, this study links sentiment extraction, bounded response modeling, and anomaly detection in a unified framework for consumer complaint assessment.

Figures & tables

Step	Records
After date restriction	$2{,}280{,}111$
Credit reporting product category	$1{,}662{,}544$
At least one detected dollar amount	$222{,}261$
Average dollar amount in $1\leq A_{i}<10{,}000$	$181{,}316$
Meritorious	$85{,}501$
Non-meritorious	$95{,}815$

Table 2.1 : Data cleaning steps and record counts.

Figure 2.1 : Distribution of average dollar amounts by complaint outcome.

Require

1. Cleaned complaint narrative corpus

\mathcal{C}_{\mathrm{clean}}

2. Complaint outcome labels

Y_{i}\in\{0,1\}

, where

Y_{i}=1

denotes meritorious.

3. Extracted average dollar amounts

A_{i}

satisfying

1\leq A_{i}<10{,}000

Compute distributional sentiment and anomaly diagnostics

1. Apply a transformer based sentiment classifier to each narrative

c_{i}

2. Extract the negative sentiment probability

s_{i}=p_{i,\mathrm{neg}}

Algorithm 3.1 : Algorithmic summary of the distributional sentiment modeling.

Panel of Figure 4 — Figure 4.1 : Negative sentiment score distributions and fitted Beta distributions by complaint outcome.

Sample	$N$	$\hat{\alpha}$	$\hat{\beta}$	Fitted mean	Fitted var.	Sample mean	Sample var.
Full sample	$181{,}316$	$1.473$	$2.296$	$0.391$	$0.050$	$0.382$	$0.052$
Meritorious	$85{,}501$	$1.593$	$2.648$	$0.376$	$0.045$	$0.367$	$0.046$
Non-meritorious	$95{,}815$	$1.394$	$2.058$	$0.404$	$0.054$	$0.395$	$0.057$

Table 4.1 : Beta fits for negative sentiment scores in the cleaned sample and by outcome group.

Figure 4.2 : QQ diagnostics for the fitted Beta models of negative sentiment scores. Each panel plots empirical against fitted Beta quantiles for the full sample and the two outcome groups; the red line is the

45^{\circ}

reference.

Metric	Value
$D_{\mathrm{KL}}(F_{1}\,\\|\,F_{0})$	$0.0168$
$D_{\mathrm{KL}}(F_{0}\,\\|\,F_{1})$	$0.0202$
Symmetric KL average	$0.0185$
Squared Hellinger distance ( $H^{2}$ )	$0.0046$

Table 4.2 : Distributional separation between the fitted Beta sentiment distributions.

F_{1}

and

F_{0}

denote the meritorious and non-meritorious fits.

Figure 4.3 : Overlap between the fitted Beta densities for meritorious and non-meritorious complaints.

Observed outcome	Sentiment region	Diagnostic interpretation
Meritorious	$s_{i}^{}<0.0666$ or $s_{i}^{}>0.5392$	More consistent with the non-meritorious density
Non-meritorious	$0.0666<s_{i}^{*}<0.5392$	More consistent with the meritorious density

Table 4.3 : Anomaly regions implied by the fitted Beta distributions

Diagnostic	Threshold	Meritorious	Non-meritorious	Total	Total rate
Outcome inconsistency	$\eta=0$	$20{,}572$	$66{,}754$	$87{,}326$	$48.16\%$
Outcome inconsistency	$\eta=0.01$	$19{,}546$	$65{,}422$	$84{,}968$	$46.86\%$
Outcome inconsistency	$\eta=0.05$	$15{,}518$	$58{,}179$	$73{,}697$	$40.65\%$
Outcome inconsistency	$\eta=0.10$	$12{,}081$	$39{,}802$	$51{,}883$	$28.61\%$
Tail anomaly	$\tau=0.01$	$1{,}257$	$795$	$2{,}052$	$1.13\%$
Tail anomaly	$\tau=0.05$	$4{,}412$	$4{,}561$	$8{,}973$	$4.95\%$

Table 4.4 : Anomaly diagnostics on the cleaned sample: baseline and threshold sensitivity.

Sentiment Analysis

Sentiment analysis detects and measures opinions, emotions, and evaluative stance in text and multimodal data.

See all 127 papers

Jun 21, 2026

From Complaint Narratives to Monetary Relief: A Hybrid Machine Learning Framework for CFPB Consumer Complaints

Consumer financial complaints provide a valuable source of information for identifying service failures, dispute frictions, and operational deficiencies in consumer-facing financial institutions. This paper proposes a hybrid machine learning framework for predicting monetary relief outcomes using Consumer Financial Protection Bureau complaint data. We formulate the task as an imbalanced binary classification problem, where complaints closed with monetary relief are treated as compensable outcomes. The proposed framework integrates multiple sources of predictive information, including complaint narrative text, LDA-based topic representations, interpretable text-engineered features, and structured categorical attributes such as company and state. An XGBoost classifier is trained using a temporal train-test split, with earlier complaints used for model development and more recent complaints reserved for out-of-sample evaluation. Compared with a TF-IDF baseline, the proposed framework substantially improves predictive performance, increasing AUC-ROC from 0.69 to 0.78 and improving PR-AUC under class imbalance. Feature importance analysis shows that textual signals, latent complaint topics, and company identity all contribute meaningful predictive information. In particular, company-level effects reveal systematic variation in complaint resolution patterns across financial institutions. These findings suggest that consumer complaint narratives can serve as alternative data for monitoring consumer harm, identifying firm-level operational weaknesses, and supporting early-stage risk surveillance in consumer finance.

Zhuoer Wang, Sizhen Zhu, Xiongyu Chen

University of Illinois Urbana-Champaign · North Carolina State University · Carnegie Mellon University

Text Classification

Ask PDF

Jun 18, 2026

What sentiment analysis can't see: Measuring whether customers were helped, and what went wrong, across 70,000 support conversations

Most companies read their customer support data at scale using sentiment analysis, which measures how customers sound rather than whether they were satisfied with the result. We tested a richer alternative on 70,450 support conversations from a leading online fundraising platform: alongside tone, we used GPT-5.4 to estimate each customer's satisfaction and to flag whether they reported a concrete problem, then validated all three readings against the 1-to-5 ratings customers left on the conversations they rated. The satisfaction estimate tracked those ratings far better than sentiment did, correlating at 0.47 against 0.36 and flagging unhappy customers with far fewer false alarms. The structured read also sees what sentiment cannot: tone and satisfaction disagree in 44% of conversations, a single "Neutral" label hides everything from quietly satisfied customers to ones who quietly gave up, and the largest group of all is "tolerated friction," customers who are satisfied but still reporting a fixable problem, a standing issue that no sentiment-based dashboard can surface. The broader finding is that LLM-based annotation can capture far more than the tonality of a customer's language, offering strong potential for new business metrics grounded instead in the customer's state (whether they were satisfied) and the cause of their problem extracted directly from the raw textual data of interactions and feedback.

Jason Potteiger

Dimension Labs

LLM-Assisted Annotation Sentiment Analysis

Ask PDF

Feb 17, 2026

Extracting Consumer Insight from Text: A Large Language Model Approach to Emotion and Evaluation Measurement

Accurately measuring consumer emotions and evaluations from unstructured text remains a core challenge for marketing research and practice. This study introduces the Linguistic eXtractor (LX), a fine-tuned, large language model trained on consumer-authored text that also has been labeled with consumers' self-reported ratings of 16 consumption-related emotions and four evaluation constructs: trust, commitment, recommendation, and sentiment. LX consistently outperforms leading models, including GPT-4 Turbo, RoBERTa, and DeepSeek, achieving 81% macro-F1 accuracy on open-ended survey responses and greater than 95% accuracy on third-party-annotated Amazon and Yelp reviews. An application of LX to online retail data, using seemingly unrelated regression, affirms that review-expressed emotions predict product ratings, which in turn predict purchase behavior. Most emotional effects are mediated by product ratings, though some emotions, such as discontent and peacefulness, influence purchase directly, indicating that emotional tone provides meaningful signals beyond star ratings. To support its use, a no-code, cost-free, LX web application is available, enabling scalable analyses of consumer-authored text. In establishing a new methodological foundation for consumer perception measurement, this research demonstrates new methods for leveraging large language models to advance marketing research and practice, thereby achieving validated detection of marketing constructs from consumer data.

Stephan Ludwig, Peter J. Danaher, Xiaohao Yang +4

Monash University, Australia · University of New South Wales, Australia · Flinders University, Australia +3

Sentiment Analysis LLM Information Extraction

Ask PDF

Abstract

Sentiment Analysis

Explore similar work

From Complaint Narratives to Monetary Relief: A Hybrid Machine Learning Framework for CFPB Consumer Complaints

What sentiment analysis can't see: Measuring whether customers were helped, and what went wrong, across 70,000 support conversations

Extracting Consumer Insight from Text: A Large Language Model Approach to Emotion and Evaluation Measurement