GCUL: Ambiguity Identification in Text Emotion Classification via Cluster-Guided Learning

Zhongqi Fan; Tianyou Zhang; Fei Chen

stat.MLSep 24, 2026

GCUL: Ambiguity Identification in Text Emotion Classification via Cluster-Guided Learning

Authors: Zhongqi Fan, Tianyou Zhang, Fei Chen

Organizations: Beijing Normal-Hong Kong Baptist University

Abstract

Selective classification enables a model to abstain from predictions on uncertain instances, but existing approaches typically reject them through confidence scores, predefined coverage constraints or instance-level distance measures. These approaches may overlook the collective geometric structure of difficult samples in learned representation spaces. We propose Guided Clustering-based Uncertain Learning (GCUL), a geometric-guided selective classification framework that identifies misclassified and ambiguous instances as a potential confusion attractor in the representation space. GCUL uses a three-phase procedure to initialize, cluster, and explicitly relabel this uncertain region, allowing the rejection boundary to emerge from the underlying representation geometry rather than from a prescribed rejection rate. We further derive a selectivity score and a geometric sufficient condition that characterizes when rejection can provide positive operational utility, enabling pre-deployment feasibility assessment. GCUL improves DistilBERT accuracy from 89.37 percent to 94.98 percent with less than 9 percent rejection. Beyond accuracy, our selectivity score correctly pre-detects the only dataset (GoEmotion) where all baselines fail, and controlled simulations yield 6.1 percent Type-I and 0 percent Type-II errors, validating the sufficient condition's conservatism. These results suggest that collective representation geometry provides a useful alternative perspective for selective prediction.

Figures & tables

Model	Accuracy (%)	F1-score	Rej. Rate
Logistic Regression	84.02	0.8433	–
Naive Bayes [ 17 ]	72.80	0.7256	–
SVM [ 13 ]	84.11	0.8421	–
Random Forest	84.75	0.8518	–
TextCNN [ 14 ]	86.35	0.8673	–
BiLSTM + Attention [ 20 , 9 ]	87.83	0.8831	–

Table 1 : Overall performance comparison of all models.

Violation Type	Count / Total	Rate (%)
Type-1	25 / 410	6.1%
Type-2	0 / 410	0%
Overall	25 / 410	6.1%

Table 2: Quantitative reliability analysis of theoretical feasibility predictions across 410 individual runs (41 noise levels

\times

10 random seeds).

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Component / Phase	Configuration
Backbone / Input	DistilBERT 768-d embeddings
Sequence Architecture	2-layer LSTM
Optimization	Adam, Batch size $=64$
Phase Schedule	Phase 1: 1 epoch; Phase 3: 4 epochs
Dimensionality Reduction	768 $\rightarrow$ 10 dims via PCA / LDA

Appendix

Table 3: Key hyperparameter configuration for the GCUL framework.

Method	Parameter	Configuration
MSP	Rejection cost $\lambda$	$\{0.1,0.2,\ldots,1.0\}$
	Threshold range	$\theta\in\{0.00,0.01,\ldots,1.00\}$
	Base classifier	Logistic Regression, max_iter $=500$
SelectiveNet	Target coverage	$\{0.60,0.64,\ldots,0.96\}$
	Utility penalty $\lambda$	$0.15$
	Training epochs	$30$

Appendix

Table 4 : Experimental parameter configurations for selective classification baselines and GCUL.

Parameter	Value	Description
$K$	5	Number of classes
$N_{\mathrm{clean}}$	2,000	Samples per clean class
$N_{\mathrm{conf}}$	2,000	Samples in confusion region
$d$	10.0	Feature dimensionality
$S$	10.0	Center scale
$r$	1.5	Cluster radius

Appendix

Table 5 : Dataset generation parameters

Parameter	Value	Description
$\lambda_{\mathrm{target}}$	0.15	Target threshold
Phase 1		Logistic Regression
solver	lbfgs	Optimization algorithm
max_iter	500	Maximum iterations
multi_class	multinomial	Multi-class strategy
Phase 2

Appendix

Table 6 : GCUL algorithm parameters

Parameter	Value	Description
$\sigma_{\mathrm{noise}}$	$0,0.4,0.8,\dots,16$	Feature noise
$p_{\mathrm{flip}}$	$\sigma_{\mathrm{noise}}\times 0.025$	Label flip probability
$\mathcal{S}$	$\{10,11,\dots,19\}$	Evaluation seeds

Appendix

Table 7 : Noise sweep parameters

Metric	Description
Base Acc	Base classifier accuracy on test set
Selective Acc	Accuracy after rejection
Acc Gain	Selective Acc $-$ Base Acc
Uncertain Rate	Proportion of samples rejected
PRE $\lambda^{*}$	Critical threshold predicted by theory
EMP $\lambda^{*}$	Critical threshold derived empirically

Appendix

Table 8: Evaluation metrics

$\lambda$	Base Acc (%)	Sel Acc (%)	Rej Rate (%)
0.1	88.71	99.04	16.74
0.2	88.71	99.04	16.74
0.3	88.71	99.04	16.74
0.4	88.71	99.04	16.74
0.5	88.71	98.12	14.56
0.6	88.71	96.94	12.40

Appendix

Table 9: MSP results on Kaggle emotion dataset

Target $c$	Base Acc (%)	Sel Acc (%)	Rej Rate (%)
0.60	89.22	99.92	39.95
0.64	89.18	99.96	35.83
0.68	89.33	99.92	31.95
0.72	89.13	99.86	27.58
0.76	89.05	99.89	23.58
0.80	89.17	99.68	19.83

Appendix

Table 10: SelectiveNet results on Kaggle emotion dataset

Retained	Rej Rate (%)	Base Acc (%)	Sel Acc (%)
Top 100%	0	88.71	88.71
Top 96%	4	88.71	91.12
Top 92%	8	88.71	93.32
Top 88%	12	88.71	95.65
Top 84%	16	88.71	97.73
Top 80%	20	88.71	99.48

Appendix

Table 11: Mahalanobis results on Kaggle emotion dataset

PCA $d$	Base Acc	Sel Acc	Rej Rate	$\lambda^{*}_{\mathrm{PRE}}$	$\lambda^{*}_{\mathrm{EMP}}$	Guaranteed
2	88.71	92.11	5.80	0.00	0.00	No
3	88.71	91.95	5.64	0.03	0.05	No
4	88.71	93.51	7.47	0.10	0.12	No
5	88.71	93.04	6.89	0.15	0.19	No
8	88.71	93.93	8.07	0.32	0.38	Yes
10	88.71	95.47	10.22	0.35	0.46	Yes

Appendix

Table 12 : GCUL results on Kaggle emotion dataset (

\lambda_{\mathrm{target}}=0.15

)

$\lambda$	Base Acc (%)	Sel Acc (%)	Rej Rate (%)
0.1	37.99	100	99.83
0.2	37.99	100	99.83
0.3	37.99	100	99.83
0.4	37.99	100	99.83
0.5	37.99	100	99.83
0.6	37.99	100	99.83

Appendix

Table 13: MSP results on GoEmotions dataset

Target $c$	Base Acc (%)	Sel Acc (%)	Rej Rate (%)
0.60	37.75	45.43	39.94
0.64	37.47	44.30	35.94
0.68	37.46	43.68	32.12
0.72	37.57	42.79	28.06
0.76	37.94	42.45	23.82
0.80	37.27	41.00	19.85

Appendix

Table 14: SelectiveNet results on GoEmotions dataset

Retained	Rej Rate (%)	Base Acc (%)	Sel Acc (%)
Top 100%	0	37.99	37.99
Top 96%	4	37.99	38.14
Top 92%	8	37.99	38.37
Top 88%	12	37.99	38.49
Top 84%	16	37.99	38.69
Top 80%	20	37.99	38.80

Appendix

Table 15: Mahalanobis results on GoEmotions dataset

PCA $d$	Base Acc	Sel Acc	Rej Rate	Guaranteed
2	37.99	38.15	3.24	No
3	37.99	38.19	2.48	No
4	37.99	38.09	3.06	No
5	37.99	38.01	2.48	No
8	37.99	38.35	4.06	No
10	37.99	38.23	2.81	No

Appendix

Table 16 : GCUL results on GoEmotions dataset (

\lambda_{\mathrm{target}}=0.15

)

$\lambda$	Base Acc (%)	Sel Acc (%)	Rej Rate (%)
0.1	92.55	98.81	11.70
0.2	92.55	98.81	11.70
0.3	92.55	98.81	11.70
0.4	92.55	97.06	8.25
0.5	92.55	94.09	3.55
0.6	92.55	93.64	2.45

Appendix

Table 17: MSP results on Dair-AI emotion dataset

Target $c$	Base Acc (%)	Sel Acc (%)	Rej Rate (%)
0.60	93.10	98.82	40.75
0.64	93.00	98.43	36.50
0.68	93.30	98.67	32.40
0.72	93.45	98.04	28.45
0.76	93.10	96.74	24.75
0.80	93.20	96.74	20.25

Appendix

Table 18: SelectiveNet results on Dair-AI emotion dataset

Retained	Rej Rate (%)	Base Acc (%)	Sel Acc (%)
Top 100%	0	92.55	92.55
Top 96%	4	92.55	93.91
Top 92%	8	92.55	93.97
Top 88%	12	92.55	94.20
Top 84%	16	92.55	94.35
Top 80%	20	92.55	94.31

Appendix

Table 19: Mahalanobis results on Dair-AI emotion dataset

PCA $d$	Base Acc	Sel Acc	Rej Rate	$\lambda^{*}_{\mathrm{PRE}}$	$\lambda^{*}_{\mathrm{EMP}}$	Guaranteed
2	92.6	93.65	2.35	0.00	0.00	No
3	92.6	95.92	5.70	0.00	0.25	No
4	92.6	95.79	4.95	0.22	0.27	Yes
5	92.6	95.73	5.10	0.33	0.31	Yes
8	92.6	95.79	4.95	0.10	0.24	No
10	92.6	95.69	4.95	0.11	0.23	No

Appendix

Table 20 : GCUL results on Dair-AI emotion dataset (

\lambda_{\mathrm{target}}=0.15

)

Noise	Base Acc	Sel Acc	Rej Rate	$\lambda^{*}_{\mathrm{PRE}}$	$\lambda^{*}_{\mathrm{EMP}}$
$\sigma_{f}$	(%)	(%)	(%)
0.0	86.50	100	16.54	0.96	0.82
0.4	86.54	100	16.54	0.96	0.81
0.8	86.88	100	16.54	0.95	0.79
1.2	86.92	100	16.54	0.94	0.79
1.6	86.92	100	16.54	0.92	0.79

Appendix

Table 21: GCUL performance under varying noise levels on synthetic data (seed 16)

Explore similar work

Improving Selective Classification with Pairwise Queries for Binary Classification
May 28, 2026Harsh Vardhan, Sunav Choudhary, Natwar Modani +1Confidence Estimation
CSE, UCSD, 9500 Gilman Drive, La Jolla, 92092, CA, USA. · Adobe Research, 345 Park Ave, San Jose, 95110, CA, USA. · Adobe Research, Marathahalli, Bengaluru, 560087, Karnataka, India. +1
ReliableNet: A Chance-Constrained Approach to Trustworthy Classification in Deep Learning
Aug 10, 2026Ange-Clément Akazan, Ineza Remy Mugenga, Abebe Geletu +2Empirical Risk Minimization Deep Learning
School of Agriculture and Science, University of KwaZulu-Natal, Pietermaritzburg, KwaZulu-Natal, South Africa · 2AIMS Research and Innovation Centre, African Institute for Mathematical Sciences, Kigali, Rwanda · 3African Institute for Mathematical Sciences, Kigali, Rwanda
GEM-FI: Gated Evidential Mixtures with Fisher Modulation
May 5, 2026Marco Mustafa Mohammed, Fatemeh Daneshfar, Pietro LiòEvidential Uncertainty Dynamic Epistemic Logic
Department of Computer Engineering, Faculty of Engineering, University of Kurdistan, Sanandaj, Iran

May 28, 2026, cs.LGJ/K move · Enter open · S save

Improving Selective Classification with Pairwise Queries for Binary Classification

Harsh Vardhan, Sunav Choudhary, Natwar Modani, Arya Mazumdar

CSE, UCSD, 9500 Gilman Drive, La Jolla, 92092, CA, USA. · Adobe Research, 345 Park Ave, San Jose, 95110, CA, USA. · Adobe Research, Marathahalli, Bengaluru, 560087, Karnataka, India.+1

Confidence Estimation

Open paper AskPDF arXiv

In selective classification, a model predicts the labels of data samples where it is confident, and abstains from predicting labels for samples on which it is not confident. The rejected samples are often labeled by an expert, which is expensive. The budget for the expert is best utilized when the model has low error on non-rejected samples. However, the estimate of a model's confidence might be inconsistent with the model's predictions, which can lead to high error on non-rejected points. Such situations can readily occur in in-context binary classification by LLMs. To remedy this, we propose making additional pairwise queries to the same model. These pairwise queries can detect high-error samples and be incorporated into selective classification techniques to reduce the error on non-rejected samples. Theoretically, we establish the conditions under which a simple algorithm using pairwise queries outperforms an inconsistent confidence estimate. We support this insight through extensive experiments for

1

synthetic and

4

in-context learning-based real binary classification datasets. In all these cases, we show that our algorithms, using pairwise queries, obtain a better accuracy-cost tradeoff than using only the raw confidence estimates, for instance, the LLM's next-token logits.

Abstract

Explore similar work

Improving Selective Classification with Pairwise Queries for Binary Classification

ReliableNet: A Chance-Constrained Approach to Trustworthy Classification in Deep Learning

GEM-FI: Gated Evidential Mixtures with Fisher Modulation