Dynamic Adaptive Attention and Supervised Contrastive Learning: A Novel Hybrid Framework for Text Sentiment Classification

Qingyang Li

cs.CLApr 12, 2026

Dynamic Adaptive Attention and Supervised Contrastive Learning: A Novel Hybrid Framework for Text Sentiment Classification

Authors: Qingyang Li

Organizations: University of California, Los Angeles Los Angeles, USA

Abstract

The exponential growth of user-generated movie reviews on digital platforms has made accurate text sentiment classification a cornerstone task in natural language processing. Traditional models, including standard BERT and recurrent architectures, frequently struggle to capture long-distance semantic dependencies and resolve ambiguous emotional expressions in lengthy review texts. This paper proposes a novel hybrid framework that seamlessly integrates dynamic adaptive multi-head attention with supervised contrastive learning into a BERT-based Transformer encoder. The dynamic adaptive attention module employs a global context pooling vector to dynamically regulate the contribution of each attention head, thereby focusing on critical sentiment-bearing tokens while suppressing noise. Simultaneously, the supervised contrastive learning branch enforces tighter intra-class compactness and larger inter-class separation in the embedding space. Extensive experiments on the IMDB dataset demonstrate that the proposed model achieves competitive performance with an accuracy of 94.67%, outperforming strong baselines by 1.5--2.5 percentage points. The framework is lightweight, efficient, and readily extensible to other text classification tasks.

Figures & tables

Fig. 1: Overall pipeline of the proposed method.

Fig. 2: Overall performance comparison: (a) training loss curves, (b) validation accuracy curves, (c) confusion matrix of proposed model, (d) t-SNE feature visualization.

Model	Overall Metrics (%)				Short Texts ( $<$ 100 words)				Long Texts ( $>$ 300 words)
Model	Acc	Prec	Rec	F1	Acc	Prec	Rec	F1	Acc	Prec	Rec	F1
Proposed (Dynamic Adaptive + SCL)	94.67	94.52	94.81	94.66	95.12	94.98	95.25	95.11	93.89	93.71	94.06	93.88
RoBERTa-base	93.18	93.05	93.29	93.17	94.05	93.92	94.18	94.05	92.34	92.19	92.48	92.33
BERT-base	92.35	92.18	92.51	92.34	92.84	92.71	92.96	92.83	90.76	90.61	90.91	90.76
XLNet-base	92.85	92.71	92.99	92.84	93.45	93.32	93.58	93.44	91.78	91.64	91.92	91.77
BiLSTM+Attention	89.45	89.32	89.61	89.46	90.12	89.98	90.25	90.11	87.91	87.76	88.05	87.90

TABLE I: Overall Performance Comparison with State-of-the-Art Baselines (IMDB Dataset)

Variant	Classification Metrics (%)				Efficiency Metrics			$\Delta$ F1 vs. Full (%)
Variant	Acc	Prec	Rec	F1	Params (M)	FLOPs (G)	Time (h)	$\Delta$ F1 vs. Full (%)
Full Model	94.67±0.42	94.52±0.38	94.81±0.47	94.66±0.41	110.3	18.7	1.87	–
w/o Dynamic Adaptive Attention	92.81±0.51	92.67±0.44	92.94±0.55	92.80±0.48	98.2	16.4	1.52	-1.86
w/o Supervised Contrastive Learning	93.12±0.39	92.98±0.35	93.25±0.42	93.11±0.37	110.3	18.7	1.65	-1.55
w/o Both Modules	91.45±0.67	91.29±0.61	91.62±0.72	91.45±0.64	98.2	16.4	1.31	-3.21
BERT Baseline	92.35±0.55	92.18±0.49	92.51±0.58	92.34±0.52	109.5	17.9	1.45	-2.32

TABLE II: Detailed Ablation Study Results (5-run averages with standard deviation)

Configuration	Overall Performance (%)			Short Texts ( $<$ 100 words)			Long Texts ( $>$ 300 words)
Configuration	Acc	F1	Conv. Epochs	Acc	F1	Drop (%)	Acc	F1	Drop (%)
LR=3e-5, Batch=32, Heads=12 (Optimal)	94.67	94.66	11	95.12	95.11	0.00	93.89	93.88	1.23
LR=5e-5, Batch=32, Heads=12	94.12	94.09	9	94.68	94.65	0.44	93.45	93.41	1.67
LR=3e-5, Batch=64, Heads=8	93.85	93.81	13	94.41	94.38	0.71	92.98	92.95	1.86
LR=1e-5, Batch=16, Heads=16	94.23	94.20	14	94.89	94.85	0.23	93.67	93.64	1.56

TABLE III: Hyperparameter Sensitivity and Text-Length Robustness Analysis

Fig. 3: Ablation study results across Accuracy, Precision, Recall, F1-score, training time, and parameter count.

Fig. 4: Performance breakdown by text length groups (Short: $<$ 100 words, Medium: 100-300 words, Long: $>$ 300 words).

Sentiment Analysis

Sentiment analysis detects and measures opinions, emotions, and evaluative stance in text and multimodal data.

See all 127 papers

From TF-IDF to Transformers: A Comparative and Ensemble Approach to Sentiment Classification
May 21, 2026Dip Biswas Shanto, Mitali Yadav, Prajwal Panth +1Sentiment Analysis Text Classification
KIIT Deemed to be University Bhubaneswar, India · School of Computer Engineering KIIT Deemed to be University Bhubaneswar, India
A Comparative Analysis of Classical Machine Learning and Deep Learning Approaches for Sentiment Classification on IMDb Movie Reviews
May 8, 2026Erma Daniar Safitri, Lia Hana Ichisasmita, Citra Agustin +3Sentiment Analysis Support Vector Machines
Department of Data Science Institut Teknologi Sumatera Lampung Selatan, 35365, Indonesia · Department of Informatics Engineering Institut Teknologi Sumatera Lampung Selatan, 35365, Indonesia
QTrans: A Quantum Transformer for Sentiment Classification
Sep 10, 2026Ren-Xin Zhao, Xinjie Huang, Yahong Liu +4Sentiment Analysis Hybrid Quantum-Classical ML
School of Computer Science, Xiangtan University, Xiangtan 411105, China · School of Electronic Information, Central South University, Changsha 410083, China · Department of Otorhinolaryngology-Head and Neck Surgery, the Third Xiangya Hospital, Central South University, Changsha 410013, China +1

May 21, 2026J/K move · Enter open · S save

From TF-IDF to Transformers: A Comparative and Ensemble Approach to Sentiment Classification

Dip Biswas Shanto, Mitali Yadav, Prajwal Panth, Suresh Chandra Satapathy

KIIT Deemed to be University Bhubaneswar, India · School of Computer Engineering KIIT Deemed to be University Bhubaneswar, India

Sentiment Analysis Text Classification

Open paper AskPDF arXiv

Sentiment analysis, also referred to as opinion mining, primarily tries to extract opinion from any text-based data. In the context of movie reviews and critics, sentimental analysis can be a helpful tool to predict whether a movie review is generally positive or negative. It can be difficult for the ML models to understand the context or metaphysical sentiment accurately, as ML models rely largely on statistical word representations. The objective of this paper is to examine and categorise movie reviews into positive and negative sentiments. Diverse machine learning models are considered in doing so, and Natural Language Processing (NLP) methodologies are employed for data preprocessing and model assessment. The IMDb dataset is used. Specifically, Naive Bayes, Logistic Regression, Support Vector Machines (SVM), LightGBM, LSTM, and transformer-based models such as RoBERTa and DistilBERT were evaluated. After a lot of testing with accuracy, precision, recall, F1-score, and ROC-AUC, RoBERTa performed better than all the other models, with an accuracy of 93.02%. A soft voting ensemble that combined all the models also improved classification performance, showing that model ensembling works well for sentiment analysis.

Abstract

Sentiment Analysis

Explore similar work

From TF-IDF to Transformers: A Comparative and Ensemble Approach to Sentiment Classification

A Comparative Analysis of Classical Machine Learning and Deep Learning Approaches for Sentiment Classification on IMDb Movie Reviews

QTrans: A Quantum Transformer for Sentiment Classification