Cross-Lingual Sentiment Misalignment: Auditing Multilingual Language Models for Inversion Risk, Dialectal Representation, and Affective Stability

Nusrat Jahan Lia; Shubhashis Roy Dipta

cs.CLFeb 19, 2026

Cross-Lingual Sentiment Misalignment: Auditing Multilingual Language Models for Inversion Risk, Dialectal Representation, and Affective Stability

Authors: Nusrat Jahan Lia, Shubhashis Roy Dipta

Organizations: Institute of Information Technology, University of Dhaka, Dhaka, Bangladesh · University of Maryland, Baltimore County, Baltimore, Maryland, USA

Abstract

Recent advances in multilingual representation learning aim to bridge the performance gap between high- and low-resource languages, yet their ability to preserve affective meaning across languages remains underexplored, particularly for underrepresented languages like Bengali. This research addresses cross-lingual sentiment misalignment between Bengali and English by introducing a controlled benchmarking framework evaluating four multilingual transformer models on parallel Bengali-English sentence pairs, stratified by dialect, to assess their representational stability. We demonstrate that a compressed model architecture exhibits a 28.7% "Sentiment Inversion Rate," fundamentally misinterpreting positive semantics as negative (or vice versa). Consequently, we identify a cross-lingual sentiment skew that we call "Asymmetric Empathy," where models systematically dampen or artificially amplify the affective weight of Bengali text relative to its exact English counterpart. Finally, we expose a key vulnerability regarding dialectal representation: a "Modern Bias" in the regional model, which exhibits a 57% increase in alignment error when processing the formal Bengali register compared to modern colloquial text. As foundational encoders continue to serve as safety classifiers and reward models for LLM pipelines, cross-lingual reliability becomes a critical concern. We therefore advocate for the integration of "Affective Stability" metrics into future cross-lingual benchmarks to detect and penalize polarity inversions, particularly in low-resource settings.

Figures & tables

Figure 1: The plot maps the predicted English sentiment vector ( $X$ -axis) against the Bengali vector ( $Y$ -axis) for a test sentence. The green corridor represents ideal cross-lingual sentiment alignment ( $Y\approx X$ ). While the large-scale XLM-T model successfully preserves polarity, the compressed (mDistilBERT) and regionally specialized (IndicBERT) architectures are projecting a negative English statement into a positive Bengali latent space (Red Zone).

Metric	Tabularis	XLM-T	IndicBERT	mDistilBERT
Mean Div.	0.200	0.276	0.375	0.417
Std Dev.	0.214	0.298	0.607	0.429
Sadhu Div.	0.239	0.286	0.459	0.456
Cholito Div.	0.161	0.266	0.292	0.379
Dialect Gap	0.078	0.020	0.167	0.077
Sadhu Err. Inc. (%)	48.4	7.6	57.1	20.5

Table 1: Metric scores per model

Metric	Formulation	Interpretation
Mean Alignment Error (Divergence)	$\mu_{D}(M)=\frac{1}{n}\sum_{i=1}^{n}D_{i}$	Lower $\mu_{D}$ indicates better average cross-lingual consistency.
Std. Deviation (Divergence)	$\sigma_{D}(M)=\sqrt{\frac{1}{n-1}\sum_{i=1}^{n}(D_{i}-\mu_{D})^{2}}$	Quantifies the variability in alignment quality across the dataset.
Robustness Index	$\mathcal{R}(M)=\frac{1}{n}\sum_{i=1}^{n}\mathbbm{1}[D_{i}<0.1]\times 100\%$	% of pairs with negligible divergence; measures the “safe operating zone.”
Inversion Rate	$\mathcal{I}_{\text{rate}}(M)=\frac{1}{n}\sum_{i=1}^{n}I_{i}\times 100\%$	% of sentence pairs exhibiting sentiment polarity flips.
Mean Directional Bias	$\mu_{B}(M)=\frac{1}{n}\sum_{i=1}^{n}B_{i}$	$>0$ : English favored; $<0$ : Bengali favored; $\approx 0$ : No systematic language skew.
Formal Penalty (Dialect Gap)	$\Delta_{\text{dialect}}(M)=\mu_{D}(M,\text{Sadhu})-\mu_{D}(M,\text{Cholito})$	$>0$ implies a “Modern Bias” where the model struggles with formal registers.

Table 2: Summary of population-level alignment and dialectal metrics.

Figure 2: Sentiment Inversion Rate Across Models

Figure 3: Distribution of Alignment Error Density

Model	Mean Div. ( $\mu_{D}$ )	Inv. Rate ( $\mathcal{I}_{rate}$ )	Affective Stability ( $\mathcal{AS}$ )
XLM-T	0.276	0.036	0.831
Tabularis	0.200	0.086	0.823
IndicBERT	0.375	0.200	0.650
mDistilBERT	0.417	0.287	0.564

Table 3: Affective Stability (

\mathcal{AS}

) of evaluated models.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Model Name	Repository (HuggingFace)
XLM-T	cardiffnlp/XLM-Toberta-sentiment Barbieri et al. (2022)
IndicBERT	ai4bharat/IndicBERTv2-sentiment
Tabularis	tabularisai/multilingual-sentiment
mDistilBERT	lxyuan/distilbert-multilingual

Appendix

Table 4: Model Repository Mapping

Figure 4: Directional Bias in Sentiment Scores (English $-$ Bengali)

Figure 5: Illustrative case study validating Asymmetric Empathy in cross-lingual sentiment alignment, demonstrating severe instance-level directional bias.

Dialect Bias in Language Models

See all 14 papers

Benchmarking Bengali Dialectal Bias: A Multi-Stage Framework Integrating RAG-Based Translation and Human-Augmented RLAIF
Mar 22, 2026K. M. Jubair Sami, Dipto Sumit, Ariyan Hossain +1Social Bias in Language Models Dialect Bias in Language Models
Department of Computer Science and Engineering BRAC University, Dhaka, Bangladesh
Sycophancy as a Multilingual Alignment Failure: How Safety Degrades Across Languages, Topics, and Models
Jun 7, 2026Arya Shah, Himanshu Beniwal, Mayank Singh +1Multilingual Language Models Multilingual Language Model Evaluation
IIT Gandhinagar · Asian Institute of Technology
Language Unalignability: Why Some Concepts Resist Cross-Cultural Benchmark Evaluation
Oct 6, 2026Shu-Kai Hsieh, Da-Chen LianCultural Knowledge in Language Models LLM Evaluation
National Taiwan University

Mar 22, 2026J/K move · Enter open · S save

Benchmarking Bengali Dialectal Bias: A Multi-Stage Framework Integrating RAG-Based Translation and Human-Augmented RLAIF

K. M. Jubair Sami, Dipto Sumit, Ariyan Hossain, Farig Sadeque

Department of Computer Science and Engineering BRAC University, Dhaka, Bangladesh

Social Bias in Language Models Dialect Bias in Language Models Language Model Bias Evaluation Low-Resource Language Processing

Open paper AskPDF arXiv

Large language models (LLMs) frequently exhibit performance biases against regional dialects of low-resource languages. However, frameworks to quantify these disparities remain scarce. We propose a two-phase framework to evaluate dialectal bias, operationalized as comprehension degradation relative to standard Bengali, in LLM question-answering across nine Bengali dialects. First, we translate and gold-label standard Bengali questions into dialectal variants adopting a retrieval-augmented generation (RAG) pipeline to prepare 4,000 question sets. Since traditional translation quality evaluation metrics fail on unstandardized dialects, we evaluate fidelity using an LLM-as-a-judge, which human correlation confirms outperforms legacy metrics. Second, we benchmark 19 LLMs across these gold-labeled sets, running 68,395 RLAIF evaluations validated through multi-judge agreement and human fallback. Our findings reveal severe performance drops linked to linguistic divergence. For instance, responses to the highly divergent Chittagong dialect score 5.44/10, compared to 7.68/10 for Tangail. Furthermore, increased model scale does not consistently mitigate this bias. We contribute a validated translation quality evaluation method, a rigorous benchmark dataset, and a Critical Bias Sensitivity (CBS) metric for safety-critical applications.

Abstract

Dialect Bias in Language Models

Explore similar work

Benchmarking Bengali Dialectal Bias: A Multi-Stage Framework Integrating RAG-Based Translation and Human-Augmented RLAIF

Sycophancy as a Multilingual Alignment Failure: How Safety Degrades Across Languages, Topics, and Models

Language Unalignability: Why Some Concepts Resist Cross-Cultural Benchmark Evaluation