Linguistic Bias in Language Models

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 66

All topics

Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages
Oct 6, 2025Tarek Naous, Anagha Savit, Carlos Rafael Catalan +17Multilingual Language Model Evaluation Cultural Bias in Language Models
Georgia Institute of Technology · Samsung R&D Institute Philippines · Sungkyunkwan University +6
Man Made Language Models? Evaluating LLMs' Perpetuation of Masculine Generics Bias
Feb 14, 2025Enzo Doyen, Amalia TodirascuGender Bias in Language Models Language Model Bias Evaluation
University of Strasbourg
Homogeneity Bias in Open-Weight LLMs Is Robust to Decoding Hyperparameters
Jan 4, 2025Messi H. J. LeeSocial Bias in Language Models Open-Weight Language Models
Independent Researcher · Seoul, Republic of Korea
General Phrase Debiaser: Debiasing Masked Language Models at a Multi-Token Level
Nov 23, 2023Bingkang Shi, Xiaodan Zhang, Dehan Kong +4Gender Bias in Language Models Debiased ML
Chinese Academy of Sciences, Institute of Information Engineering, Beijing 100093, China · Alibaba Group, Alibaba Artificial Intelligence Governance Laboratory, Beijing, China · University of Bristol, Department of Electrical and Electronic Engineering, Bristol, BS8 1UB, UK
Alignment Reduces Expressed but Not Encoded Gender Bias: A Unified Framework and Study
Date pendingNour Bouchouchi, Thibault Laugel, Xavier Renard +3Gender Bias in Language Models LLM Alignment
Sorbonne Université, CNRS, LIP6, F-75005 Paris, France · AXA, Paris, France · Polish Academy of Science, IBS PAN, Warsaw, Poland
Left-Branching Transformers Excel at Right-Branching Languages: Data Shapes Word Order Preferences in Language Models
Date pendingVarvara Arzt, Allan Hanbury, Terra BlevinsMultilingual Language Models Decoder-Only Language Models
Faculty of Informatics, TU Wien · Khoury College of Computer Sciences, Northeastern University

Oct 6, 2025, cs.CLJ/K move · Enter open · S save

Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages

Tarek Naous, Anagha Savit, Carlos Rafael Catalan, Geyang Guo+16

Georgia Institute of Technology · Samsung R&D Institute Philippines · Sungkyunkwan University+6

Multilingual Language Model Evaluation Cultural Bias in Language Models Language Model Bias Evaluation Linguistic Bias in Language Models Cultural Knowledge in Language Models

Open paper AskPDF arXiv

As Large Language Models (LLMs) develop stronger multilingual capabilities, their sensitivity to culturally diverse entities becomes increasingly important. Prior work by Naous et al. (2024) has shown that LLMs often favor Western-associated entities in Arabic. Due to the lack of entity-centric multilingual benchmarks, it remains unclear if such biases also manifest in various non-Western languages. In this paper, we introduce Camellia, a benchmark for evaluating entity-centric cultural biases in nine Asian languages, spanning six Asian cultures. Camellia includes 19,530 manually annotated entities associated with the covered Asian or Western cultures, as well as 2,173 masked contexts for these entities derived from social media posts. Using Camellia, we evaluate cultural biases in four recent multilingual LLMs across three tasks: cultural context adaptation, sentiment association, and entity extractive QA. Our analyses show that LLMs struggle with cultural adaptation across these languages, with performance differing across models developed in different regions. We further observe that different LLM families can hold distinct biases, reflected in the ways they link cultures to particular sentiments. Lastly, we find that LLMs can struggle with context understanding in some Asian languages, creating performance gaps between cultures in entity extraction.

Momentum

Related topics

Latest papers 66

Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages

Man Made Language Models? Evaluating LLMs' Perpetuation of Masculine Generics Bias

Homogeneity Bias in Open-Weight LLMs Is Robust to Decoding Hyperparameters

General Phrase Debiaser: Debiasing Masked Language Models at a Multi-Token Level

Alignment Reduces Expressed but Not Encoded Gender Bias: A Unified Framework and Study

Left-Branching Transformers Excel at Right-Branching Languages: Data Shapes Word Order Preferences in Language Models