Language Model Bias Evaluation

Follow this topic Ask about this topic

Momentum

40 papers in the last four weeks, up 122% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 321

All topics

IndoBias: A Dual Track Culturally Grounded Benchmark for LLMs Bias Evaluation in Indonesian Languages
May 31, 2026Ikhlasul Akmal Hanif, Muhammad Falensi Azmi, Filbert Aurelian Tjiaranata +2Multilingual Language Model Evaluation Cultural Bias in Language Models
Mohamed bin Zayed University of Artificial Intelligence · Independent Researcher · Universitas Indonesia
Implicit Geographic Inference in LLM Medical Triage: Language-Driven Disparities in Emergency Recommendations
May 31, 2026Qi Han WongMultilingual Language Model Evaluation Clinical Triage
Isolating LLM Lexical Bias: A Curation-Free Triangulated Metric for Preference-Stage Learning
May 29, 2026Xiaoyang Ming, Jose Hernandez, Thomas Stephan JuzekLLM Alignment LLM Fine-Tuning
Florida State University
A Visually Impaired Assistance Benchmark for VLM-as-a-Judge Evaluation
May 29, 2026Yi Zhao, Siqi Wang, Zhe Hu +2VLM Evaluation LLM-as-a-Judge
Department of Computing, The Hong Kong Polytechnic University
Anchoring LLM Gender Bias to Human Baselines: A Cross-Lingual Audit
May 29, 2026Jiwoo Choi, Seonwoo Ahn, Tongxin Zhang +1Gender Bias in Language Models Multilingual Language Model Evaluation
School of Digital Humanities and Computational Social Sciences, KAIST, South Korea
COFT: Counterfactual-Conformal Decoding for Fair Chain-of-Thought Reasoning in Large Language Models
May 28, 2026Arya Fayyazi, Mehdi Kamal, Massoud PedramSocial Bias in Language Models Counterfactual Fairness
Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California, USA.
When English Rewrites Local Knowledge: Global Narrative Dominance in Large Language Models
May 28, 2026Md Arid Hasan, Ruwad Naswan, Farhan Samir +2Multilingual Language Model Evaluation LLM Grounding
University of Toronto · BUET · University of Illinois Urbana-Champaign
Label Over Logic? How Source Cues Bias Human Fallacy Judgments More Than LLMs
May 28, 2026Mahjabin Nahar, Nafis Irtiza Tripto, Aiping Xiong +2LLM Evaluation Language Model Bias Evaluation
The Pennsylvania State University
PRAIB: Peer Review AI Benchmark of Behaviour of LLM-Assisted Reviewing
May 28, 2026Krzysztof Żurawicki, Julia Farganus, Arkadiusz Gaweł +2LLM Evaluation Scholarly Peer Review
Department of Artificial Intelligence Wrocław University of Science and Technology
Persona Prompting in Multimodal Urban Perception: Descriptive Convergence and Interpretive Variation
May 27, 2026Neemias da Silva, Matt Ratto, Myriam Delgado +3Multimodal Large Language Models Language Model Bias Evaluation
University of Toronto, Toronto, Canada · Universidade Tecnologica Federal do Parana, Curitiba, Brazil
Do LLMs Favor Their Providers? Measuring Vertical Integration Bias in Code Generation
May 27, 2026Melih Catal, Alex Wolf, Tiago Ferreiro Matos +2Agentic Code Generation Algorithmic Bias
University of Zurich · University of Mannheim
DEPART: DEcomposing PARiTy across Multilingual LLMs
May 27, 2026Manan Uppadhyay, Prashant Kodali, Pranjal Chitale +3Multilingual Language Model Evaluation Cross-Lingual Reasoning in Language Models
Microsoft Research India · IIT Gandhinagar
Examining Agents' Bias Amplification versus Suppression in Multi-Agent Systems
May 27, 2026Zejian Eric Wu, Zhongyi Jiang, Yuan Zhuang +1Multi-Agent LLM Systems Algorithmic Fairness
Oregon State University · Independent Researcher · Amazon +1
MIRA: A Bilingual Benchmark for Medical Information Response Audit
May 27, 2026Mengyu Xu, Qiaoxin Yang, Qianqian Wang +3Multilingual Language Model Evaluation Healthcare
The University of Chicago · The Chinese University of Hong Kong, Shenzhen · Jinzhou Medical University +3
Reward Bias Substitution: Single-Axis Bias Mitigations Redirect Optimization Pressure
May 27, 2026Max Lamparth, Daniel Fein, Andreas Haupt +2Reward Modeling RL for Language Models
Stanford University · University of Pennsylvania
Auditing Stance Asymmetry in Generative Explanations
May 27, 2026Jiarui HanLanguage Model Generation Evaluation Social Bias in Language Models
LLM-Ideoplasticity: Measuring Ideological Plasticity in the Political Behavior of LLMs as a Context-Conditioned Distribution
May 26, 2026Adib Sakhawat, Syed Rifat Raiyan, Tahsin Islam +3Political Bias in Language Models Language Model Bias Evaluation
Systems and Software Lab (SSL) Department of Computer Science and Engineering Islamic University of Technology, Dhaka, Bangladesh
Insidious by Design: Implications of Large Language Model algorithmic bias for the Global South
May 26, 2026Sioux McKenna, Nompilo TshumaGender Bias in Language Models Cultural Bias in Language Models
Rhodes University, Makhanda, South Africa · Stellenbosch University, Stellenbosch, South Africa
LLM-as-a-Reviewer: Benchmarking Their Ability, Divergence, and Prompt Injection Resistance as Paper Reviewers
May 25, 2026Lingyao Li, Junjie Xiong, Changjia Zhu +5LLM Evaluation LLM-as-a-Judge
University of South Florida · Missouri University of Science and Technology · University of Alabama +3
Side-by-side Comparison Amplifies Dialect Bias in Language Models
May 23, 2026Kritee Kondapally, Claire J. Smerdon, Pooja C. Patel +5Social Bias in Language Models Algorithmic Fairness
University of Southern California, USA
Omissive Bias in Religious Representation: Benchmarking LLM Answers to Everyday Ethical Decision-making
May 23, 2026David Wingate, Sheryl Carty, Joshua Coates +13Moral Reasoning in Language Models Social Bias in Language Models
1Brigham Young University · 2B. H. Roberts Foundation · 5Yeshiva University +2
It's the humans, not the data: Geopolitical bias in LLMs originates in post-training, amplified by the language of the prompt
May 22, 2026Stuart Bladon, Brinnae BentPolitical Bias in Language Models Language Model Bias Evaluation
Duke University
When AI Takes Sides on Questions of Faith: Persistent Asymmetries in AI-Mediated Faith Guidance
May 21, 2026Brett Israelsen, Sheryl Carty, Josh Coates +3Social Bias in Language Models Language Model Bias Evaluation
Brigham Young University · B. H. Roberts Foundation
Evaluating Commercial AI Chatbots as News Intermediaries
May 21, 2026Mirac Suzgun, Emily Shen, Federico Bianchi +5LLM Evaluation Question Answering
1Stanford University · 2Independent Researcher · 3Together AI.
Reducing Political Manipulation with Consistency Training
May 21, 2026Long Phan, Devin Kim, Alexander Pan +3Social Bias in Language Models Political Bias in Language Models
Center for AI Safety · UC Berkeley
AMEL: Accumulated Message Effects on LLM Judgments
May 21, 2026Sid-Ali TemkitLLM Evaluation Language Model Generation Evaluation
Mechanics of Bias and Reasoning: Interpreting the Impact of Chain-of-Thought Prompting on Gender Bias in LLMs
May 19, 2026Edie Pearman, Sophia Osborne, Mira Kandlikar-Bloch +3Gender Bias in Language Models LLM Interpretability
1Mila – Quebec AI Institute, Montreal, Canada · 2McGill University, Montreal, Canada · 3HEC Montreal, Montreal, Canada
Occupational Prompting Reveals Cultural Bias in Large Language Models
May 19, 2026Maksim E. Eren, Andrea Brennen, Ryan C. Barron +1LLM Prompting Social Bias in Language Models
Computational Intelligence & Modeling, Los Alamos National Laboratory, Los Alamos, New Mexico, USA. · IQT, USA. · Advanced Research in Cyber Systems, Los Alamos National Laboratory, Los Alamos, New Mexico, USA.
Diagnosing Korean-Language LLM Political Bias via Census-Grounded Agent Simulation
May 18, 2026Sungwoo KangPolitical Bias in Language Models Language Model Calibration
Department of Electrical and Computer Engineering Korea University, Seoul, Republic of Korea
GPF-LiveNews: A Streaming Evaluation Protocol for Group-Conditioned Framing in Large Language Models
May 16, 2026Mohd Ariful Haque, Fahad Rahman, Kishor Datta Gupta +1LLM Evaluation LLM Auditing
Clark Atlanta University · United International University

May 31, 2026, cs.CLJ/K move · Enter open · S save

IndoBias: A Dual Track Culturally Grounded Benchmark for LLMs Bias Evaluation in Indonesian Languages

Ikhlasul Akmal Hanif, Muhammad Falensi Azmi, Filbert Aurelian Tjiaranata, Eryawan Presma Yulianrifat+1

Mohamed bin Zayed University of Artificial Intelligence · Independent Researcher · Universitas Indonesia

Multilingual Language Model Evaluation Cultural Bias in Language Models Language Model Bias Evaluation Linguistic Bias in Language Models

Open paper AskPDF arXiv

Despite being home to more than 1300 ethnic groups and 700 indigenous languages, bias in Large Language Models has not been fully studied in Indonesia, thus leaving a critical gap in evaluating representational fairness and localized stereotypes within its uniquely vast, multilingual, and diverse sociocultural landscape. To address this, we introduce IndoBias as a culturally-grounded bias benchmark to assess LLMs bias in Indonesian and three local languages: Javanese, Sundanese, and Makasar. IndoBias features dual perspective evaluation tracks: depth-oriented (with contrastive-pairs) and breadth-oriented (with generation-based), where the latter is grounded in social science frameworks (SPI, O*NET, and WGI). Our results show that existing LLMs -- particularly decoder models -- exhibit strong bias towards prototypical sentences in Indonesian, while local languages suffer higher bias under Ideology and Religion category. We also find that LLMs responses exhibit a non-uniform Stereotype Polarity when prompted with various local entities. Finally, we discover that, in Indonesian, Common Crawl texts introduce more bias during pretraining, compared to human-reviewed article texts (e.g., Wikipedia, News), whereas introducing local languages to pretraining generally increases bias. This work highlights the importance of studying bias in culture-specific context. Warning: This paper contains example data that may be offensive, harmful, or biased.

Momentum

Related topics

Latest papers 321