LLM Bias

Large language models (LLMs) often exhibit biases reflecting societal prejudices, leading to unfair or discriminatory outputs. Current research focuses on developing methods to detect and mitigate these biases, encompassing both implicit and explicit forms across various protected attributes like race, gender, and age, using techniques such as prompt engineering, attention mechanism analysis, and counterfactual evaluations applied to models like GPT-3.5 and others. Understanding and addressing LLM bias is crucial for ensuring fairness and ethical deployment of these powerful technologies, impacting both the development of responsible AI and the avoidance of harmful societal consequences.

Papers

December 15, 2023

Taxonomy-based CheckList for Large Language Model Evaluation
Damin Zhang
Large Language Model Pre Trained Language Model Autoregressive Large Language Model LLM Bias Taxonomy Construction Ethical NaTural Language

November 15, 2023

Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis
David F. Jenny, Yann Billeter, Mrinmaya Sachan, Bernhard Schölkopf, Zhijing Jin
Language Model Absolute Stance Bias Bias Mitigation Argument Quality LLM Bias Dependency Structure Causal Fairness Open Environment

November 6, 2023

Unraveling Downstream Gender Bias from Large Language Models: A Study on AI Educational Writing Assistance
Thiemo Wambsganss, Xiaotian Su, Vinitra Swamy, Seyed Parsa Neshaei, Roman Rietsche, Tanja Käser
Study Feature LLM Bias AI Pipeline

September 16, 2023

Investigating Subtler Biases in LLMs: Ageism, Beauty, Institutional, and Nationality Bias in Generative Models
Mahammed Kamruzzaman, Md. Minul Islam Shovon, Gene Louis Kim
Generative Model Medical LLM Absolute Stance Bias Topic Bias LLM Bias Subtle Bias Negative Information Nationality Bias

September 15, 2023

Casteist but Not Racist? Quantifying Disparities in Large Language Model Bias between India and the West
Khyati Khandelwal, Manuel Tonneau, Andrew M. Bean, Hannah Rose Kirk, Scott A. Hale
Language Model Social Bias Region Specific Health Disparity Stereotypical Bias LLM Bias

September 11, 2023

Detecting Natural Language Biases with Prompt-based Learning
Md Abdul Aowal, Maliha T Islam, Priyanka Mary Mammen, Sandesh Shetty
Language Model Complex Prompt Prompt Based Learning Language Bias LLM Bias Subtle Bias

June 13, 2023

Sociodemographic Bias in Language Models: A Survey and Forward Path
Vipul Gupta, Pranav Narayanan Venkit, Shomir Wilson, Rebecca J. Passonneau
Language Model Timely Survey LLM Bias Publication Bias Way Forward

October 6, 2022

Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Tasks
Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki
Social Bias Downstream Task Masked Language Self Debiasing Bias Metric LLM Bias

LLM Bias

Papers

Taxonomy-based CheckList for Large Language Model Evaluation

Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis

Unraveling Downstream Gender Bias from Large Language Models: A Study on AI Educational Writing Assistance

Investigating Subtler Biases in LLMs: Ageism, Beauty, Institutional, and Nationality Bias in Generative Models

Casteist but Not Racist? Quantifying Disparities in Large Language Model Bias between India and the West

Detecting Natural Language Biases with Prompt-based Learning

Sociodemographic Bias in Language Models: A Survey and Forward Path

Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Tasks