Fairness Definitions in Language Models Explained

Zhipeng Yin; Zichong Wang; Avash Palikhe; Wenbin Zhang

cs.CLJul 26, 2024

Fairness Definitions in Language Models Explained

Authors: Zhipeng Yin, Zichong Wang, Avash Palikhe, Wenbin Zhang

Organizations: Florida International University, Miami, USA

Abstract

Language Models (LMs) have demonstrated exceptional performance across various Natural Language Processing (NLP) tasks. Despite these advancements, LMs can inherit and amplify societal biases related to sensitive attributes such as gender and race, limiting their adoption in real-world applications. Therefore, fairness has been extensively explored in LMs, leading to the proposal of various fairness notions. However, the lack of clear agreement on which fairness definition to apply in specific contexts and the complexity of understanding the distinctions between these definitions can create confusion and impede further progress. To this end, this paper proposes a systematic survey that clarifies the definitions of fairness as they apply to LMs. Specifically, we begin with a brief introduction to LMs and fairness in LMs, followed by a comprehensive, up-to-date overview of existing fairness notions in LMs and the introduction of a novel taxonomy that categorizes these concepts based on their transformer architecture: encoder-only, decoder-only, and encoder-decoder LMs. We further illustrate each definition through experiments, showcasing their practical implications and outcomes. Finally, we discuss current research challenges and open questions, aiming to foster innovative ideas and advance the field. The repository is publicly available online at https://github.com/vanbanTruong/Fairness-in-Large-Language-Models/tree/main/definitions.

Figures & tables

Figure 1: An overview of the proposed taxonomy of fairness definitions in language models.

Notations	Descriptions
$T$	The socially sensitive topic
$G$	The set of demographic groups
$A_{i}$	The list of sensitive attributes associated with $i$ -th demographic group
$S_{i}$	The set of sentences represented for $i$ -th demographic group
$g_{i}$	The $i$ -th demographic group
$a_{i,j}$	The $j$ -th sensitive attribute of the $i$ -th demographic group

Table 1: Notations employed for defining the problem and describing the methodology.

Architecture	Bias type	Definition	Model	Dataset	Sensitive attribute	References
Encoder-only LMs	Intrinsic bias	Similarity -based Disparity	BERT [ 48 ]	Caliskan et al. [ 24 ]	gender race age disease	[ 24 ] , [ 102 ] , [ 64 ]
		Probability -based Disparity		Bias-in-Bios [ 46 ] CrowS-Pairs [ 108 ] StereoSet [ 107 ] WinoBias [ 207 ] XNLI [ 42 ]	gender race religion nationality	[ 169 ] , [ 82 ] , [ 6 ] , [ 124 ] , [ 107 ] , [ 108 ] , [ 78 ]
	Extrinsic bias	Equal Opportunity	RoBERTa [ 97 ]	Bias-in-Bios [ 46 ] BBQ [ 113 ] WinoBias [ 207 ]	gender race	[ 66 ] , [ 131 ] , [ 46 ]
		Fair Inference				[ 7 ] , [ 19 ] , [ 47 ]
		Context -based Disparity				[ 113 ] , [ 45 ]
Decoder-only LMs	Intrinsic bias	Attention head -based Disparity	GPT-2 [ 119 ]	StereoSet [ 107 ] TheRedPill corpus [ 56 ] Winogender [ 123 ]	gender occupation	[ 173 ] , [ 146 ]

Table 2: Summary of experimental setup.

Figure 2: An example of similarity-based bias in encoder-only LMs.

Figure 3: An example of probability-based bias with masked token metrics in encoder-only LMs.

Figure 4: An example of probability-based bias with pseudo-log-likelihood metrics in encoder-only LMs.

Figure 5: An example of the extrinsic bias of encoder-only LMs in classification task.

Figure 6: An example of the extrinsic bias of encoder-only LMs in natural language inference downstream task.

Figure 7: An example of the extrinsic bias in encoder-only LMs in question-answering downstream task.

Figure 8: An example of attention head-biased bias in a decoder-only LM.

Figure 9: An example in the intrinsic bias of decoder-ony LMs based on the stereotypical association.

Figure 10: An example in the extrinsic bias of decoder-only LMs based on the counterfactual fairness.

Figure 11: An example in the extrinsic bias of decoder-only LMs based on the performance disparities evaluation method.

Figure 12: An example in the extrinsic bias of decoder-only LMs based on the demographic representation.

Figure 13: An example of algorithmic bias in an encoder-decoder LM.

Figure 14: An example of stereotypical association in encoder-decoder LMs.

Figure 15: An example of position-based-bias in encoder-decoder LMs.

Figure 16: An example of the extrinsic bias of encoder-only LMs in cross-lingual natural language inference task.

Figure 17: An example of individual fairness in encoder-decoder LMs.

Figure 18: An example of the extrinsic bias of encoder-decoder LMs in the recommendation downstream task.

Social Bias in Language Models

Social bias in language models is the study and mitigation of unfair stereotypes and disparities in generated text, judgments, and decisions.

See all 199 papers

FairLMs: A Turnkey Library for Fairness in Language Models
Sep 18, 2026Jiale Zhang, Michael Larionov, Zichong Wang +2Language Model Bias Evaluation Social Bias in Language Models
Department of Biostatistics and Health Data Science Indiana University Indianapolis, IN, United States · Department of Statistics and Data Science Carnegie Mellon University Pittsburgh, PA, United States · Knight Foundation School of Computing and Information Sciences Florida International University Miami, FL, United States
Explanation Fairness in Large Language Models: An Empirical Analysis of Disparities in How LLMs Justify Decisions Across Demographic Groups
May 9, 2026Gautam VeldandaLLM Auditing Algorithmic Fairness
Independent Researcher
Beyond Model Weights: Sampling-Dependent Fairness in Diffusion Language Models
Oct 20, 2026Koshiro Saito, Daisuke Oba, Danushka Bollegala +1Social Bias in Language Models Diffusion Language Models
Institute of Science Tokyo · University of Liverpool · Amazon +2

Sep 18, 2026J/K move · Enter open · S save

FairLMs: A Turnkey Library for Fairness in Language Models

Jiale Zhang, Michael Larionov, Zichong Wang, Zhipeng Yin+1

Department of Biostatistics and Health Data Science Indiana University Indianapolis, IN, United States · Department of Statistics and Data Science Carnegie Mellon University Pittsburgh, PA, United States · Knight Foundation School of Computing and Information Sciences Florida International University Miami, FL, United States

Language Model Bias Evaluation Social Bias in Language Models Language Model Debiasing

Open paper AskPDF arXiv

Fairness research on language models involves measuring bias, applying mitigation methods, and examining the evidence on which an evaluation rests. Existing tools offer complementary functionality through different interfaces, so combining them requires reconciling model interfaces, evidence formats, access constraints, and result types before applicability can be checked or methods compared. We introduce FairLMs, a Python library that connects these activities through explicit declarations of model capabilities and input requirements. It provides 33 intrinsic and extrinsic metrics, 14 mitigation components spanning four intervention categories, 14 dataset and scoring-instrument diagnostics, adapters for the three Transformer architectures and supported hosted completion APIs, and benchmark loaders. Declarations are checked before execution and results carry the configuration under which they were obtained, so that compatible components can be combined, methods compared under a common protocol, and workflows extended to new models and datasets. The source code is available at: https://github.com/FairLMs/FairLMs.

Abstract

Social Bias in Language Models

Explore similar work

FairLMs: A Turnkey Library for Fairness in Language Models

Explanation Fairness in Large Language Models: An Empirical Analysis of Disparities in How LLMs Justify Decisions Across Demographic Groups

Beyond Model Weights: Sampling-Dependent Fairness in Diffusion Language Models