Causally Fair Generation with Large Language Models

Patrik Okanovic; Torsten Hoefler; Drago Plecko

cs.AIOct 3, 2026

Causally Fair Generation with Large Language Models

Authors: Patrik Okanovic, Torsten Hoefler, Drago Plecko

Organizations: Department of Computer Science, ETH Zurich · Department of Statistics & Data Science, UCLA

Abstract

Large language models (LLMs) are increasingly used to generate, complete, and transform information in settings where their outputs can shape consequential decisions, raising concerns about their impact on demographic disparities. In this context, causal inference provides a principled basis for assessing fairness, because it attributes observed disparities to the mechanisms that generated them, which a purely statistical approach cannot do even with infinite data. In LLM generation, a query may request several causally related variables, each of which is both an outcome of interest and a possible cause of other outputs, and the information supplied in the prompt need not follow a topological or a temporal order. This calls for methods that can analyze and selectively remove disparities from such a flexible generation process. In this paper we introduce Causally Fair Generation with LLMs (CFG, for short). CFG extracts relevant concepts, grounds generation in a reference population and causal diagram, and removes user-selected causal effects. CFG also allows pathways deemed justifiable for the task's utility to be retained, which is known in legal literature as business necessity. Further, we provide formal guarantees for our method when eliminating all discriminatory causal effects in the adapted population model, under appropriate causal assumptions. We evaluate CFG with four LLMs in three real-world settings based on population data and on a synthetic dataset with a known causal ground truth.

Figures & tables

Panel of Figure 1 — Figure 1: Causal diagrams for the standard fairness model and its LLM adaptation. Panel (b) orders outputs topologically from left to right. Its template includes $Y_{i}\to Y_{j}$ for every $i<j$ ; only selected forward edges are drawn for readability. The general formulation permits any subset of these edges.

Figure 2: Overview of Cfg for causally fair LLM generation. A concept agent extracts and annotates concepts from a fairness-relevant query. Cfg then estimates a population model and obtains a causal diagram. A debiasing agent presents the user with supported debiasing choices, checks their compatibility across outputs, and records the selected effects and business-necessity assumptions. Finally, Cfg constructs a causally fair output.

Figure 3: Absolute recovery residuals under topological and non-topological queries.

Figure 4: Rows 1–3: topological queries with empty BN set; Row 4: non-topological query; Row 5: topological query with BN requirement. Intervals indicate 95% confidence intervals.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Twin-network over counterfactual outcomes, related to the hybrid completion setting where the user supplies descendants of $X$ . The upper path represents the factual values of $X$ and the supplied prefix $\mathbf{Y}_{\leq i}$ , and the generated suffix $\mathbf{Y}_{>i}$ ; the lower path represents the recursively substituted world obtained by setting $X=x_{0}$ .

Panel of Figure 6 — Figure 6: Standard fairness models for NSDUH, BRFSS, ACS, and Census-Synth. Generated attributes appear in topological order from left to right. In panels (a)–(c), nonconsecutive forward edges are omitted for clarity; each generated attribute also points to every later attribute. Panel (d) is shown in full.

Generator	Agreement (%)
Qwen3-4B	98.96
Llama 3-8B	98.73
Qwen3.8-27B	99.75
Gemma-4-31B	99.99

Appendix

Table 1: Sampled–annotated agreement for Cfg .

Figure 7: Empirical cumulative distributions of absolute recovery residuals.

Generator	Order	Method	Intermediate target				Final target
			TV	DE	IE	SE	TV	DE	IE	SE
Qwen3-4B	Topo.	Vanilla	7.79	4.56	2.26	1.11	10.25	8.58	0.83	2.43
		CoT	9.64	5.51	4.04	0.81	7.62	9.51	0.73	2.48
		RAG	13.92	5.88	7.16	0.88	1.74	4.80	2.30	5.36
		Cfg	0.41	0.77	0.36	0.20	0.66	0.49	0.40	0.47
	Non-topo.	Vanilla	19.90	12.38	7.85	4.99	30.68	14.59	10.04	17.58

Appendix

Table 2: Mean absolute recovery error in percentage points by generator, query order, and target position for Cfg and the prompting baselines. Dataset contributions are weighted equally after averaging ACS employer categories. Bold values indicate the smallest error within each comparison.

Figure 8: Extended effect-removal results with an empty BN set. The upper four rows show final outcomes under topological queries; the bottom row shows Census-Synth employment under non-topological queries. Columns report the selected effect, or TV when all effects are selected for removal. Intervals indicate 95% confidence intervals.

Figure 9: Causal effect removal for intermediate outcomes under topological queries. Each panel compares the effect selected for removal after debiasing with its original reference value; zero corresponds to complete removal.

Removal setting	Effect	Intermediate target		Final target
		Reference	Cfg	Reference	Cfg
Direct effect	DE	5.59	0.75	6.82	1.32
Indirect effect	IE	3.33	1.04	2.62	0.97
Spurious effect	SE	0.76	0.09	2.98	0.08
All effects	TV	8.65	0.64	6.56	0.54
	DE	5.59	0.69	6.82	0.40

Appendix

Table 3: Mean absolute effects in percentage points for the reference and Cfg -debiased distributions under topological queries. When all effects are selected for removal, we report both TV and its components.

Removal setting	Effect	Reference	Cfg
Direct effect	DE	5.39	0.19
Indirect effect	IE	4.49	0.98
Spurious effect	SE	1.44	0.07
All effects	TV	8.44	0.97
All effects	DE	5.39	0.26
All effects	IE	4.49	0.65

Appendix

Table 4: Mean absolute Census-Synth employment effects in percentage points for the reference and Cfg -debiased distributions under non-topological queries, averaged across four generators.

Figure 10: Extended effect-removal results for Census-Synth salary with

\mathbf{B}=\{\text{education}\}

, for both topological and non-topological queries. Intervals indicate 95% confidence intervals.

Removal setting	Effect	Reference	Topological	Non-topological
Direct effect	DE	6.65	2.25	1.22
Indirect effect	IE	6.96	7.26	6.33
Spurious effect	SE	4.34	0.05	0.08
All effects	TV	9.27	4.15	4.06

Appendix

Table 5: Mean absolute Census-Synth salary effects in percentage points for the reference and Cfg -debiased distributions with

\mathbf{B}=\{\text{education}\}

, averaged across four generators.

Model	Name $F_{1}$	Record $F_{1}$	In-domain	Out-of-domain	Challenge
Qwen3-4B	95.5	90.8	93.1	87.7	92.3
Llama 3-8B	91.6	85.7	90.4	83.0	78.9
Qwen3.8-27B	97.9	94.3	96.2	91.1	97.5
Gemma 4-31B	95.9	91.4	94.1	86.8	95.7

Appendix

Table 6: Few-shot performance of the concept agent for Cfg on the 100-prompt benchmark. All entries are percentages. Name and record scores are micro-averaged after semantic normalization; the final three columns report record

F_{1}

within each benchmark split.

Estimator	Log loss $\downarrow$	Brier score $\downarrow$
DRF	$\mathbf{1.1016\pm 0.0150}$	$\mathbf{0.5487\pm 0.0073}$
XGBoost	$1.1169\pm 0.0036$	$0.5519\pm 0.0070$

Appendix

Table 7: Survey-weighted held-out goodness of fit on NSDUH for the population estimators available in Cfg . Entries are means

\pm

standard deviations across five stratified 80/20 splits.

Case	$X$	$\mathbf{Z}$	Reference $\mathbf{Y}$ order
NSDUH	race	sex, age	education $\prec$ income $\prec$ monthly marijuana use
ACS	sex	age group, race, economic region	education $\prec$ weekly hours worked $\prec$ employer type $\prec$ annual salary
Hiring	gender	age	education $\prec$ relevant experience $\prec$ interview score $\prec$ hiring decision
Promotion	race	age, sex	employer training access $\prec$ job performance $\prec$ promotion decision
Salary offer	sex	age	relevant experience $\prec$ initial salary offer $\prec$ negotiation response $\prec$ final salary offer
Admissions	first generation	age, sex	household resources $\prec$ school preparation $\prec$ entrance exam score $\prec$ admission decision

Appendix

Table 8: The twelve graphs used to evaluate causal-diagram acquisition for Cfg . The first two are paper references; the remaining ten are hand-curated examples. Spaces replace underscores in variable identifiers for readability. The symbol

\prec

denotes precedence in the topological order. Adding every forward outcome edge to the fixed SFM template fully specifies each reference graph. The target input supplies only

X

and the unordered remaining names; the

\mathbf{Z}

memberships and

\mathbf{Y}

orders shown here are withheld.

Model	Role macro- $F_{1}$	Order agreement	Exact graph	Valid output
Qwen3-4B	78.1	56.4	23.3	78.3
Llama 3-8B	73.2	45.8	11.7	78.3
Qwen3.8-27B	95.3	83.3	58.3	100.0
Gemma 4-31B	94.5	87.5	60.0	100.0
ChatGPT 6 Astra (high)	97.1	95.8	85.0	100.0

Appendix

Table 9: SFM diagram retrieval for Cfg on twelve graphs, with five variable-order presentations per graph and model. All entries are percentages, and higher is better.

Method	TV	DE	SE	IE
Vanilla	9.18	7.25	4.92	3.87
CoT	9.96	6.85	5.48	3.82
RAG	9.20	5.07	3.43	9.19
Cfg	1.59	1.85	1.19	0.73

Appendix

Table 10: Mean absolute errors in reference-effect matching with supplied

(X,\mathbf{Z})

, averaged across generators, in percentage points.

Figure 11: Effect removal for final outcomes with supplied

(X,\mathbf{Z})

Algorithmic Fairness

Algorithmic fairness studies how automated decisions distribute errors, opportunities, and harms across groups, and how to measure and reduce disparities.

See all 420 papers

Causal Bias Detection in Generative Artificial Intelligence
May 12, 2026Drago PleckoAlgorithmic Fairness Language Model Bias Evaluation
Department of Statistics & Data Science UCLA
COFT: Counterfactual-Conformal Decoding for Fair Chain-of-Thought Reasoning in Large Language Models
May 28, 2026Arya Fayyazi, Mehdi Kamal, Massoud PedramSocial Bias in Language Models Counterfactual Fairness
Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California, USA.
Automatic Causal Fairness Analysis with LLM-Generated Reporting
Apr 29, 2026Alessia Berarducci, Eric Rossetto, Alessandro Antonucci +1Algorithmic Fairness Counterfactual Fairness
Istituto Dalle Molle di Studi sull’Intelligenza Artificiale (IDSIA), USI-SUPSI, Lugano, Switzerland

May 12, 2026J/K move · Enter open · S save

Causal Bias Detection in Generative Artificial Intelligence

Drago Plecko

Department of Statistics & Data Science UCLA

Algorithmic Fairness Language Model Bias Evaluation Causal Inference Demographic Bias in Language Models

Open paper AskPDF arXiv

Automated systems built on artificial intelligence (AI) are increasingly deployed across high-stakes domains, raising critical concerns about fairness and the perpetuation of demographic disparities that exist in the world. In this context, causal inference provides a principled framework for reasoning about fairness, as it links observed disparities to underlying mechanisms and aligns naturally with human intuition and legal notions of discrimination. Prior work on causal fairness primarily focuses on the standard machine learning setting, where a decision-maker constructs a single predictive mechanism

f_{\widehat Y}

for an outcome variable

Y

, while inheriting the causal mechanisms of all other covariates from the real world. The generative AI setting, however, is markedly more complex: generative models can sample from arbitrary conditionals over any set of variables, implicitly constructing their own beliefs about all causal mechanisms rather than learning a single predictive function. This fundamental difference requires new developments in causal fairness methodology. We formalize the problem of causal fairness in generative AI and unify it with the standard ML setting under a common theoretical framework. We then derive new causal decomposition results that enable granular quantification of fairness impacts along both (a) different causal pathways and (b) the replacement of real-world mechanisms by the generative model's mechanisms. We establish identification conditions and introduce efficient estimators for causal quantities of interest, and demonstrate the value of our methodology by analyzing race and gender bias in large language models across different datasets.

Abstract

Algorithmic Fairness

Explore similar work

Causal Bias Detection in Generative Artificial Intelligence

COFT: Counterfactual-Conformal Decoding for Fair Chain-of-Thought Reasoning in Large Language Models

Automatic Causal Fairness Analysis with LLM-Generated Reporting