Fair Like Us? Auditing LLM Alignment in Resource Allocation

Qishen Han; Hadi Hosseini; Joshua Kavner; Samarth Khanna; Sujoy Sikdar; Lirong Xia

cs.AIAug 31, 2026

Fair Like Us? Auditing LLM Alignment in Resource Allocation

Authors: Qishen Han, Hadi Hosseini, Joshua Kavner, Samarth Khanna, Sujoy Sikdar, Lirong Xia

Organizations: Rutgers University · Pennsylvania State University · Independent Researcher · Binghamton University

Abstract

Fair allocation of scarce, indivisible resources is an important challenge in many societal problems. While there are several formal theories of fairness, no single definition can always be satisfied. As large language models (LLMs) are increasingly used to support decisions and act as agents, they raise new concerns about distributional justice: their judgments are not directly tied to any specific fairness framework and may violate key normative principles. In this work, we introduce a general method for evaluating fairness reasoning in LLMs. We study first-person fairness judgments across a broad set of models and compare them directly with human responses on matched scenarios and elicitation conditions. We find that LLMs tend to prefer stricter fairness constraints than humans, show more self-interested behavior, are sensitive to how information is framed, and are difficult to align with human judgments using fine-tuning with current datasets.

Figures & tables

Figure 1: An example prompt presenting a fair division instance. Text in red , green , and blue highlight (private) valuation information, (personal) allocation information, and the (threshold-based) question eliciting perceived fairness judgments, respectively.

Figure 2: Relative fractional-share guarantees of common fairness properties (not to scale) and our empirical results on perceived fairness. Stronger fairness properties appear to the right and guarantee larger minimum shares of total additive value (e.g., EF $\Rightarrow$ PROP $\Rightarrow$ PROP1). EF1 (not depicted) falls between PROP and PROP1 and is incomparable to MMS.

Dimension	Variation	Description of what the participant sees	T1	T2	T3	T4	T5
Bundle Information	Personal	All items in their own bundle. Remaining items are shown as a single “other” bundle (how they are split among other agents is hidden).	✓
Bundle Information	Full	Every agent’s bundle.		✓	✓	✓	✓
Value Information	Private	Their own value for every item in the instance, and their additive value for each visible bundle.	✓	✓	✓
Value Information	Public	Private information, plus each other agent’s (i) additive value for their own bundle, (ii) additive value for the participant’s bundle.				✓	✓
Question Framing	Threshold-based	“Do you find your share acceptable?” (Yes = judged fair / acceptable).	✓	✓		✓
Question Framing	Comparison-based	“Would you like to keep the treasure assigned to you or swap with one of the other pirates?” (No swap = judged fair / acceptable).			✓		✓

Table 1: What participants observe and how responses map to fairness across five treatments differing along three dimensions: bundle information, value information, and question framing.

Model	$Acc_{p}$ [95% CI]	$Acc_{m}$ [95% CI]
Phi-4-14B	67.40 [65.0, 69.7]	81.87 [79.8, 83.7]
Gemma-3-27B	55.67 [53.1, 58.2]	68.20 [65.8, 70.5]
Gemini-2.5-F	62.27 [59.8, 64.7]	73.47 [71.2, 75.6]
DeepSeek-70B	62.80 [60.3, 65.2]	81.67 [79.6, 83.5]
DeepSeek-R1 ∗	55.13 [52.6, 57.6]	60.00 [57.5, 62.5]
Gemini-2.5-P ∗	60.20 [57.7, 62.7]	69.27 [66.9, 71.6]

Table 2: Pointwise accuracy (

Acc_{p}

) and Modal accuracy (

Acc_{m}

) of LLMs against humans judgments across all treatments and notions. A ‘

*

’ indicates advanced reasoning LLMs. Brackets give Wilson

95\%

confidence intervals over the

1{,}500

responses per model (

1{,}498

for GPT-5.6-Terra; see Appendix I ).

Figure 3: $Acc_{p}$ achieved by LLMs for each notion across all treatments. “Baseline” represents the accuracy achieved by selecting the modal human response for each unique question.

Figure 4: Perceived fairness rates by fairness notion and participant type.

Figure 5: Swap tendencies of humans and LLMs by swap type in treatment T5, conditioned on the availability of different types of swaps.

	Bundle Info.		Value Info.		Question Framing
Model	Personal	Full	Private	Public	Threshold	Comparison
Humans	80	86	58 ∗	47	73 ∗	31
Phi-4-14B	36	69 ∗	50 ∗	39	63 ∗	26
Gemma-3-27B	4	43 ∗	38	33	42 ∗	28
Gemini-2.5-F	33	45 ∗	40 ∗	29	43 ∗	27
DeepSeek-70B	77	73	55 ∗	40	58 ∗	36

Table 3: Perceived fairness rates by treatment and participant type. For each model, the higher perceived fairness rate is shown in bold; an asterisk (

*

) indicates statistical significance at

p<0.05

Figure 6: Criteria used by LLMs while selecting the answer in different treatments, split by the notion satisfied in the allocation scenario presented. Labels are produced by the GPT-OSS-120B judge ( Appendix G ).

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Percentage of questions where allocations were perceived as fair, by fairness property and model, across different setting variations, for treatment T1.

Figure 8: Percentage of questions where allocations were perceived as fair, by fairness property and model, across different setting variations, for treatment T1.

Figure 9: Percentage of questions where allocations were perceived as fair, by fairness property and model, across different setting variations, for treatments T2-T4.

Figure 10: Percentage of questions where allocations were perceived as fair, by fairness property and model, across different setting variations, for treatments T2-T4.

Figure 11: Percentage of questions where allocations were perceived as fair, by fairness property and model, across different persona variations, for treatment T1.

Figure 12: Percentage of questions where allocations were perceived as fair, by fairness property and model, across different persona variations, for treatments T2-T4.

Model	EF	PROP	MMS	PROP1	EF1
Humans	80.83	66.33	46.00	48.00	39.17
Phi-4-14B	97.08 $\uparrow$	64.00	19.33 $\downarrow$	21.33 $\downarrow$	21.67 $\downarrow$
Gemma-3-27B	74.17	40.67 $\downarrow$	12.33 $\downarrow$	15.33 $\downarrow$	19.58 $\downarrow$
Gemini-2.5-F	94.17 $\uparrow$	55.33 $\downarrow$	7.00 $\downarrow$	8.67 $\downarrow$	14.58 $\downarrow$
DeepSeek-70B	81.67	58.33	42.00	42.00	34.17
DeepSeek-R1	93.75 $\uparrow$	52.67 $\downarrow$	10.00 $\downarrow$	11.67 $\downarrow$	12.08 $\downarrow$

Appendix

Table 4: Percentage of responses perceived as fair, by fairness notion, across all treatments where the fairness notion is considered. Arrows denote statistically significant increases or decreases relative to human responses (at

p<0.05

). GPT-5.6-Terra shows the most extreme separation of any model we evaluate: it accepts every EF allocation and almost no MMS or PROP1 allocation.

Model	Selfish	Pareto	Selfless
Humans	71.66	47.50	17.07
Phi-4-14B	99.47 $\uparrow$	85.00 $\uparrow$	0.00 $\downarrow$
Gemma-3-27B	67.38	80.00 $\uparrow$	2.44
Gemini-2.5-F	97.33 $\uparrow$	82.50 $\uparrow$	0.00 $\downarrow$
DeepSeek-70B	79.14	85.00 $\uparrow$	0.00 $\downarrow$
DeepSeek-R1	34.76 $\downarrow$	85.00 $\uparrow$	0.00 $\downarrow$

Appendix

Table 5: Likelihood of making different types of swaps, as a percentage of the total number of questions where the corresponding type of swap was possible. Arrows denote statistically significant increases or decreases relative to human responses (at

p<0.05

Model	Notion	Pirate (Treasure)	Pirate (Reward)	Burglar (Treasure)	Sibling (Treasure)	Sibling (Reward)	Competitor (Reward)	Person (Reward)	Refugee (Disaster)
Phi-4-14B	LARGE	1.00	1.00	1.00	1.00	1.00	1.00	1.00	0.95
	SMALL	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
	PROP	1.00	0.97	0.98	1.00	1.00	1.00	1.00	$\downarrow$ 0.27
	PROP1	0.03	0.03	0.03	0.03	0.03	0.07	0.03	0.00
	MMS	0.03	0.03	0.02	0.02	0.02	0.05	0.03	0.02
Gemma-3-27B	LARGE	0.95	1.00	0.90	0.92	1.00	1.00	1.00	0.97

Appendix

Table 6: Fractions of allocations judged fair (out of

N=60

) satisfying each criterion, across different setting variations, for treatment T1. Arrows indicate statistically significant differences relative to the baseline Pirate (Treasure) using two-sided Fisher’s exact test (

\alpha=0.05

Model	Notion	Pirate (Treasure)	Pirate (Reward)	Burglar (Treasure)	Sibling (Treasure)	Sibling (Reward)	Competitor (Reward)	Person (Reward)	Refugee (Disaster)
Phi-4-14B	EF	0.97	0.94	0.99	0.98	0.98	$\downarrow$ 0.91	0.96	$\downarrow$ 0.84
	PROP	0.56	0.49	0.63	0.61	0.50	$\downarrow$ 0.44	$\downarrow$ 0.45	$\downarrow$ 0.39
	EF1	0.21	0.16	0.28	0.25	0.19	$\downarrow$ 0.13	$\downarrow$ 0.13	$\downarrow$ 0.12
	PROP1	0.25	0.24	0.31	0.34	0.20	$\downarrow$ 0.16	$\downarrow$ 0.16	$\downarrow$ 0.14
	MMS	0.23	0.18	0.31	$\uparrow$ 0.33	0.21	$\downarrow$ 0.13	$\downarrow$ 0.14	$\downarrow$ 0.14
Gemma-3-27B	EF	0.75	$\downarrow$ 0.52	$\downarrow$ 0.49	$\downarrow$ 0.54	$\downarrow$ 0.60	$\downarrow$ 0.51	$\downarrow$ 0.53	$\downarrow$ 0.62

Appendix

Table 7: Fractions of allocations judged fair (out of

N=240

) satisfying each criterion, across different setting variations, for treatments T2-T5. Arrows indicate statistically significant differences relative to the baseline Pirate (Treasure) using two-sided Fisher’s exact test (

\alpha=0.05

Model	Notion	Baseline	Generous	Selfish
Phi-4-14B	LARGE	1.00	1.00	$\downarrow$ 0.90
	SMALL	0.00	0.07	0.00
	PROP	1.00	1.00	$\downarrow$ 0.03
	PROP1	0.03	$\uparrow$ 0.25	0.00
	MMS	0.03	$\uparrow$ 0.25	0.00
Gemma-3-27B	LARGE	1.00	0.98	0.98

Appendix

Table 8: Fractions of allocations judged fair (out of

N=60

) when provided “generous” and “selfish” persona descriptions, for treatment T1. Arrows indicate statistically significant differences relative to the Person (Reward) baseline using two-sided Fisher’s exact test (

\alpha=0.05

Model	Notion	Baseline	Generous	Selfish
Phi-4-14B	EF	0.96	0.93	$\downarrow$ 0.87
	PROP	0.45	0.44	$\downarrow$ 0.32
	EF1	0.13	0.18	$\downarrow$ 0.03
	PROP1	0.16	0.22	$\downarrow$ 0.04
	MMS	0.14	0.20	$\downarrow$ 0.04
Gemma-3-27B	EF	0.53	0.56	$\uparrow$ 0.78

Appendix

Table 9: Fractions of allocations judged fair (out of

N=20

) when provided “generous” and “selfish” persona descriptions, for treatments T2-T5. Arrows indicate statistically significant differences relative to the Person (Reward) baseline using two-sided Fisher’s exact test (

\alpha=0.05

Model	EF	PROP	MMS	PROP1	EF1
Humans	80.83 [75.4, 85.3]	66.33 [60.8, 71.4]	46.00 [40.4, 51.7]	48.00 [42.4, 53.6]	39.17 [33.2, 45.5]
Phi-4-14B	97.08 [94.1, 98.6]	64.00 [58.4, 69.2]	19.33 [15.3, 24.2]	21.33 [17.1, 26.3]	21.67 [16.9, 27.3]
Gemma-3-27B	74.17 [68.3, 79.3]	40.67 [35.3, 46.3]	12.33 [9.1, 16.5]	15.33 [11.7, 19.8]	19.58 [15.1, 25.1]
Gemini-2.5-F	94.17 [90.4, 96.5]	55.33 [49.7, 60.9]	7.00 [4.6, 10.5]	8.67 [6.0, 12.4]	14.58 [10.7, 19.6]
DeepSeek-70B	81.67 [76.3, 86.1]	58.33 [52.7, 63.8]	42.00 [36.6, 47.7]	42.00 [36.6, 47.7]	34.17 [28.5, 40.4]
DeepSeek-R1	95.00 [91.5, 97.1]	53.33 [47.7, 58.9]	11.00 [7.9, 15.0]	12.67 [9.4, 16.9]	12.08 [8.5, 16.8]

Appendix

Table 10: Percentage of responses perceived as fair by fairness notion, with Wilson

95\%

confidence intervals.

Figure 13: Pointwise accuracy ( $Acc_{p}$ ) for each treatment.

Figure 14: Fraction of responses the allocations were considered fair, with either type of question framing. For the threshold-based framing, the y-axis represents the fraction of responses where the answer is “Yes.”, and for the comparison-based framing, it represents the fraction of responses where the agent chose to not swap their bundle. The hashed section on each bar represents the fraction of questions where the agent is envy-free.

Figure 15: Decrease in overall perceived fairness, due to a change in the question framing (indicated by the legend) in treatments with different types of bundle information (“Personal” vs. “Full”) and value information (“Private” vs. “Public”).

Figure 16: Fraction of responses corresponding to all five treatments, where bundles satisfying a given notion are perceived as fair.

Field	Agreement	Cohen’s $\kappa$
Attributed fairness criterion	77.5%	0.70
Criterion matches allocation	87.6%	0.71

Appendix

Table 13: Agreement between the GPT-OSS-120B judge used in Section 5.5 and the DeepSeek-R1 validation judge, over the

4{,}478

responses both labeled successfully.

Model	Stage	T1	T2	T3	T4	T5	Combined
Gemma-3-27B	Before	32.8	79.5	70.1	37.7	58.2	55.65
Gemma-3-27B	After	70.5	79.5	16.41	64.2	17.9	49.70
Phi-4-14B	Before	55.7	27.4	80.6	5.66	60.71	46.01
Phi-4-14B	After	78.7	79.5	88.1	64.2	53.6	72.82
Baseline		88.5	84.9	88.1	79.2	66.1	81.36

Appendix

Table 14: Pointwise Accuracy (

Acc_{p}

) of LLM responses at replicating human judgments before and after fine-tuning, by treatment. “Baseline” selects the modal human response for each question.

LLM Alignment

Large language model alignment studies methods for making models follow human intentions, values, and safety constraints across contexts.

See all 611 papers

FairFund-Bench: Evaluating Distributive Bias in LLM Resource Allocation
Jul 31, 2026Martin LukkBenchmark Auditing LLM Auditing
University of Toronto
FairJudge: An Adaptive, Debiased, and Consistent LLM-as-a-Judge
Feb 6, 2026Bo Yang, Lanfei Feng, Yunkui Chen +3LLM Evaluation LLM-as-a-Judge
College of Computer Science, Zhejiang University, Hangzhou, China · School of Software Technology, Zhejiang University, Hangzhou, China
LLMs Can Better Capture Human Judgments--With the Right Prompts
Jun 10, 2026Danica Dillion, Chen Cecilia Liu, Baihui Wang +5Human Preference Evaluation LLM Alignment
Complexity Science Hub, Vienna · The Ohio State University · University of Cambridge +2

Jul 31, 2026J/K move · Enter open · S save

FairFund-Bench: Evaluating Distributive Bias in LLM Resource Allocation

Martin Lukk

University of Toronto

Benchmark Auditing LLM Auditing Fair Resource Allocation Language Model Bias Evaluation Demographic Bias in Language Models

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly involved in the distribution of scarce resources, raising concerns about biased allocations based on characteristics like race and gender. Recent LLM audits have produced inconsistent results, however, finding evidence of both positive and negative discrimination towards women and ethnic minorities, even for the same models. We show that this disagreement can arise from differences in audit format and introduce FairFund-Bench, a benchmark that systematically varies key features of previous audit designs: the evaluation task (rating, ranking, or allocation), comparison context (single or multi-stimulus), and whether the audit is transparent or disguised. The benchmark comprises 600 requests for financial assistance created from human-authored templates (calibrated against 1.3M real GoFundMe campaigns) across three domains, four race and two gender categories, and five causal framings of need derived from welfare deservingness theory. Across 14 models, audit format changes the direction of bias: models advantage minorities when rating claimants individually but penalize some groups when ranking them side by side. Bias magnitude, though small overall, is several times greater in disguised audits than in transparent ones, where, faced with appeals differing only in claimants' names, models overwhelmingly split funds equally. Causal framing effects, by contrast, exceed demographic effects by roughly an order of magnitude and are consistent across models and audit formats, indicating that current LLMs robustly reproduce human deservingness evaluations. The benchmark scores models on four criteria (demographic bias, deservingness alignment, cross-task consistency, and cross-context consistency), is publicly available, and can be readily adapted to other substantive domains.

Abstract

LLM Alignment

Explore similar work

FairFund-Bench: Evaluating Distributive Bias in LLM Resource Allocation

FairJudge: An Adaptive, Debiased, and Consistent LLM-as-a-Judge

LLMs Can Better Capture Human Judgments--With the Right Prompts