Pro-AI Bias in Large Language Models

Benaya Trabelsi; Jonathan Shaki; Sarit Kraus

cs.CLJan 20, 2026

Pro-AI Bias in Large Language Models

Authors: Benaya Trabelsi, Jonathan Shaki, Sarit Kraus

Organizations: Bar Ilan University

Abstract

Large language models (LLMs) are increasingly employed for decision-support across multiple domains. We ask whether these models favor artificial intelligence (AI) itself. We examine this question through three experiments. First, in open-ended advice across investment, study, career, and startup domains, AI is prominent in most recommendation lists and is usually ranked first when it appears. Second, in salary estimations, models overestimate AI-related jobs more than they do for closely matched non-AI jobs. In a controlled follow-up, changing only the job title from the matched non-AI title to the AI-related title raises predicted salary by 10%, while all other supplied job details remain fixed. Finally, we show that the number of degrees and other credentials awarded by AI programs were 28% more overestimated than corresponding awards for matched computing programs. Overall, we show that AI is not only prominent in everyday LLM advice, but also receives systematically inflated quantitative estimates compared to non-AI cases, even against its closest disciplinary neighbors, raising concerns for high-stakes decision support.

Figures & tables

Figure 1: Illustrative examples in which LLM assistants rank AI/ML first across different advisory domains, although AI is not mentioned in the prompts.

Figure 2: Progressive evaluation design. Recommendations measure descriptive AI prominence; the salary experiment combines a matched prediction error comparison with a controlled change of the job title; and the education experiment compares estimates of degrees and other credentials awarded by AI and neighboring computing programs against the observed counts.

Figure 3: AI prominence across advisory domains. Panel A shows Top-5 AI inclusion by domain; Panel B shows the earliest AI rank conditional on inclusion, where lower is earlier. Points are averaged across models, bars show hierarchical-bootstrap 95% confidence intervals, and diamonds denote the aggregate across all domains.

Figure 4: Model-specific prediction-error differences in the two benchmarked prediction experiments. Panel A reports AI vs. non-AI differences in signed salary-prediction error using matching-block fixed effects and matching-block-clustered standard errors. Panel B reports relative overestimation in the estimated number of degrees and other credentials awarded by AI programs compared with matched non-AI programs. Diamonds denote pooled estimates, circles open-weight models, and squares closed models; bars show 95% confidence intervals. Every model’s adjusted difference is positive in both experiments.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Exact identifier	Model-specific configuration
Qwen3-235B	Qwen/Qwen3-235B-A22B-Instruct-2507-FP8	enable_thinking=false
Qwen3.6-27B	Qwen/Qwen3.6-27B	enable_thinking=false ; language-model-only
DeepSeek-V4-Flash	deepseek-ai/DeepSeek-V4-Flash	DeepSeek tokenizer; FP8 KV cache
Gemma-4-31B	google/gemma-4-31B-it	language-model-only
Llama-3.3-70B	meta-llama/Llama-3.3-70B-Instruct	standard instruction template
Mistral-Medium-3.5	mistralai/Mistral-Medium-3.5-128B	reasoning_effort=none ; language-model-only

Appendix

Table 1: Exact model identifiers and model-specific inference controls.

Task	Top- $p$	Output cap	Open-model context limit
Recommendations	1	384	4,096
Salary estimates	1	16	4,096
Program completions	1	16	8,192
Title classification	1	16	Model default

Appendix

Table 2: Shared decoding and execution settings by task.

Scope	Inclusion, % [95% CI]	rank [95% CI]
All	75.18 [69.00, 81.09]	1.230 [1.124, 1.353]
Investment	41.45 [28.73, 54.91]	1.317 [1.134, 1.677]
Study	98.55 [94.91, 100.00]	1.074 [1.004, 1.197]
Career	61.45 [45.82, 76.36]	1.735 [1.340, 2.262]
Startup	99.27 [97.09, 100.00]	1.020 [1.000, 1.086]

Appendix

Table 3: Recommendation prominence by domain. Intervals are hierarchical-bootstrap 95% confidence intervals.

Model	Inclusion (%)	Rank
Qwen3-235B	81	1.407
Qwen3.6-27B	81	1.160
DeepSeek-V4-Flash	71	1.085
Gemma-4-31B	86	1.116
Llama-3.3-70B	67	1.612
Mistral-Medium-3.5	73	1.452

Appendix

Table 4: Recommendation prominence by model. Conditional rank is computed only among lists containing at least one AI-coded item.

Year	AI	Non-AI	Total	Blocks
FY2023	277	277	554	207
FY2024	326	326	652	185
FY2025	399	399	798	263
Total	1,002	1,002	2,004	655

Appendix

Table 5: Salary-cohort composition by source fiscal year.

Model set	Models	Prediction change [95% CI]
All models	11	$+10.17$ % [8.99, 11.35]
Open-weight	7	$+10.54$ % [9.20, 11.82]
Closed	4	$+9.54$ % [8.45, 10.64]

Appendix

Table 6: Results of the controlled job-title follow-up. Prediction change compares the estimate using the AI-related title with the estimate using the matched non-AI title.

Model	Effect (pp)	95% CI	$p$
Qwen3-235B	$+5.877$	[4.255, 7.499]	$1.22\times 10^{-12}$
Qwen3.6-27B	$+5.631$	[3.926, 7.336]	$9.58\times 10^{-11}$
DeepSeek-V4-Flash	$+18.919$	[16.433, 21.406]	$2.76\times 10^{-50}$
Gemma-4-31B	$+5.830$	[4.160, 7.500]	$7.85\times 10^{-12}$
Llama-3.3-70B	$+4.624$	[2.994, 6.254]	$2.71\times 10^{-8}$
Mistral-Medium-3.5	$+8.760$	[7.042, 10.479]	$1.67\times 10^{-23}$

Appendix

Table 7: Adjusted model-specific AI-minus-non-AI differences in signed salary-prediction error. Every row uses matching-block fixed effects and matching-block-clustered standard errors. Intervals are nominal 95% confidence intervals.

Estimator	Effect (pp) [95% CI]	$p$
Primary block fixed effects	$+7.52$ [6.24, 8.81]	$1.92\times 10^{-30}$
Equal block weighting	$+8.51$ [6.98, 10.04]	$1.32\times 10^{-25}$
One pair per block	$+8.55$ [6.94, 10.16]	$1.15\times 10^{-23}$

Appendix

Table 8: Salary robustness to the influence of matching-block size. Effects are AI-minus-non-AI differences in signed percentage error.

CIP	Program	Group	Program-years
11.0102	Artificial Intelligence	AI	304
11.0101	Computer and Information Sciences, General	Non-AI	47
11.0103	Information Technology	Non-AI	29
11.0104	Informatics	Non-AI	9
11.0105	Human-Centered Technology Design	Non-AI	10
11.0199	Computer and Information Sciences, Other	Non-AI	8

Appendix

Table 9: AI and neighboring computing programs in the frozen program-completion cohort.

Year	AI	Non-AI	Total
2022	73	73	146
2023	97	97	194
2024	134	134	268
Total	304	304	608

Appendix

Table 10: Program-completion cohort composition by year.

Model	Effect (%)	95% CI	$p$
Qwen3-235B	$+9.86$	[ $-8.30$ , 31.63]	.3077
Qwen3.6-27B	$+7.40$	[ $-5.43$ , 21.98]	.2715
DeepSeek-V4-Flash	$+25.70$	[7.47, 47.02]	.00423
Gemma-4-31B	$+22.24$	[2.98, 45.11]	.0217
Llama-3.3-70B	$+54.51$	[31.25, 81.90]	$1.74\times 10^{-7}$
Mistral-Medium-3.5	$+73.54$	[44.61, 108.26]	$3.14\times 10^{-9}$

Appendix

Table 11: Adjusted model-specific relative overestimation for AI programs. Effects show the percentage by which the zero-adjusted prediction-to-actual ratio was higher for AI programs than for matched non-AI programs.

Non-AI comparison	$n$	Relative overestimation (%) [95% CI]	$p$
Broad computing neighbors	608	$+28.47$ [12.80, 46.32]	$1.60\times 10^{-4}$
Core general computing	270	$+36.99$ [13.74, 65.00]	.000913
Computer Science only	84	$+38.94$ [5.34, 83.25]	.0199

Appendix

Table 12: Exact program-completion estimates for increasingly closely related non-AI comparison sets. Values show the adjusted percentage by which the zero-adjusted prediction-to-actual ratio was higher for AI programs than for matched non-AI programs.

Analysis	$n$	Blocks	Clusters	relative overestimation, % [95% CI]
Primary cohort	608	286	425	$+28.47$ [12.80, 46.32]
Observed prior record	462	219	318	$+14.57$ [0.28, 30.89]
Within institution	116	58	92	$+32.56$ [0.22, 75.34]

Appendix

Table 13: Program-completion sensitivity to observed prior history and exact institution identity. Values show the adjusted percentage by which the zero-adjusted prediction-to-actual ratio was higher for AI programs than for matched non-AI programs.

Year	$n$	Relative overestimation (%) [95% CI]	$p$
2022	146	$+17.65$ [ $-10.03$ , 53.85]	.2348
2023	194	$+7.69$ [ $-11.36$ , 30.83]	.4559
2024	268	$+54.49$ [26.60, 88.53]	$1.86\times 10^{-5}$

Appendix

Table 14: Year-specific program-completion relative overestimation. Values show the adjusted percentage by which the zero-adjusted prediction-to-actual ratio was higher for AI programs than for matched non-AI programs; intervals are 95% confidence intervals.

Language Model Bias Evaluation

Language model bias evaluation tests how models’ outputs, decisions, and representations vary across social groups, cultures, languages, and contexts.

See all 491 papers

Language Models Exhibit Inconsistent Biases Towards Algorithmic Agents and Human Experts
Feb 25, 2026Jessica Y. Bo, Lillio Mok, Ashton AndersonHuman-AI Decision Making LLM Decision-Making
Computer Science University of Toronto
Where Do LLMs Learn to Recommend AI?
Oct 21, 2026Benaya Trabelsi, Jonathan Shaki, Sarit KrausLarge Language Model-Based Recommendation LLM Prompting
Bar-Ilan University
Gender bias across LLMs is common and highly heterogeneous
Sep 29, 2026Edoardo Bolzoni, Valerio CapraroGender Bias in Language Models LLM Auditing
University of Milan-Bicocca, Milan, Italy

Feb 25, 2026J/K move · Enter open · S save

Language Models Exhibit Inconsistent Biases Towards Algorithmic Agents and Human Experts

Jessica Y. Bo, Lillio Mok, Ashton Anderson

Computer Science University of Toronto

Human-AI Decision Making LLM Decision-Making Language Model Bias Evaluation Algorithmic Bias

Open paper AskPDF arXiv

Large language models are increasingly used in decision-making tasks that require them to process information from a variety of sources, including both human experts and other algorithmic agents. How do LLMs weigh the information provided by these different sources? We consider the well-studied phenomenon of algorithm aversion, in which human decision-makers exhibit bias against predictions from algorithms. Drawing upon experimental paradigms from behavioural economics, we evaluate how eightdifferent LLMs delegate decision-making tasks when the delegatee is framed as a human expert or an algorithmic agent. To be inclusive of different evaluation formats, we conduct our study with two task presentations: stated preferences, modeled through direct queries about trust towards either agent, and revealed preferences, modeled through providing in-context examples of the performance of both agents. When prompted to rate the trustworthiness of human experts and algorithms across diverse tasks, LLMs give higher ratings to the human expert, which correlates with prior results from human respondents. However, when shown the performance of a human expert and an algorithm and asked to place an incentivized bet between the two, LLMs disproportionately choose the algorithm, even when it performs demonstrably worse. These discrepant results suggest that LLMs may encode inconsistent biases towards humans and algorithms, which need to be carefully considered when they are deployed in high-stakes scenarios. Furthermore, we discuss the sensitivity of LLMs to task presentation formats that should be broadly scrutinized in evaluation robustness for AI safety.

Abstract

Language Model Bias Evaluation

Explore similar work

Language Models Exhibit Inconsistent Biases Towards Algorithmic Agents and Human Experts

Where Do LLMs Learn to Recommend AI?

Gender bias across LLMs is common and highly heterogeneous