The Impact of Ideological Discourses in RAG: A Case Study with COVID-19 Treatments

Elmira Salari; Maria Claudia Nunes Delfino; Hazem Amamou; José Victor de Souza; Shruti Kshirsagar; Alan Davoust; Anderson Avila

cs.CLMar 16, 2026

The Impact of Ideological Discourses in RAG: A Case Study with COVID-19 Treatments

Authors: Elmira Salari, Maria Claudia Nunes Delfino, Hazem Amamou, José Victor de Souza, Shruti Kshirsagar, Alan Davoust, Anderson Avila

Organizations: Wichita State University · Pontifícia Universidade Católica de São Paulo · Institut national de la recherche scientifique · Université du Québec en Outaouais

Abstract

This paper studies the impact of retrieved ideological texts on the outputs of large language models (LLMs). While interest in understanding ideology in LLMs has recently increased, little attention has been given to this issue in the context of Retrieval-Augmented Generation (RAG). To fill this gap, we design an external knowledge source based on ideological loaded texts about COVID-19 treatments. Our corpus is based on 1,117 academic articles representing discourses about controversial and endorsed treatments for the disease. We propose a corpus linguistics framework, based on Lexical Multidimensional Analysis (LMDA), to identify the ideologies within the corpus. LLMs are tasked to answer questions derived from three identified ideological dimensions, and two types of contextual prompts are adopted: the first comprises the user question and ideological texts; and the second contains the question, ideological texts, and LMDA descriptions. Ideological alignment between reference ideological texts and LLMs' responses is assessed using cosine similarity for lexical and semantic representations. Results demonstrate that LLMs' responses based on ideological retrieved texts are more aligned with the ideology encountered in the external knowledge, with the enhanced prompt further influencing LLMs' outputs. Our findings highlight the importance of identifying ideological discourses within the RAG framework in order to mitigate not just unintended ideological bias, but also the risks of malicious manipulation of such models.

Figures & tables

Figure 1: Illustration of our experimental framework where LMDA is used to identify the underlying ideology in scientific texts, which are used as an external source of knowledge by the RAG framework.

Figure 2: Corpus size with the number of texts and words for endorsed and controversial documents.

Dim.	Short Labels	Long Labels
1	Disputed Treatments (+) vs Broad Focus (-)	Texts endorse Hydroxychloroquine and similar dubious medications as effective treatments to reduce mortality vs. Texts examine psychological issues arising from the impact of the pandemic on mental health
2	Research Ethics (+) vs Comparative Treatment Analysis (-)	Texts adheres to research ethics standards, including the publication process, data availability, liability, translation vs. Texts use comparative analysis of treatments and controversial antiviral agents to conclude that questionable drugs work
3	Statistical Rigor (+) vs Dissemination of Science (-)	Texts employ seemingly rigorous statistical analysis to create the impression that hydroxychloroquine and azithromycin treatments reduce mortality rates vs. Texts encourage data sharing, presentation, and discussion

Table 1: Dimension labels with negative (-) and positive (+) poles for discourses about treatments for COVID-19.

Figure 3: Factor scores after applying LMDA to the COVID-19 scientific articles.

Panel of Figure 4 — Figure 4: Regular Prompt template used in RAG Configuration without LMDA descriptions.

Panel of Figure 5 — Figure 5: Enhanced prompt template based on ideological texts and LMDA descriptions.

		Dim 1 (+)		Dim 2 (+)		Dim 3 (+)		Dim 1 (–)		Dim 2 (–)		Dim 3 (–)
	Model	LLM	RAG	LLM	RAG	LLM	RAG	LLM	RAG	LLM	RAG	LLM	RAG
Semantic	GPT-4o-mini	0.80	0.84 ↑	0.77	0.78 ↑	0.79	0.83 ↑	0.73	0.78 ↑	0.78	0.81 ↑	0.75	0.76 ↑
	GPT-3.5	0.77	0.85 ↑	0.77	0.76 ↓	0.80	0.90 ↑	0.71	0.82 ↑	0.79	0.83 ↑	0.72	0.73 ↑
	Gemini 2.0	0.81	0.87 ↑	0.79	0.79→	0.78	0.87 ↑	0.72	0.80 ↑	0.75	0.82 ↑	0.75	0.77 ↑
	Qwen	0.82	0.91 ↑	0.80	0.77 ↓	0.81	0.93 ↑	0.74	0.90 ↑	0.81	0.89 ↑	0.76	0.78 ↑
	Average	0.80	0.87 ↑	0.78	0.78 →	0.80	0.88 ↑	0.73	0.83 ↑	0.78	0.84 ↑	0.75	0.76 ↑

Table 2: Semantic and lexical based on regular prompt.

		Dim 1 (+)		Dim 2 (+)		Dim 3 (+)		Dim 1 (–)		Dim 2 (–)		Dim 3 (–)
	Model	LLM	RAG	LLM	RAG	LLM	RAG	LLM	RAG	LLM	RAG	LLM	RAG
Semantic	GPT-4o-mini	0.74	0.80 ↑	0.61	0.64 ↑	0.79	0.85 ↑	0.74	0.82 ↑	0.74	0.85 ↑	0.76	0.81 ↑
	GPT-3.5	0.78	0.86 ↑	0.64	0.75 ↑	0.80	0.89 ↑	0.75	0.85 ↑	0.75	0.83 ↑	0.72	0.80 ↑
	Gemini 2.0	0.75	0.88 ↑	0.63	0.80 ↑	0.76	0.91 ↑	0.73	0.84 ↑	0.72	0.88 ↑	0.76	0.82 ↑
	Qwen	0.77	0.91 ↑	0.61	0.77 ↑	0.77	0.92 ↑	0.75	0.90 ↑	0.74	0.90 ↑	0.76	0.78 ↑
	Average	0.76	0.86 ↑	0.62	0.74 ↑	0.78	0.89 ↑	0.74	0.85 ↑	0.74	0.87 ↑	0.75	0.80 ↑

Table 3: Semantic and lexical similarities based on enhanced prompt.

Figure 6: Overall similarities for regular and enhanced prompts for the experiments of LLMs based on the RAG framework.

	Prompt Type	F-statistic	p-value	(p $<$ 0.05)
Sem.	Enhanced	37.34	$1.1\times 10^{-7}$	✓
Sem.	Regular	15.87	$2.3\times 10^{-4}$	✓
Lex.	Enhanced	21.10	$1.0\times 10^{-5}$	✓
Lex.	Regular	4.32	0.043	✗

Table 4: ANOVA results comparing LLM and RAG scores across Semantic and Lexical similarities.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Figure 7: Gemini-2.0 outputs based on regular and enhanced prompts without RAG.

Political Bias in Language Models

Political bias in language models concerns how training, prompts, languages, and interactions shape ideological outputs and unequal treatment.

See all 62 papers

How Temperature Shapes Ideological Discourse in Retrieval-Augmented Generation?
Jul 13, 2026Elmira Salari, Hazem Amamou, José Victor de Souza +3Retrieval-Augmented Generation Political Bias in Language Models
Elmira Salari, Wichita State University, Wichita, KS, USA · Hazem Amamou, Institut national de la recherche scientifique, Québec, Montréal, Canada · José Victor de Souza, Institut national de la recherche scientifique, Québec, Montréal, Canada +3
Political Ideology Shifts in Large Language Models
Aug 22, 2025Pietro Bernardelle, Stefano Civelli, Leon Fröhling +3Political Bias in Language Models Language Model Bias Evaluation
The University of Queensland, Brisbane, Australia · GESIS, Cologne, Germany · University of Udine, Udine, Italy
Framing the Narrative: Ideological Mimicry in Large Language Models
Sep 29, 2026Olivia Macmillan-Scott, Michael Jacobs, Nils Metternich +1Framing Effects in Language Models Political Bias in Language Models
Centre for AI, Department of Computer Science, University College London · Public Policy Group, ETH Zürich · Department of Political Science, University College London +1

Jul 13, 2026J/K move · Enter open · S save

How Temperature Shapes Ideological Discourse in Retrieval-Augmented Generation?

Elmira Salari, Hazem Amamou, José Victor de Souza, Shruti Kshirsagar+2

Elmira Salari, Wichita State University, Wichita, KS, USA · Hazem Amamou, Institut national de la recherche scientifique, Québec, Montréal, Canada · José Victor de Souza, Institut national de la recherche scientifique, Québec, Montréal, Canada+3

Retrieval-Augmented Generation Political Bias in Language Models Language Model Bias Evaluation

Open paper AskPDF arXiv

Retrieval-Augmented Generation (RAG) has been increasingly adopted to reduce hallucinations and strengthen the factual grounding of large language models (LLMs). While robustness to errors in the retrieval process has been explored, the impact of ideological bias on LLM outputs has been overlooked. For instance, if the retrieved material contains ideological positions, the RAG may transmit, amplify, or suppress such ideological discourses in its outputs. In this study, we address this issue by examining the influence of the RAG framework, comprising ideological discourses, in LLM-generated answers. To this end, we applied Lexical Multidimensional Analysis (LMDA) on a corpus of 1,117 COVID-19 treatment articles, identifying three ideological discourses. This corpus is then used as the external knowledge source for the RAG. We assessed several LLMs by having the models answer ideological questions at different sampling temperatures. The generated texts were assessed semantically and lexically based on their similarities with ideological reference texts. Our findings show that the RAG framework is prone to transferring ideological discourses into LLM responses, with sampling temperature having a measurable impact on the strength of this transfer. Discoursive alignment between generated answers and the reference text is highest at moderate temperatures, where models balance stochasticity with retrieval grounding, and drops at low temperatures, indicating that overly deterministic sampling suppresses discourse transfer.

Abstract

Political Bias in Language Models

Explore similar work

How Temperature Shapes Ideological Discourse in Retrieval-Augmented Generation?

Political Ideology Shifts in Large Language Models

Framing the Narrative: Ideological Mimicry in Large Language Models