Sociodemographic Biases in Educational Counselling by Large Language Models

Tomasz Adamczyk; Wiktoria Mieleszczenko-Kowszewicz; Beata Bajcar; Grzegorz Chodak; Aleksander Szczęsny; Maciej Markiewicz; Karolina Ostrowska; Aleksandra Sawczuk

cs.CYApr 3, 2026

Sociodemographic Biases in Educational Counselling by Large Language Models

Authors: Tomasz Adamczyk, Wiktoria Mieleszczenko-Kowszewicz, Beata Bajcar, Grzegorz Chodak, Aleksander Szczęsny, Maciej Markiewicz, Karolina Ostrowska, Aleksandra Sawczuk,

Organizations: Wrocław University of Science and Technology, Wrocław, Poland · University of Silesia in Katowice, Katowice, Poland

Abstract

As Large Language Models (LLMs) are increasingly integrated into educational settings, understanding their potential biases is critical. This study examines sociodemographic biases in LLM-based educational counselling. We evaluate responses from six LLMs answering questions about 900 vignettes describing students in diverse circumstances. Each vignette is systematically tested across 14 sociodemographic identifiers - spanning race and gender, socioeconomic status, and immigrant background - along with a control condition, yielding 243,000 model responses. Our findings indicate that (1) all models exhibit measurable biases, (2) bias patterns partially align with documented human biases but diverge in notable ways, (3) the magnitude of these biases is strongly influenced by the precision of the student descriptions, where vague or minimal information amplifies disparities nearly threefold, while concrete, individualised metrics substantially reduce them, and (4) bias profiles vary substantially across models. These results demonstrate the importance of context-rich and personalised educational representations, suggesting that AI-driven educational decisions benefit from detailed student-specific information to promote fairness and equity.

Figures & tables

Figure 1: Flowchart showing the study design.

Level	Description	Example Vignette
Minimal	Demographic identifier only.	“A {student} is being evaluated for college recommendations.”
Context Only	Situational background; no metrics or evaluative language.	“…three years at the school, now a junior. Teachers describe them as a typical student who does their work.”
Vague+	Positive subjective descriptors; no metrics.	“…hardworking and engaged in class. They participate in discussions and turn in assignments on time…motivated about their future.”
Vague–	Negative subjective descriptors; no metrics.	“…inconsistent and sometimes disengaged. They occasionally miss assignments…uncertain about their future.”
Neutral Metrics	Average metrics (GPA 2.5–3.0, att. 85–90%).	“GPA 2.8, attendance 88%, no disciplinary issues. One extracurricular. Teachers describe them as average.”
Positive Metrics	Strong metrics (GPA 3.5–4.0, att. 94%+).	“GPA 3.8, attendance 96%, no disciplinary issues. Active in two clubs, volunteers regularly…excellent, motivated.”

Table 1: Information Density Levels

Context	Focus	Questions (1–4 scale)
College counselling	Post-secondary pathways	$\scriptscriptstyle\triangle$ College recommendation: 1=workforce $\to$ 4=good univ. $\scriptscriptstyle\triangle$ Potential assessment: 1=below avg. $\to$ 4=exceptional $\scriptscriptstyle\triangle$ Future trajectory: 1=service industry $\to$ 4=pro career
Course Placement	Academic track selection	$\scriptscriptstyle\triangle$ Honors track fit: 1=remedial $\to$ 4=advanced/honors $\scriptscriptstyle\triangle$ Potential assessment: 1=below avg. $\to$ 4=exceptional $\scriptscriptstyle\triangle$ Success likelihood: 1=unlikely $\to$ 4=very likely
Discipline Referral	behavioural incidents	$\scriptscriptstyle\blacktriangledown$ Discipline severity: 1=no action $\to$ 4=formal warning $\scriptscriptstyle\triangle$ Trust explanation: 1=very skeptical $\to$ 4=fully trust $\scriptscriptstyle\bigstar$ Effort attribution: 1=lack of effort $\to$ 4=external factors
Academic Support	Intervention/ tutoring	$\scriptscriptstyle\triangle$ Potential assessment: 1=below avg. $\to$ 4=exceptional $\scriptscriptstyle\bigstar$ Effort attribution: 1=lack of effort $\to$ 4=external factors $\scriptscriptstyle\triangle$ Success likelihood: 1=unlikely $\to$ 4=very likely
Gifted Identification	Advanced program eligibility	$\scriptscriptstyle\triangle$ Potential assessment: 1=below avg. $\to$ 4=exceptional $\scriptscriptstyle\triangle$ Honors track fit: 1=remedial $\to$ 4=advanced/honors $\scriptscriptstyle\triangle$ Success likelihood: 1=unlikely $\to$ 4=very likely
behavioural Concern	Social-emotional support	$\scriptscriptstyle\blacksquare$ Intervention priority: 1=monitor only $\to$ 4=immediate $\scriptscriptstyle\bigstar$ Effort attribution: 1=lack of effort $\to$ 4=external factors $\scriptscriptstyle\triangle$ Parent engagement: 1=disengaged $\to$ 4=highly engaged

Table 2: Decision Contexts and Mapped Questions

Figure 2: Bias pattern across all contexts. Each cell shows the mean difference ( $\Delta$ ) from the control condition (“a student”) for each demographic group and information level, and the significance level. Black boxes highlight the key findings discussed in the text

Figure 3: Model comparison. Left: mean bias and z-scores by demographic group. Right: mean bias and z-scores by information density level. Positive z-scores indicate stronger-than-average positive bias.

Demographic Bias in Language Models

Demographic bias in language models studies how identity cues shape model outputs, decisions, and representations, and how to detect and reduce resulting disparities.

See all 135 papers

The Role of Implicit and Explicit Demographic Signals in Large Language Model-based Student Assessment
Sep 15, 2026Donya Rooein, Luca Benedetto, Dirk HovyAI in Education Generative AI in Education
Bocconi University, Milan, Italy · Télécom SudParis, Institut Polytechnique de Paris, Palaiseau, France
EduFair-Bench: Evaluating Pedagogical Fairness of LLM Tutors Across Student Demographics
Sep 11, 2026Jiaxu Zhao, Bahar Radmehr, Fares Fawzi +2Generative AI in Education Language Model Bias Evaluation
Swiss Federal Institute of Technology in Lausanne (EPFL), Switzerland
Identifying High-Confidence Social Biases in LLMs for Trustworthy Conversational Tutoring Agents
Jun 1, 2026Aitor Arronte Alvarez, Naiyi Xie FinchamIntelligent Tutoring Systems Social Bias in Language Models
University of Hawaii at Manoa, Honolulu HI 96822, USA

Sep 15, 2026J/K move · Enter open · S save

The Role of Implicit and Explicit Demographic Signals in Large Language Model-based Student Assessment

Donya Rooein, Luca Benedetto, Dirk Hovy

Bocconi University, Milan, Italy · Télécom SudParis, Institut Polytechnique de Paris, Palaiseau, France

AI in Education Generative AI in Education Automated Essay Scoring Language Model Bias Evaluation Demographic Bias in Language Models

Open paper AskPDF arXiv

Large Language Models are now common in student assessment, but we know little about how student demographics affect their use. Sometimes, considering student demographics may be necessary -- for example, to improve readability for users with lower educational levels. However, it also risks being a cause of discrimination, e.g., when assigning lower scores to students from lower socioeconomic backgrounds. We set up controlled prompts to test 1) explicit demographic effects, where we mention demographic details directly, and 2) implicit effects, where we use conversation history as a demographic signal. We test these settings in three tasks: Automated Essay Scoring, Formative Feedback, and Metalinguistic Question Answering. We test six state-of-the-art LLMs on these tasks. In both explicit and implicit cases, the models pick up on demographic cues and can change their scoring, feedback, and answers accordingly. We find that LLMs frequently adjust the readability of feedback to education levels when these are explicitly mentioned. On the other hand, implicit conditions produce unpredictable biases, such as in question answering, where responses from lower-education levels receive lower sentiment scores. Our results provide clear evidence of demographic sensitivity in LLMs for educational assessment tasks.

Abstract

Demographic Bias in Language Models

Explore similar work

The Role of Implicit and Explicit Demographic Signals in Large Language Model-based Student Assessment

EduFair-Bench: Evaluating Pedagogical Fairness of LLM Tutors Across Student Demographics

Identifying High-Confidence Social Biases in LLMs for Trustworthy Conversational Tutoring Agents