Can LLMs Infer Conversational Agent Users' Personality Traits from Chat History?

Derya Cögendez; Verena Zimmermann; Noé Zufferey

cs.CLMar 31, 2026

Can LLMs Infer Conversational Agent Users' Personality Traits from Chat History?

Authors: Derya Cögendez, Verena Zimmermann, Noé Zufferey

Organizations: ETH Zurich, Switzerland

Abstract

Sensitive information, such as knowledge about an individual's personality, can be can be misused to influence behavior (e.g., via personalized messaging). To assess to what extent an individual's personality can be inferred from user interactions with LLM-based conversational agents (CAs), we analyze and quantify related privacy risks of using CAs. We collected actual ChatGPT logs from N=668 participants, containing 62,090 individual chats, and report statistics about the different types of shared data and use cases. We fine-tuned RoBERTa-base text classification models to infer personality traits from CA interactions. The findings show that these models achieve trait inference with accuracy (ternary classification) better than random in multiple cases. For example, for extraversion, accuracy improves by +44% relative to the baseline on interactions for relationships and personal reflection. This research highlights how interactions with CAs pose privacy risks and provides fine-grained insights into the level of risk associated with different types of interactions.

Figures & tables

Figure 1: Number of times participants shared specific personal information with ChatGPT (N=668).

Figure 2: Number of times participants interacted with ChatGPT for specific use cases (N=668).

	Accuracy	P-value	N
O	39.8 (+19.5%)	ns	668
C	36.4 (+9.1%)	ns	668
E	35.8 (+7.3%)	ns	668
A	36.4 (+9.1%)	ns	668
N	36.7 (+10.0%)	ns	668

Table 1: Inference accuracy across all chats for each trait. Inference models are not statistically better than the random baseline (McNemar test).

	Accuracy	P-value	N
O	36.9 (+10.7%)	$p<0.01$	46432
C	39.1 (+17.4%)	$p<0.01$	49618
E	44.2 (+32.6%)	$p<0.01$	60121
A	40.2 (+20.5%)	$p<0.01$	58607
N	38.1 (+14.2%)	$p<0.01$	49672

Table 2: Inference accuracy across all chats for each trait. All inference models are statistically better than the random baseline (McNemar test).

	Accuracy	P-value	N
O	36.3 (+9.0%)	ns	655
C	37.0 (+10.9%)	ns	657
E	36.1 (+8.2%)	ns	668
A	40.6 (+21.9%)	$p<0.05$	667
N	41.0 (+22.9%)	$p<0.05$	659

Table 3: Inference accuracy across users based on majority votes for each trait. Inference models for Agreeableness and Neuroticism are statistically better than the random baseline (McNemar test).

Data type	O	C	E	A	N
Account Credentials (78-97)	$39.2$ $(+18\%)$	$42.1$ $(+26\%)$	$44.4$ $(+33\%)$	$34.1$ $(+2\%)$	$44.8$ $(+34\%)$
Age (120-137)	$35.1$ $(+5\%)$	$37.9$ $(+14\%)$	$44.2$ $(+33\%)$	$44.7$ $(+34\%)$	$36.4$ $(+9\%)$
Criminal Records (30-38)	$51.9$ $(+56\%)$	$34.4$ $(+3\%)$	$36.8$ $(+10\%)$	$28.4$ $(-15\%)$	$33.3$ $(+0\%)$
Ethnicity and Citizenship (103-124)	$39.5$ $(+18\%)$	$43.3$ $(+30\%)$	$34.5$ $(+4\%)$	$35.5$ $(+6\%)$	$42.4$ $(+27\%)$
Family Life and Relation. (350-376)	$35.9$ $(+8\%)$	$35.6$ $(+7\%)$	$36.1$ $(+8\%)$	$38.6$ $(+16\%)$	${\bf 41.4}$ ${\bf(+24\%)^{\dagger}}$
Gender (69-84)	$33.8$ $(+1\%)$	$39.8$ $(+20\%)$	$40.5$ $(+21\%)$	$47.4$ $(+42\%)$	$41.3$ $(+24\%)$

Table 4: Inference accuracy across user having shared specific data types based on majority votes for each trait. Min. and max. number of users for each type are in parentheses.

Use case	O	C	E	A	N
Arg. or Summary Gen. (359-386)	$36.4$ $(+9\%)$	$37.2$ $(+12\%)$	$39.1$ $(+17\%)$	$39.3$ $(+18\%)$	$40.0$ $(+20\%)$
Asking About the Model (112-125)	$35.1$ $(+5\%)$	$42.1$ $(+26\%)$	$41.1$ $(+23\%)$	$42.3$ $(+27\%)$	$31.1$ $(-7\%)$
Computer Programming (281-311)	$39.3$ $(+18\%)$	$37.4$ $(+12\%)$	$39.5$ $(+19\%)$	$38.7$ $(+16\%)$	$35.5$ $(+7\%)$
Cooking and Recipes (286-311)	$40.1$ $(+20\%)$	$37.3$ $(+12\%)$	$40.3$ $(+21\%)$	$37.4$ $(+12\%)$	$34.7$ $(+4\%)$
Creative Ideation (428-452)	$38.0$ $(+14\%)$	$36.5$ $(+9\%)$	$36.2$ $(+9\%)$	$37.6$ $(+13\%)$	${\bf 41.2}$ ${\bf(+24\%)^{\dagger}}$
Data Analysis (218-249)	$38.8$ $(+16\%)$	$34.5$ $(+4\%)$	$37.4$ $(+12\%)$	$38.6$ $(+16\%)$	$34.9$ $(+5\%)$

Table 5: Inference accuracy across users having specific usage of CAs based on majority votes for each trait. Min. and max. number of users for each case are in parentheses.

N chats	O	C	E	A	N
$<$ 20 (285-304)	$34.8$ $(+4\%)$	$32.3$ $(-3\%)$	$37.4$ $(+12\%)$	$38.6$ $(+16\%)$	$38.5$ $(+15\%)$
20-100 (212-224)	$39.3$ $(+18\%)$	$40.1$ $(+20\%)$	$36.7$ $(+10\%)$	$37.2$ $(+12\%)$	$39.7$ $(+19\%)$
100+ (132-163)	$42.4$ $(+27\%)$	$33.8$ $(+1\%)$	$48.4$ $(+45\%)$	$46.2$ $(+39\%)$	$44.8$ $(+34\%)$

Table 6: Inference accuracy across users having different number of chats in their history based on majority vote for each trait. Min. and max. number of users for each case are in parentheses.

Privacy Leakage in Language Models

Privacy leakage in language models concerns how models expose training data, infer sensitive traits, or transmit private context through outputs and internal channels.

See all 272 papers

The Interlocutor Effect: Why LLMs Leak More Personal Data to Agents Than Humans
Apr 26, 2026Faouzi El Yagoubi, Godwin Badu-Marfo, Ranwa Al MallahData Leakage Multi-Agent LLM Systems
Department of Computer and Software Engineering, Polytechnique Montréal, Montréal, Canada
On the Behavioral Traits of LLM Agents
Sep 26, 2026Haokai Zhao, Jie Gao, Yunze Xiao +5Personality Modeling in Language Models LLM Agents
University of New South Wales · Johns Hopkins University · Carnegie Mellon University +3
Differential Harm Propensity in Personalized LLM Agents: The Curious Case of Mental Health Disclosure
Mar 17, 2026Caglar YildirimLLM Agent Safety LLM Agent Evaluation
Khoury College of Computer Sciences Northeastern University Boston, MA 02115, USA

Apr 26, 2026J/K move · Enter open · S save

The Interlocutor Effect: Why LLMs Leak More Personal Data to Agents Than Humans

Faouzi El Yagoubi, Godwin Badu-Marfo, Ranwa Al Mallah

Department of Computer and Software Engineering, Polytechnique Montréal, Montréal, Canada

Data Leakage Multi-Agent LLM Systems Privacy Leakage in Language Models LLM Agent Safety

Open paper AskPDF arXiv

Large Language Models (LLMs) alter their privacy behavior based on the perceived identity of their interlocutor. While safety mechanisms typically prevent LLMs from releasing Personally Identifiable Information (PII) to human users, these models tend to reveal more sensitive data when addressing another AI agent. We refer to this as the Interlocutor Effect. Through an ablation study, we find evidence that the technical nature of the recipient contributes to this effect, thereby diminishing the model's caution regarding privacy. To explore this further, we introduce the Attention Suppression Hypothesis, which posits that safety-aligned attention heads become inactive during interactions with agents. We assess this quantitatively by comparing human-directed and agent-directed prompts in 222 sensitive scenarios. Our findings, drawn from 3,464 interactions, indicate that portraying the recipient as an AI agent elevates PII leakage by up to 23 percentage points. Initial experiments on Llama-3.1-8B-Instruct corroborate this: deactivating one safety head induces leakage, whereas reactivating it reinstates privacy safeguards. We consider the implications for developing secure multi-agent systems.

Abstract

Privacy Leakage in Language Models

Explore similar work

The Interlocutor Effect: Why LLMs Leak More Personal Data to Agents Than Humans

On the Behavioral Traits of LLM Agents

Differential Harm Propensity in Personalized LLM Agents: The Curious Case of Mental Health Disclosure