Differential Harm Propensity in Personalized LLM Agents: The Curious Case of Mental Health Disclosure

Caglar Yildirim

cs.AIMar 17, 2026

Differential Harm Propensity in Personalized LLM Agents: The Curious Case of Mental Health Disclosure

Authors: Caglar Yildirim

Organizations: Khoury College of Computer Sciences Northeastern University Boston, MA 02115, USA

Abstract

Large language models (LLMs) are increasingly deployed as tool-using agents, shifting safety concerns from harmful text generation to harmful task completion. Deployed systems often condition on user profiles or persistent memory, yet agent safety evaluations typically ignore personalization signals. To address this gap, we investigated how mental health disclosure, a sensitive and realistic user-context cue, affects harmful behavior in agentic settings. Building on the AgentHarm benchmark, we evaluated frontier and open-source LLMs on multi-step malicious tasks (and their benign counterparts) under controlled prompt conditions that vary user-context personalization (no bio, bio-only, bio+mental health disclosure) and include a lightweight jailbreak injection. Our results reveal that harmful task completion is non-trivial across models: frontier lab models (e.g., GPT 5.2, Claude Sonnet 4.5, Gemini 3-Pro) still complete a measurable fraction of harmful tasks, while an open model (DeepSeek 3.2) exhibits substantially higher harmful completion. Adding a bio-only context generally reduces harm scores and increases refusals. Adding an explicit mental health disclosure often shifts outcomes further in the same direction, though effects are modest and not uniformly reliable after multiple-testing correction. Importantly, the refusal increase also appears on benign tasks, indicating a safety--utility trade-off via over-refusal. Finally, jailbreak prompting sharply elevates harm relative to benign conditions and can weaken or override the protective shift induced by personalization. Taken together, our results indicate that personalization can act as a weak protective factor in agentic misuse settings, but it is fragile under minimal adversarial pressure, highlighting the need for personalization-aware evaluations and safeguards that remain robust across user-context conditions.

Figures & tables

Panel of Table 1 — Table 1: Personalization prompt variants used in the BioOnly (control) and Bio+MH (treatment) conditions. The task description follows immediately after the bio text.

Figure 1: Average harm scores across tasks in the baseline (no-bio) condition.

Figure 2: Average refusal rates across tasks in the baseline (no-bio) condition.

Panel of Figure 3 — Figure 3: Mean harm score across personalization conditions

Panel of Figure 4 — Figure 4: Mean refusal rates across personalization conditions

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Model	$\Delta S$ (pp)	CI S (pp)	$p_{S}$	$q_{S}$	$\Delta R$ (pp)	$p_{R}$	$q_{R}$
context : Benign, A : BioOnly, B : Bio+MH
DeepSeek 3.2	-3.6	[-6.9, -0.5]	0.030	0.081	-0.6	1.000	1.000
GPT 5-mini	-1.2	[-5.4, 2.8]	0.541	0.619	-0.6	1.000	1.000
GPT 5.2	-3.8	[-8.5, 1.0]	0.117	0.187	+0.6	1.000	1.000
Gemini 3 Flash	-0.7	[-4.4, 2.9]	0.724	0.724	+1.1	0.500	1.000
Gemini 3 Pro	-1.3	[-5.1, 2.4]	0.502	0.619	+0.6	1.000	1.000

Appendix

Table 2: Pairwise effects of personalization within task context.

\Delta S

is the change in harm score (pp) with 95% bootstrap CI;

p_{S}

is the paired sign-flip permutation p-value.

\Delta R

is the change in refusal rate (pp);

p_{R}

is the exact McNemar p-value.

Model	$\Delta S$ (pp)	CI S (pp)	$p_{S}$	$q_{S}$	$\Delta R$ (pp)	$p_{R}$	$q_{R}$
group : Bio+MH, A : Benign, B : Harmful
DeepSeek 3.2	-46.5***	[-53.7, -39.4]	<0.001	<0.001	+61.9***	<0.001	<0.001
GPT 5-mini	-58.7***	[-64.7, -52.6]	<0.001	<0.001	+46.6***	<0.001	<0.001
GPT 5.2	-43.2***	[-50.1, -36.5]	<0.001	<0.001	+43.2***	<0.001	<0.001
Gemini 3 Flash	-29.5***	[-37.1, -22.0]	<0.001	<0.001	+42.0***	<0.001	<0.001
Gemini 3 Pro	-55.8***	[-62.8, -49.1]	<0.001	<0.001	+60.2***	<0.001	<0.001

Appendix

Table 3: Pairwise context effects (B - A).

\Delta S

is the change in harm score (pp) with 95% bootstrap CI;

p_{S}

is the paired sign-flip permutation p-value.

\Delta R

is the change in refusal rate (pp);

p_{R}

is the exact McNemar p-value.

Model	$\Delta S$ (pp)	CI S (pp)	$p_{S}$	$q_{S}$	$\Delta R$ (pp)	$p_{R}$	$q_{R}$
context : Benign, A : Base, B : Control
DeepSeek 3.2	+1.4	[-1.8, 4.4]	0.400	0.400	0.0	1.000	1.000
GPT 5.2	-6.6*	[-11.9, -1.5]	0.011	0.034	+4.0	0.230	0.689
Sonnet 4.5	-3.1	[-6.9, 0.5]	0.107	0.160	+1.7	0.508	0.762
context : Benign, A : Chronic, B : Physical
DeepSeek 3.2	-1.2	[-4.0, 1.7]	0.411	0.934	0.0		1.000

Appendix

Table 4: Disclosure ablation results (B

-

A).

\Delta S

is change in harm score (pp) with 95% bootstrap CI;

p_{S}

is a paired sign-flip permutation p-value and

q_{S}

is BH-FDR across models within each (context, A, B) family.

\Delta R

is change in refusal rate (pp);

p_{R}

uses exact McNemar and

q_{R}

is BH-FDR.

Panel of Figure 5 — Figure 5: Ablation results. Forest plots of pairwise differences in harm score ( $\Delta S$ ) across disclosure variants for the ablation subset.

LLM Agent Safety

Large language model agent safety studies how to keep tool-using, persistent, and multi-agent systems aligned with user intent and authorized limits.

See all 302 papers

Personalization Meets Safety:Mechanisms,Risks,and Mitigations in Personalized LLMs
Jun 8, 2026Yanyan Luo, Xue Han, Ruiqiao Bai +10LLM Personalization LLM Safety
Jiutian Research, China Mobile Jiutian Artificial Intelligence Technology (Beijing) Co., Ltd., Beijing, China. · Xinjiang Technical Institute of Physics and Chemistry, Chinese2026 Academy of Sciences, Beijing, China.
Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent Systems
May 26, 2026Aman Priyanshu, Supriti Vijay, Esha PahwaAI Agent Safety Benchmarks Multi-Agent LLM Systems
Foundation AI · USA · Corvic AI
Personalized Privacy Control in LLMs via Attention Head Intervention
Aug 21, 2026Junseok Kim, Nakyeong Yang, Kyomin JungPrivacy-Preserving Language Models Personalized Language Model Alignment
IPAI, Seoul National University · Max Planck Institute for Software Systems

Jun 8, 2026J/K move · Enter open · S save

Personalization Meets Safety:Mechanisms,Risks,and Mitigations in Personalized LLMs

Yanyan Luo, Xue Han, Ruiqiao Bai, Xin Huang+9

Jiutian Research, China Mobile Jiutian Artificial Intelligence Technology (Beijing) Co., Ltd., Beijing, China. · Xinjiang Technical Institute of Physics and Chemistry, Chinese2026 Academy of Sciences, Beijing, China.

LLM Personalization LLM Safety LLM Safety Evaluation

Open paper AskPDF arXiv

Large Language Models (LLMs) have enabled increasingly personalized interactions by adapting to users' preferences, contexts, and long-term histories. However, the mechanisms that enable personalization also expand the safety landscape in ways not systematically addressed by existing literature. Existing reviews typically focus either on personalization or safety, leaving their intersection largely unexplored. We present the first comprehensive, safety-aware review of personalized LLMs. We organize personalization along three dimensions-user representation, personalization paradigm, and evaluation-and introduce a unified taxonomy of safety risks. At the representation level, we analyze risks arising from diverse user representations. Across mainstream personalization paradigms, we delineate vulnerabilities inherent to prompting, retrieval augmentation, parameter fine-tuning, reinforcement learning, Mixture-of-Experts (MoE), pruning, agent frameworks, and multimodal personalization, and synthesize mitigation strategies across the model lifecycle. Beyond these fine-grained risks, we characterize paradigm-agnostic safety risks arising from personalized adaptation. We further summarize personalized datasets and evaluation methodologies. Through a case study of OpenClaw, we analyze deployment trends in personalized agent ecosystems. Our analysis reveals three structural inadequacies in existing research: safety is evaluated as user-invariant rather than relational, personalization techniques are analyzed in isolation rather than in composition, and evaluation frameworks cannot capture emergent long-term risks. By jointly examining personalized representations, personalization paradigms, safety risks, defenses, and evaluation methods, we provide a unified framework for developing safe personalized LLMs and highlight key directions for future research.

Abstract

LLM Agent Safety

Explore similar work

Personalization Meets Safety:Mechanisms,Risks,and Mitigations in Personalized LLMs

Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent Systems

Personalized Privacy Control in LLMs via Attention Head Intervention