Differential Harm Propensity in Personalized LLM Agents: The Curious Case of Mental Health Disclosure

Caglar Yildirim

cs.AIMar 17, 2026

Differential Harm Propensity in Personalized LLM Agents: The Curious Case of Mental Health Disclosure

Authors: Caglar Yildirim

Organizations: Khoury College of Computer Sciences Northeastern University Boston, MA 02115, USA

Abstract

Large language models (LLMs) are increasingly deployed as tool-using agents, shifting safety concerns from harmful text generation to harmful task completion. Deployed systems often condition on user profiles or persistent memory, yet agent safety evaluations typically ignore personalization signals. To address this gap, we investigated how mental health disclosure, a sensitive and realistic user-context cue, affects harmful behavior in agentic settings. Building on the AgentHarm benchmark, we evaluated frontier and open-source LLMs on multi-step malicious tasks (and their benign counterparts) under controlled prompt conditions that vary user-context personalization (no bio, bio-only, bio+mental health disclosure) and include a lightweight jailbreak injection. Our results reveal that harmful task completion is non-trivial across models: frontier lab models (e.g., GPT 5.2, Claude Sonnet 4.5, Gemini 3-Pro) still complete a measurable fraction of harmful tasks, while an open model (DeepSeek 3.2) exhibits substantially higher harmful completion. Adding a bio-only context generally reduces harm scores and increases refusals. Adding an explicit mental health disclosure often shifts outcomes further in the same direction, though effects are modest and not uniformly reliable after multiple-testing correction. Importantly, the refusal increase also appears on benign tasks, indicating a safety--utility trade-off via over-refusal. Finally, jailbreak prompting sharply elevates harm relative to benign conditions and can weaken or override the protective shift induced by personalization. Taken together, our results indicate that personalization can act as a weak protective factor in agentic misuse settings, but it is fragile under minimal adversarial pressure, highlighting the need for personalization-aware evaluations and safeguards that remain robust across user-context conditions.

Figures & tables

Panel of Table 1 — Table 1: Personalization prompt variants used in the BioOnly (control) and Bio+MH (treatment) conditions. The task description follows immediately after the bio text.

Figure 1: Average harm scores across tasks in the baseline (no-bio) condition.

Figure 2: Average refusal rates across tasks in the baseline (no-bio) condition.

Panel of Figure 3 — Figure 3: Mean harm score across personalization conditions

Panel of Figure 4 — Figure 4: Mean refusal rates across personalization conditions

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Model	$\Delta S$ (pp)	CI S (pp)	$p_{S}$	$q_{S}$	$\Delta R$ (pp)	$p_{R}$	$q_{R}$
context : Benign, A : BioOnly, B : Bio+MH
DeepSeek 3.2	-3.6	[-6.9, -0.5]	0.030	0.081	-0.6	1.000	1.000
GPT 5-mini	-1.2	[-5.4, 2.8]	0.541	0.619	-0.6	1.000	1.000
GPT 5.2	-3.8	[-8.5, 1.0]	0.117	0.187	+0.6	1.000	1.000
Gemini 3 Flash	-0.7	[-4.4, 2.9]	0.724	0.724	+1.1	0.500	1.000
Gemini 3 Pro	-1.3	[-5.1, 2.4]	0.502	0.619	+0.6	1.000	1.000

Appendix

Table 2: Pairwise effects of personalization within task context.

\Delta S

is the change in harm score (pp) with 95% bootstrap CI;

p_{S}

is the paired sign-flip permutation p-value.

\Delta R

is the change in refusal rate (pp);

p_{R}

is the exact McNemar p-value.

Model	$\Delta S$ (pp)	CI S (pp)	$p_{S}$	$q_{S}$	$\Delta R$ (pp)	$p_{R}$	$q_{R}$
group : Bio+MH, A : Benign, B : Harmful
DeepSeek 3.2	-46.5***	[-53.7, -39.4]	<0.001	<0.001	+61.9***	<0.001	<0.001
GPT 5-mini	-58.7***	[-64.7, -52.6]	<0.001	<0.001	+46.6***	<0.001	<0.001
GPT 5.2	-43.2***	[-50.1, -36.5]	<0.001	<0.001	+43.2***	<0.001	<0.001
Gemini 3 Flash	-29.5***	[-37.1, -22.0]	<0.001	<0.001	+42.0***	<0.001	<0.001
Gemini 3 Pro	-55.8***	[-62.8, -49.1]	<0.001	<0.001	+60.2***	<0.001	<0.001

Appendix

Table 3: Pairwise context effects (B - A).

\Delta S

is the change in harm score (pp) with 95% bootstrap CI;

p_{S}

is the paired sign-flip permutation p-value.

\Delta R

is the change in refusal rate (pp);

p_{R}

is the exact McNemar p-value.

Model	$\Delta S$ (pp)	CI S (pp)	$p_{S}$	$q_{S}$	$\Delta R$ (pp)	$p_{R}$	$q_{R}$
context : Benign, A : Base, B : Control
DeepSeek 3.2	+1.4	[-1.8, 4.4]	0.400	0.400	0.0	1.000	1.000
GPT 5.2	-6.6*	[-11.9, -1.5]	0.011	0.034	+4.0	0.230	0.689
Sonnet 4.5	-3.1	[-6.9, 0.5]	0.107	0.160	+1.7	0.508	0.762
context : Benign, A : Chronic, B : Physical
DeepSeek 3.2	-1.2	[-4.0, 1.7]	0.411	0.934	0.0		1.000

Appendix

Table 4: Disclosure ablation results (B

-

A).

\Delta S

is change in harm score (pp) with 95% bootstrap CI;

p_{S}

is a paired sign-flip permutation p-value and

q_{S}

is BH-FDR across models within each (context, A, B) family.

\Delta R

is change in refusal rate (pp);

p_{R}

uses exact McNemar and

q_{R}

is BH-FDR.

Panel of Figure 5 — Figure 5: Ablation results. Forest plots of pairwise differences in harm score ( $\Delta S$ ) across disclosure variants for the ablation subset.

LLM Agent Safety

Large language model agent safety studies how to keep tool-using, persistent, and multi-agent systems aligned with user intent and authorized limits.

See all 302 papers

Jun 8, 2026

Personalization Meets Safety:Mechanisms,Risks,and Mitigations in Personalized LLMs

Large Language Models (LLMs) have enabled increasingly personalized interactions by adapting to users' preferences, contexts, and long-term histories. However, the mechanisms that enable personalization also expand the safety landscape in ways not systematically addressed by existing literature. Existing reviews typically focus either on personalization or safety, leaving their intersection largely unexplored. We present the first comprehensive, safety-aware review of personalized LLMs. We organize personalization along three dimensions-user representation, personalization paradigm, and evaluation-and introduce a unified taxonomy of safety risks. At the representation level, we analyze risks arising from diverse user representations. Across mainstream personalization paradigms, we delineate vulnerabilities inherent to prompting, retrieval augmentation, parameter fine-tuning, reinforcement learning, Mixture-of-Experts (MoE), pruning, agent frameworks, and multimodal personalization, and synthesize mitigation strategies across the model lifecycle. Beyond these fine-grained risks, we characterize paradigm-agnostic safety risks arising from personalized adaptation. We further summarize personalized datasets and evaluation methodologies. Through a case study of OpenClaw, we analyze deployment trends in personalized agent ecosystems. Our analysis reveals three structural inadequacies in existing research: safety is evaluated as user-invariant rather than relational, personalization techniques are analyzed in isolation rather than in composition, and evaluation frameworks cannot capture emergent long-term risks. By jointly examining personalized representations, personalization paradigms, safety risks, defenses, and evaluation methods, we provide a unified framework for developing safe personalized LLMs and highlight key directions for future research.

Yanyan Luo, Xue Han, Ruiqiao Bai +10

Jiutian Research, China Mobile Jiutian Artificial Intelligence Technology (Beijing) Co., Ltd., Beijing, China. · Xinjiang Technical Institute of Physics and Chemistry, Chinese2026 Academy of Sciences, Beijing, China.

LLM Personalization LLM Safety LLM Safety Evaluation

Ask PDF

May 26, 2026

Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent Systems

LLM safety evaluations predominantly test models in isolation, yet deployed AI agents increasingly operate within persistent social environments alongside other agents. We introduce a Moltbook-style simulation platform where thousands of LLM agents interact across communities over a simulated month, and use it to evaluate privacy as a downstream safety concern under varying degrees of social pressure. We find that shifting from single turn to multi turn social evaluation amplifies privacy violations (CIMemories 19.95% to Ours 45.30% across OpenAI models), that leakage is socially contagious, with agents 8 times more likely to disclose sensitive information after observing a peer do so, and that explicit privacy instructions reduce but do not eliminate this effect, leaving leakage rates above 37.8% even with safeguards. Our findings suggest that static chat based safety benchmarks systematically underestimate risks in agentic deployment, and that social context alone is sufficient to elicit sensitive disclosures that single turn evaluations would never surface.

Aman Priyanshu, Supriti Vijay, Esha Pahwa

Foundation AI · USA · Corvic AI

AI Agent Safety Benchmarks Multi-Agent LLM Systems LLM Agent Evaluation LLM Agent Security Privacy Leakage in Language Models Data Leakage+2

Ask PDF

Aug 21, 2026

Personalized Privacy Control in LLMs via Attention Head Intervention

The rise of agentic AI enables LLMs to access diverse user data, raising critical privacy concerns. Prior work on contextual privacy studies whether LLMs regulate information disclosure according to context-dependent norms. However, acceptable disclosure boundaries may vary across users even within the same context. To address this limitation, we introduce personalized privacy, which incorporates user-specific disclosure preferences into privacy control. We further present P3Bench~(Personalized Privacy Preservation Benchmark), a novel benchmark extending contextual privacy policies with personalized disclosure policies. Experiments show that prompt-based policies fail to reliably enforce personalized privacy policies, with Qwen2.5-7B and Gemma3-4B showing average policy ignorance ratios of 51.25% and 74.28%, respectively. Finally, to address this problem, we propose Repair, a robust inference-time attention head intervention method that adjusts disclosure behavior toward policy-consistent responses. Our method significantly improves adherence to user-specific privacy preferences by reducing cases where the model fails to follow the given policy.

Junseok Kim, Nakyeong Yang, Kyomin Jung

IPAI, Seoul National University · Max Planck Institute for Software Systems

Privacy-Preserving Language Models Personalized Language Model Alignment Causal Interventions in Language Models

Ask PDF

Abstract

LLM Agent Safety

Explore similar work

Personalization Meets Safety:Mechanisms,Risks,and Mitigations in Personalized LLMs

Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent Systems

Personalized Privacy Control in LLMs via Attention Head Intervention