PsychAgent: An Experience-Driven Lifelong Learning Agent for Self-Evolving Psychological Counselor

Yutao Yang; Junsong Li; Qianjun Pan; Jie Zhou; Kai Chen; Qin Chen; Jingyuan Zhao; Ningning Zhou

cs.AIApr 1, 2026

PsychAgent: An Experience-Driven Lifelong Learning Agent for Self-Evolving Psychological Counselor

Authors: Yutao Yang, Junsong Li, Qianjun Pan, Jie Zhou, Kai Chen, Qin Chen, Jingyuan Zhao, Ningning Zhou,

Organizations: School of Computer Science and Technology, East China Normal University · Shanghai AI Laboratory · School of Psychology and Cognitive Science, East China Normal University

Abstract

Existing methods for AI psychological counselors predominantly rely on supervised fine-tuning using static dialogue datasets. However, this contrasts with human experts, who continuously refine their proficiency through clinical practice and accumulated experience. To bridge this gap, we propose an Experience-Driven Lifelong Learning Agent (PsychAgent) for psychological counseling. First, we establish a Memory-Augmented Planning Engine tailored for longitudinal multi-session interactions, which ensures therapeutic continuity through persistent memory and strategic planning. Second, to support self-evolution, we design a Skill Evolution Engine that extracts new practice-grounded skills from historical counseling trajectories. Finally, we introduce a Reinforced Internalization Engine that integrates the evolved skills into the model via rejection fine-tuning, aiming to improve performance across diverse scenarios. Comparative analysis shows that our approach achieves higher scores than strong general LLMs (e.g., GPT-5.4, Gemini-3) and domain-specific baselines across all reported evaluation dimensions. These results suggest that lifelong learning can improve the consistency and overall quality of multi-session counseling responses.

Figures & tables

Figure 1: The overall architecture of PsychAgent , an experience-driven lifelong learning framework for psychological counseling. The system operates through a synergistic closed-loop mechanism: (a) The Memory-Augmented Planning Engine ensures therapeutic continuity across longitudinal sessions by reasoning over dynamic client profiles and formulating strategic goals; (b) The Skill Evolution Engine manages a hierarchical repository of therapeutic techniques, supporting both real-time context-aware retrieval and the post-session abstraction of novel skills; (c) The Reinforced Internalization Engine solidifies these capabilities via rejection fine-tuning, where optimal trajectories are selected to transform explicit skills into the model’s endogenous intuition.

Model	Counselor-level		Client-level
Model	Shared	Specific	Shared	Specific
General-purpose LLMs
GPT-5.4	5.54	7.41	5.07	7.72
Gemini-3	5.34	7.04	4.97	7.52
Qwen3-Max	5.88	7.74	5.41	7.81
DeepSeek-V3.2	5.54	7.12	5.06	7.70

Table 1: Main results in terms of counselor- and client-level metrics. † means the model is based on Qwen3-8B. Best results are in bold and second-best results are underlined.

Model	Human A	Human B	LLM	Rank
PsychAgent	4.295	4.370	4.303	1
Qwen3-Max	3.943	4.024	4.039	2
TheraMind	3.743	3.649	3.573	3

Table 2: Results of human evaluation.

	Counselor-level		Client-level
	Shared	Specific	Shared	Specific
PsychAgent	7.32	7.91	5.92	8.24
w/o MAPE	7.07	7.66	5.86	8.03
w/o SEE	7.02	7.52	5.61	7.90
w/o RIE	7.05	7.67	5.67	7.89

Table 3: The results of ablation studies.

Figure 2: Clients’ Emotional Trajectories.

	Counselor-level		Client-level
	Shared	Specific	Shared	Specific
DeepSeek-V3.2	5.54	7.12	5.06	7.70
+ Vanilla RAG	5.45	6.79	4.92	7.67
+ Graph RAG	5.45	6.88	4.92	7.73
+ Mem0	5.45	6.94	4.87	7.68
+ MemoryBank	5.49	6.92	4.90	7.70

Table 4: Influence of Memory Mechanisms.

Figure 3: Detailed results across Counselor- and Client-Level therapy metrics

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Level	Group	$n$	Human A vs Human B	LLM vs Human A	LLM vs Human B
Overall	All ratings	2088	0.675	0.770	0.877
By dimension	Ethics	522	0.606	0.756	0.823
	Interaction	522	0.576	0.691	0.841
	Intervention	522	0.747	0.812	0.907
	Perception	522	0.546	0.660	0.843
By model	PsychAgent	692	0.626	0.752	0.865

Appendix

Table 5: Quadratic Weighted Kappa (QWK) across raters.

Level	Group	$n$	LLM $-$ Human A	LLM $-$ Human B
Overall	All ratings	2088	$-0.022$	$-0.043$
By dimension	Ethics	522	$-0.031$	$-0.102$
	Interaction	522	$-$ 0.025	$-0.048$
	Intervention	522	$-0.042$	$-0.038$
	Perception	522	$-0.038$	$-$ 0.017
By model	PsychAgent	692	$-$ 0.009	$-0.066$

Appendix

Table 6: Systematic mean-score bias of LLM Rater (Gemini-3) relative to each human rater. Positive values indicate that the LLM assigns higher scores on average.

Metric	SFT	Internalization
Dialogue / session count	6701	729
Counselor turns	154,966	20,869
Turns with <skill> tags	154,959	20,865
Unique matched multi-skill bundles	19,296	2,396
Isolated-skill turn ratio	50.8%	65.8%
Multi-skill turn ratio	49.2%	34.2%

Appendix

Table 7: Overall structural comparison between the SFT baseline and internalization data. The main shift is not a larger share of multi-skill turns, but more stable functional packaging and clearer family-level reuse across contexts.

Functional skill	Therapy	Internal	SFT	Status
Goal-to-Record Bridge	CBT	30	58	stable
Rated Reappraisal Loop	CBT	22	104	stable
Dose-Specified Practice Scaffold	BT	18	72	stable
Validation-Wrapped Pacing Regulation	PDT	6	0	stable
Alternative-Story Amplification	PMT	241	1338	stable
Resource-to-Action Scaffold	HET	54	166	stable

Appendix

Table 8: Representative candidate functional skills after internalization. These packages are best interpreted as operationally new functional units built from known micro-skills, rather than as entirely new canonical techniques.

Functional family	Therapies involved	Internal	SFT
Progress Consolidation Package	BT, HET, PDT, PMT	292	2287
Goal-to-Practice Bridge	BT, CBT, HET, PMT	207	984
Affect Attunement Before Deeper Work	CBT, HET, PDT	28	48

Appendix

Table 9: Representative functional families reused across therapies after internalization. The main evidence comes from family-level reuse rather than from frequent exact mixed-therapy bundles in a single turn.

Lifelong Learning Agents

Lifelong learning agents adapt after deployment by retaining useful experience, acquiring skills, and reusing knowledge across changing tasks.

See all 51 papers

PsyEvo: A Personalized Counseling Agent That Self-Evolves at Test Time
Oct 2, 2026Yuting Yan, Shihao Xu, Junhao Yu +6LLM Agent Self-Improvement Mental Health Counseling
Lyncia Lab · Tianqiao and Chrissy Chen Institute
MAGneT: Coordinated Multi-Agent Generation of Synthetic Multi-Turn Mental Health Counseling Sessions
Sep 4, 2025Aishik Mandal, Tanmoy Chakraborty, Iryna GurevychMental Health Counseling Mental Health
Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science and Hessian Center for AI (hessian.AI), Technische Universität Darmstadt · National Research Center for Applied Cybersecurity ATHENE, Germany · Department of Electrical Engineering, Indian Institute of Technology Delhi, India +1
Psy-Chronicle: Synthesizing Campus Counseling Grounded in a Student's Semester Timeline
May 21, 2026Chaogui GouMental Health Counseling Emotional Support Conversation
University of Science and Technology Beijing

Oct 2, 2026J/K move · Enter open · S save

PsyEvo: A Personalized Counseling Agent That Self-Evolves at Test Time

Yuting Yan, Shihao Xu, Junhao Yu, Mingcong Zuo+5

Lyncia Lab · Tianqiao and Chrissy Chen Institute

LLM Agent Self-Improvement Mental Health Counseling Personalized Language Models Agent Skill Learning Preference Optimization

Open paper AskPDF arXiv

Mental health disorders affect a substantial proportion of the global population, yet a persistent shortage of trained practitioners leaves the majority without adequate care. Large language model (LLM)-based counselors present a promising direction for delivering scalable conversational psychological support. Offline model training alone leaves limited room to adapt to individual clients or to learn from ongoing therapeutic interaction at test time. We introduce PsyEvo, an LLM-based counseling framework that enables both client-specific personalization and response-policy improvement at test time through three components: Hierarchical Bayesian Skill Policy (HBSP) personalizes what intervention to apply by maintaining a per-client skill posterior updated from session feedback; Inter-session Listwise Preference Optimization (LiPO) improves how the selected skill is expressed by updating a shared response adapter from cross-client preference evidence; and State-conditioned Ordinal Credit Assignment (SOCA) supplies candidate preferences and trajectory credit to the two components through consistency-checked comparisons and ordinal projection. In simulated-client evaluation with shared online cohort adaptation, PsyEvo obtains 7.684 Overall on PsychEval and exceeds every component variant in each of three matched runs. Removing individual components lowers mean overall score by 0.138--0.171 under the shared configuration, supporting conditional contributions within the complete scaffold. Our code is available at https://github.com/Lingxi-mental-health/PsyEvo

Abstract

Lifelong Learning Agents

Explore similar work

PsyEvo: A Personalized Counseling Agent That Self-Evolves at Test Time

MAGneT: Coordinated Multi-Agent Generation of Synthetic Multi-Turn Mental Health Counseling Sessions

Psy-Chronicle: Synthesizing Campus Counseling Grounded in a Student's Semester Timeline