PsychAgent: An Experience-Driven Lifelong Learning Agent for Self-Evolving Psychological Counselor
Authors: Yutao Yang, Junsong Li, Qianjun Pan, Jie Zhou, Kai Chen, Qin Chen, Jingyuan Zhao, Ningning Zhou, Xin Li, Liang He
Organizations: School of Computer Science and Technology, East China Normal University · Shanghai AI Laboratory · School of Psychology and Cognitive Science, East China Normal University
Existing methods for AI psychological counselors predominantly rely on supervised fine-tuning using static dialogue datasets. However, this contrasts with human experts, who continuously refine their proficiency through clinical practice and accumulated experience. To bridge this gap, we propose an Experience-Driven Lifelong Learning Agent (PsychAgent) for psychological counseling. First, we establish a Memory-Augmented Planning Engine tailored for longitudinal multi-session interactions, which ensures therapeutic continuity through persistent memory and strategic planning. Second, to support self-evolution, we design a Skill Evolution Engine that extracts new practice-grounded skills from historical counseling trajectories. Finally, we introduce a Reinforced Internalization Engine that integrates the evolved skills into the model via rejection fine-tuning, aiming to improve performance across diverse scenarios. Comparative analysis shows that our approach achieves higher scores than strong general LLMs (e.g., GPT-5.4, Gemini-3) and domain-specific baselines across all reported evaluation dimensions. These results suggest that lifelong learning can improve the consistency and overall quality of multi-session counseling responses.
Figures & tables
Figure 1: The overall architecture of PsychAgent , an experience-driven lifelong learning framework for psychological counseling. The system operates through a synergistic closed-loop mechanism: (a) The Memory-Augmented Planning Engine ensures therapeutic continuity across longitudinal sessions by reasoning over dynamic client profiles and formulating strategic goals; (b) The Skill Evolution Engine manages a hierarchical repository of therapeutic techniques, supporting both real-time context-aware retrieval and the post-session abstraction of novel skills; (c) The Reinforced Internalization Engine solidifies these capabilities via rejection fine-tuning, where optimal trajectories are selected to transform explicit skills into the model’s endogenous intuition.
Model
Counselor-level
Client-level
Shared
Specific
Shared
Specific
General-purpose LLMs
GPT-5.4
5.54
7.41
5.07
7.72
Gemini-3
5.34
7.04
4.97
7.52
Qwen3-Max
5.88
7.74
5.41
7.81
DeepSeek-V3.2
5.54
7.12
5.06
7.70
Table 1: Main results in terms of counselor- and client-level metrics. † means the model is based on Qwen3-8B. Best results are in bold and second-best results are underlined.
Model
Human A
Human B
LLM
Rank
PsychAgent
4.295
4.370
4.303
1
Qwen3-Max
3.943
4.024
4.039
2
TheraMind
3.743
3.649
3.573
3
Table 2: Results of human evaluation.
Counselor-level
Client-level
Shared
Specific
Shared
Specific
PsychAgent
7.32
7.91
5.92
8.24
w/o MAPE
7.07
7.66
5.86
8.03
w/o SEE
7.02
7.52
5.61
7.90
w/o RIE
7.05
7.67
5.67
7.89
Table 3: The results of ablation studies.
Figure 2: Clients’ Emotional Trajectories.
Counselor-level
Client-level
Shared
Specific
Shared
Specific
DeepSeek-V3.2
5.54
7.12
5.06
7.70
+ Vanilla RAG
5.45
6.79
4.92
7.67
+ Graph RAG
5.45
6.88
4.92
7.73
+ Mem0
5.45
6.94
4.87
7.68
+ MemoryBank
5.49
6.92
4.90
7.70
Table 4: Influence of Memory Mechanisms.
Figure 3: Detailed results across Counselor- and Client-Level therapy metrics
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Level
Group
n
Human A vs Human B
LLM vs Human A
LLM vs Human B
Overall
All ratings
2088
0.675
0.770
0.877
By dimension
Ethics
522
0.606
0.756
0.823
Interaction
522
0.576
0.691
0.841
Intervention
522
0.747
0.812
0.907
Perception
522
0.546
0.660
0.843
By model
PsychAgent
692
0.626
0.752
0.865
Appendix
Table 5: Quadratic Weighted Kappa (QWK) across raters.
Level
Group
n
LLM − Human A
LLM − Human B
Overall
All ratings
2088
−0.022
−0.043
By dimension
Ethics
522
−0.031
−0.102
Interaction
522
− 0.025
−0.048
Intervention
522
−0.042
−0.038
Perception
522
−0.038
− 0.017
By model
PsychAgent
692
− 0.009
−0.066
Appendix
Table 6: Systematic mean-score bias of LLM Rater (Gemini-3) relative to each human rater. Positive values indicate that the LLM assigns higher scores on average.
Metric
SFT
Internalization
Dialogue / session count
6701
729
Counselor turns
154,966
20,869
Turns with <skill> tags
154,959
20,865
Unique matched multi-skill bundles
19,296
2,396
Isolated-skill turn ratio
50.8%
65.8%
Multi-skill turn ratio
49.2%
34.2%
Appendix
Table 7: Overall structural comparison between the SFT baseline and internalization data. The main shift is not a larger share of multi-skill turns, but more stable functional packaging and clearer family-level reuse across contexts.
Functional skill
Therapy
Internal
SFT
Status
Goal-to-Record Bridge
CBT
30
58
stable
Rated Reappraisal Loop
CBT
22
104
stable
Dose-Specified Practice Scaffold
BT
18
72
stable
Validation-Wrapped Pacing Regulation
PDT
6
0
stable
Alternative-Story Amplification
PMT
241
1338
stable
Resource-to-Action Scaffold
HET
54
166
stable
Appendix
Table 8: Representative candidate functional skills after internalization. These packages are best interpreted as operationally new functional units built from known micro-skills, rather than as entirely new canonical techniques.
Functional family
Therapies involved
Internal
SFT
Progress Consolidation Package
BT, HET, PDT, PMT
292
2287
Goal-to-Practice Bridge
BT, CBT, HET, PMT
207
984
Affect Attunement Before Deeper Work
CBT, HET, PDT
28
48
Appendix
Table 9: Representative functional families reused across therapies after internalization. The main evidence comes from family-level reuse rather than from frequent exact mixed-therapy bundles in a single turn.
Mental health disorders affect a substantial proportion of the global population, yet a persistent shortage of trained practitioners leaves the majority without adequate care. Large language model (LLM)-based counselors present a promising direction for delivering scalable conversational psychological support. Offline model training alone leaves limited room to adapt to individual clients or to learn from ongoing therapeutic interaction at test time. We introduce PsyEvo, an LLM-based counseling framework that enables both client-specific personalization and response-policy improvement at test time through three components: Hierarchical Bayesian Skill Policy (HBSP) personalizes what intervention to apply by maintaining a per-client skill posterior updated from session feedback; Inter-session Listwise Preference Optimization (LiPO) improves how the selected skill is expressed by updating a shared response adapter from cross-client preference evidence; and State-conditioned Ordinal Credit Assignment (SOCA) supplies candidate preferences and trajectory credit to the two components through consistency-checked comparisons and ordinal projection. In simulated-client evaluation with shared online cohort adaptation, PsyEvo obtains 7.684 Overall on PsychEval and exceeds every component variant in each of three matched runs. Removing individual components lowers mean overall score by 0.138--0.171 under the shared configuration, supporting conditional contributions within the complete scaffold. Our code is available at https://github.com/Lingxi-mental-health/PsyEvo
The growing demand for scalable psychological counseling highlights the need for high-quality, privacy-compliant data, yet such data remains scarce. Here we introduce MAGneT, a novel multi-agent framework for synthetic psychological counseling session generation that decomposes counselor response generation into coordinated sub-tasks handled by specialized LLM agents, each modeling a key psychological technique. Unlike prior single-agent approaches, MAGneT better captures the structure and nuance of real counseling. We further propose a unified evaluation framework that consolidates diverse automatic metrics and expands expert assessment from four to nine counseling aspects, thus addressing inconsistencies in prior evaluation protocols. Empirically, MAGneT substantially outperforms existing methods: experts prefer MAGneT-generated sessions in 77.2% of cases on average across the nine aspects over the strongest baseline, and sessions generated by MAGneT using Llama3-8B-Instruct backbone yield 3.2% higher general counseling skills and 4.3% higher CBT-specific skills on cognitive therapy rating scale (CTRS). An open source Llama3-8B-Instruct model fine-tuned on MAGneT-generated data also outperforms models fine-tuned using baseline synthetic datasets by 6.9% on average on CTRS. We make our code, data and fine-tuned model public.
Aishik Mandal, Tanmoy Chakraborty, Iryna Gurevych
Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science and Hessian Center for AI (hessian.AI), Technische Universität Darmstadt · National Research Center for Applied Cybersecurity ATHENE, Germany · Department of Electrical Engineering, Indian Institute of Technology Delhi, India +1
Psychological counseling unfolds over many sessions, and what a client brings to each session depends on what has happened in their life since the last one. Existing synthetic counseling corpora are mostly single-session, and multi-session ones advance along a therapy plan without modeling the client's life between sessions. We present Psy-Chronicle, an event-driven framework that grounds multi-session counseling synthesis in a student's semester timeline: each simulated student receives a profile and a semester-long temporal stress event graph, and counseling sessions are generated along this timeline with memory carried across them. Applied to campus counseling, it yields CPCD, an open Chinese dataset of 100 students and 1{,}248 sessions spanning a full semester. The dialogues follow their timelines: dependencies between events carry across sessions, and references to earlier sessions rarely introduce content absent from the trajectory. Under a published evaluation protocol, in basic quality and exceeds all single-session corpora in working alliance. A model fine-tuned on CPCD connects each week's events to earlier ones more often than a model trained on the same data without the dependencies between events, while matching it when the client's situation does not change.