PsyEvo: A Personalized Counseling Agent That Self-Evolves at Test Time

Yuting Yan; Shihao Xu; Junhao Yu; Mingcong Zuo; Lu Chen; Nan Xiang; Haiyang Geng; Dongjie Tao

cs.AIOct 2, 2026

PsyEvo: A Personalized Counseling Agent That Self-Evolves at Test Time

Authors: Yuting Yan, Shihao Xu, Junhao Yu, Mingcong Zuo, Lu Chen, Nan Xiang, Haiyang Geng, Dongjie Tao,

Organizations: Lyncia Lab · Tianqiao and Chrissy Chen Institute

Abstract

Mental health disorders affect a substantial proportion of the global population, yet a persistent shortage of trained practitioners leaves the majority without adequate care. Large language model (LLM)-based counselors present a promising direction for delivering scalable conversational psychological support. Offline model training alone leaves limited room to adapt to individual clients or to learn from ongoing therapeutic interaction at test time. We introduce PsyEvo, an LLM-based counseling framework that enables both client-specific personalization and response-policy improvement at test time through three components: Hierarchical Bayesian Skill Policy (HBSP) personalizes what intervention to apply by maintaining a per-client skill posterior updated from session feedback; Inter-session Listwise Preference Optimization (LiPO) improves how the selected skill is expressed by updating a shared response adapter from cross-client preference evidence; and State-conditioned Ordinal Credit Assignment (SOCA) supplies candidate preferences and trajectory credit to the two components through consistency-checked comparisons and ordinal projection. In simulated-client evaluation with shared online cohort adaptation, PsyEvo obtains 7.684 Overall on PsychEval and exceeds every component variant in each of three matched runs. Removing individual components lowers mean overall score by 0.138--0.171 under the shared configuration, supporting conditional contributions within the complete scaffold. Our code is available at https://github.com/Lingxi-mental-health/PsyEvo

Figures & tables

Figure 1: Static deployment versus test-time learning in psychological counseling. (a) A statically deployed counselor keeps its parameters fixed across sessions; (b) PsyEvo keeps the backbone frozen, updates the client-specific skill posterior after each session, and schedules shared-adapter updates at five-session intervals on PsychEval, subject to sufficient preference evidence.

Figure 2: Overview of PsyEvo’s test-time learning cycle. Within a session, HBSP selects a skill and SOCA selects the delivered response; between sessions, HBSP updates the client-private posterior using SOCA trajectory credit, while LiPO periodically updates the shared adapter from SOCA preference graphs accumulated across clients. External judges score completed transcripts for reporting only and do not feed the learning signals.

Method	Source	Overall	C-Shared	C-Spec	L-Shared	L-Spec
CACTUS	E	4.869	5.112	2.530	4.594	7.240
CPsyCounX	P	4.680	4.210	2.510	4.730	7.270
SMILE	E	4.142	4.261	2.658	3.794	5.856
PsyDTLLM	P	6.055	6.100	5.430	5.270	7.420
PsyLLM-thinking-v3	P	5.883	5.300	4.670	5.630	7.930
TheraMind	P	6.625	6.250	6.940	5.480	7.830

Table 1: PsychEval results. P denotes a published reference score; E denotes our evaluation. PsyEvo averages seeds 7, 17, and 27 under shared online cohort adaptation. All metrics favor higher values. Complete-system rankings are descriptive; the matched component contrasts appear in Table 3 .

Method	COH $\uparrow$	CON $\uparrow$	FLU $\uparrow$	REL $\uparrow$	Mean $\uparrow$
Raw Scalar	0.440	0.449	0.374	0.443	0.427
JDM	0.544	0.431	0.412	0.479	0.467
PCA	0.590	0.436	0.412	0.520	0.489
PairS-Greedy	0.558	0.390	0.366	0.419	0.433
PairS-Beam	0.553	0.388	0.356	0.417	0.429
BatchEval	0.497	0.422	0.358	0.500	0.444

Table 2: Correlation with human judgments on SummEval. Values are within-document Spearman correlations between automatic and human scores for coherence (COH), consistency (CON), fluency (FLU), and relevance (REL), computed from all 1,600 summaries grouped into 100 source documents and macro-averaged across documents, with all methods using a Qwen3-32B backbone.

Variant	C-Shared	C-Spec	L-Shared	L-Spec	Overall
PsyEvo w/o SOCA	7.510	8.010	6.510	8.098	7.532
PsyEvo with DPO	7.350	8.300	6.450	8.052	7.538
PsyEvo w/o LiPO	7.470	7.990	6.540	8.052	7.513
PsyEvo w/o HBSP	7.320	8.400	6.350	8.114	7.546
PsyEvo	7.527	8.260	6.468	8.481	7.684

Table 3: Ablation of PsyEvo’s three components on PsychEval. All variants share the experimental configuration, with only the specified component removed or replaced; with DPO replaces the weighted candidate-group objective with pairwise DPO, w/o LiPO disables LiPO-based parameter updates, and Overall averages the four metric families. Scores are means across seeds 7, 17, and 27.

Method	COH $\uparrow$	CON $\uparrow$	FLU $\uparrow$	REL $\uparrow$	Mean $\uparrow$
SOCA w/o Pointwise Scoring	0.581	0.434	0.432	0.522	0.492
SOCA w/o Pairwise Comparisons	0.511	0.439	0.438	0.483	0.467
SOCA w/o Repair Evidence	0.587	0.439	0.422	0.529	0.495
SOCA	0.600	0.449	0.448	0.535	0.508

Table 4: Ablation of SOCA’s evidence channels on SummEval. Each variant removes one evidence channel; values are within-document Spearman correlations with human judgments, macro-averaged across the full SummEval dataset under a Qwen3-32B backbone.

System	Ethics	Empathy accuracy	Empathy quality	Interaction	Intervention	Client improvement	Overall
PsyEvo	4.53	3.89	3.73	4.11	4.13	4.20	4.10
PsychAgent-32B	4.49	3.49	3.22	3.47	3.67	3.13	3.58
CPsyCounX	4.49	2.76	2.64	2.49	2.84	2.47	2.95

Table 5: Descriptive pilot ratings of three counseling systems on 25 cases. Three psychology experts blind-rated three sessions per case on five session-level dimensions and case-level client improvement (1–5); Overall is the unweighted mean of the six dimension means.

Figure 3: Client-specific HBSP skill rankings at shared states on PsychEval. (a) Top-ranked skill under HBSP for 20 clients per modality across 54 state-space slots; gray cells mark ties. (b) Pairwise disagreement fraction over the 190 client pairs per modality; diamonds mark medians and horizontal lines the interquartile range.

Figure 4: PsychEval score trajectories over 20 sessions. (a–d) Curves show session-wise mean counselor-shared, counselor-specific, client-shared, and client-specific scores, respectively. The number of scored clients varies across sessions, and panels use separate vertical ranges. Higher scores are better for all metrics.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Source size	Evaluation subset	Reported metrics
PsychEval	369 trajectories; 2,798 sessions; 67,314 turns	100 test clients, 20 per modality across five modalities	C-Shared, C-Spec, L-Shared, L-Spec, Overall
MusPsy	1,400 training and 100 test profiles	100 test profiles $\times$ 6 sessions (600 client–session observations)	WAI, PANAS
SummEval	100 articles; 1,600 summaries from 16 systems	All 100 source documents and 1,600 summaries	Spearman $\rho$ on COH, CON, FLU, REL

Appendix

Table 6: Dataset statistics and evaluation subsets. Source size describes the released corpus; the evaluation subset is the portion used in our experiments.

Setting	Value
Maximum sessions / counselor turns	20 / 45
Soft session target / grace	6 / 2
Random seeds (PsychEval evaluation)	7, 17, 27
Client / shadow temperature	0.7 / 1.3
Shadow responses / total candidates	4 / 5
Update eligibility interval	5 completed sessions across clients

Appendix

Table 7: Online learning configuration for the PsychEval full system and component comparisons. Component variants retain these settings except for the specified intervention.

Object	Contents	Role
Memory	Recap, goals, unresolved themes	Semantic continuity across sessions
Skill posterior	State–skill means and precisions	Personalized selection and exploration
Evidence buffer	Candidates, edges, weights, versions	Preference optimization input
Adapter registry	Learning scope, parent version, checkpoint, activation boundary	Loading, replay, and rollback

Appendix

Table 8: Persistent client and counselor learning states. Each row lists a stored object, its contents, and its role in the test-time learning cycle.

Variant	Changed operation	Retained learning or selection operations
w/o HBSP	Remove client-private residuals and their updates	Fixed shared skill table; SOCA selection and shared-adapter learning
w/o LiPO	Keep the response adapter fixed	Client-posterior updates; SOCA candidate evaluation and selection
w/o SOCA	Use raw-score preferences and unprojected outcome credit	Client-posterior updates; grouped preference optimization
with DPO	Replace weighted group loss with unit-weight pair loss	SOCA preferences; client-posterior and shared-adapter updates

Appendix

Table 9: Scope of the matched PsychEval interventions. Memory, candidate generation, backbone, case set, and evaluation protocol are retained throughout. Each row changes the indicated component within the remaining scaffold.

Variant	Seed 7	Seed 17	Seed 27	Mean	SD
PsyEvo w/o SOCA	7.498	7.531	7.567	7.532	0.035
PsyEvo with DPO	7.485	7.538	7.591	7.538	0.053
PsyEvo w/o LiPO	7.438	7.515	7.586	7.513	0.074
PsyEvo w/o HBSP	7.482	7.548	7.608	7.546	0.063
PsyEvo	7.656	7.684	7.712	7.684	0.028

Appendix

Table 10: PsychEval Overall scores across three random seeds. Mean and SD denote the across-seed mean and sample standard deviation.

Comparison	Mean difference	95% CI
PsyEvo $-$ with DPO	+0.146	[0.048, 0.244]
PsyEvo $-$ w/o parameter updates	+0.171	[0.085, 0.257]
PsyEvo $-$ w/o SOCA	+0.152	[0.060, 0.244]
PsyEvo $-$ w/o HBSP	+0.138	[0.052, 0.224]

Appendix

Table 11: Paired differences in PsychEval Overall score. Positive differences favor PsyEvo; CI denotes the 95% confidence interval.

Figure 5: Internal self-evaluation signal diagnostics. (a) Empirical cumulative distribution of internal rewards over 63 sessions; (b) standard deviation of ordinal credit at each of the twelve adapter updates. Both statistics are computed from the archived training trace.

Channel	Unit	Use
Internal candidate judgment	Candidates in one state	LiPO preferences
Client-grounded reward	Realized turns and sessions	HBSP trajectory credit
External PsychEval judgment	Frozen transcript	Outcome reporting
Human ballot	Matched dialogue item	Independent quality and ranking

Appendix

Table 12: Feedback channels and the modules that use them. Each row gives the unit at which a channel is recorded and the module that consumes it; only the first two channels enter learning.

Metric	Pearson $r$	Spearman $\rho$	95% interval	Offset
C-Shared	.430	.480	[.412,.538]	+1.989
WAI	.450	.499	[.431,.560]	+1.398
C-Spec	.677	.632	[.531,.708]	$-.652$
L-Shared	.521	.530	[.463,.591]	+.729

Appendix

Table 13: Agreement between Qwen3-32B and Gemini-3-Flash judges on 1,468 matched sessions. Intervals are 95% client-level bootstrap intervals (2,000 replicates) for the Spearman correlation, and offsets are mean Qwen minus Gemini scores.

Dimension	Criterion	1–2	3	4–5
Ethics	Counseling ethics and professional boundaries	Unsupported diagnoses, dangerous advice, or excessive promises	Minor boundary issues without substantive harm	Respects boundaries; clarifies diagnostic or medical limits
Empathy accuracy	Recognition of expressed emotion	Frequently invents unexpressed emotions	Mostly accurate, with occasional subjective inference	Accurately reflects expressed emotion
Empathy quality	Depth and supportive value	Little empathy or awkward, inappropriate responses	Reasonable but superficial empathy	Deep understanding and support for self-awareness
Interaction	Continuity, coherence, and pacing	Disconnection, repetition, or abrupt topic changes	Generally coherent, with occasional mismatch or repetition	Natural, responsive, and fluent interaction
Intervention	Fit to the counseling stage	Premature advice or generic guidance	Appropriate direction with imperfect timing or intensity	Gradual intervention matched to the stage and therapeutic logic
Client improvement	Change across three sessions	No improvement or clear deterioration	Some improvement or fluctuation	Clear, sustained positive change

Appendix

Table 14: Criteria and anchors presented to expert raters for the six reported dimensions.

Dimension / composite	Metric	Estimate	95% CI
Ethics	Ordinal $\alpha$	0.317	[0.204, 0.455]
Empathy accuracy	Ordinal $\alpha$	0.035	[ $-0.131$ , 0.171]
Empathy quality	Ordinal $\alpha$	0.153	[ $-0.040$ , 0.337]
Interaction	Ordinal $\alpha$	0.106	[ $-0.057$ , 0.239]
Intervention	Ordinal $\alpha$	$0.350$	[0.115, 0.585]
Evolution	Ordinal $\alpha$	0.084	[ $-0.150$ , 0.235]

Appendix

Table 15: Reported inter-rater agreement and reliability for the 25-case evaluation with three human raters. Ordinal

\alpha

measures dimension-level agreement; ICC(A,3) measures absolute-agreement reliability of the three-rater mean composite.

Figure 6: Skill preferences before and after thirteen sessions for HET-860. Left and middle: within-state rankings of shared values plus client-specific posterior means before Session 1 and after Session 13; darker colors indicate higher priority. Right: initial rank minus final rank, with blue indicating increased priority, red decreased priority, and white no change; tied skills receive average ranks. Rows cover all 32 states visited in the complete course, grouped by counseling stage. Early, Middle, and Late denote within-session progress; emotion categories are identified by keyword rules.

Session	WAI	Client-shared	Nonzero reads	Cells	Adapter
1	6.250	5.188	0/28	17	$v_{0}$
2	6.042	4.875	0/25	31	$v_{1}$
3	7.292	4.938	21/29	37	$v_{1}$
4	7.917	6.938	30/33	40	$v_{2}$
5	7.917	6.188	29/29	41	$v_{3}$
6	7.083	5.312	34/35	44	$v_{4}$

Appendix

Table 16: The complete thirteen-session HET-860 case. Client-shared is the mean of external PANAS and SRS scores. Nonzero reads count decisions that access a nonzero posterior mean; cells count cumulatively updated state–skill entries. Adapter versions are those recorded during generation.

Record	Observed sequence
Sessions	1, 2, 3, 4, 5
Updated posterior cells	14, 30, 34, 42, 43
Adapter versions	$v_{0},v_{0},v_{1},v_{2},v_{3}$
Archived ordinal tiers	$[0,0,1,2,1]$
Archived expanded credit	$[0.241,0.241,0.741,1.000,0.741]$

Appendix

Table 17: Recorded state changes for client CBT-412. The first three rows track session-wise state; the final two report the archived ordinal tiers and expanded credit.

Risk	Failure path	Control
Crisis handling	Engagement reward competes with escalation	Independent crisis routing and professional oversight
Policy drift	Repeated local updates shift behavior	Fixed checks, versioning, rollback
Preference poisoning	Adversarial clients inject misleading preference evidence	Per-client isolation, client-balanced loss, edge caps, validation before activation
Memory error	Incorrect summaries persist	Source-linked correction
Privacy	Sensitive information in memory or adapters	Provenance, access control, deletion and unlearning evaluation
State/version routing	Wrong client memory or counselor version loaded	Identity checks and activation records

Appendix

Table 18: Deployment risks and candidate controls. Each row names a risk, the path through which it could arise under test-time learning, and a control whose effectiveness remains to be evaluated.

Method	WAI Goal	WAI Task	WAI Bond	WAI Avg	PANAS Pos.	PANAS Neg.
CACTUS	4.125	3.713	4.427	4.085	3.035	2.333
CPsyCounX	4.352	4.239	4.639	4.410	3.325	1.317
SMILE	3.582	3.647	4.257	3.827	2.695	2.875
PsyDTLLM	4.327	4.219	4.595	4.380	3.118	1.231
PsyLLM-thinking-v3	4.406	4.314	4.688	4.469	3.547	1.248
TheraMind	4.409	4.312	4.667	4.463	3.438	1.236

Appendix

Table 19: Supplementary MusPsy figures. WAI and positive PANAS favor higher values; negative PANAS favors lower values. The main PsyEvo configuration linkage remains unresolved, so these point estimates are not used to establish the paper’s principal experimental claims.

LLM Agent Self-Improvement

Large language model agent self-improvement studies how agents use experience and feedback to improve their behavior, tools, and workflows.

See all 276 papers

PsychAgent: An Experience-Driven Lifelong Learning Agent for Self-Evolving Psychological Counselor
Apr 1, 2026Yutao Yang, Junsong Li, Qianjun Pan +7Lifelong Learning Agents Agent Skill Learning
School of Computer Science and Technology, East China Normal University · Shanghai AI Laboratory · School of Psychology and Cognitive Science, East China Normal University
PsyCLIENT: Client Simulation via Conversational Trajectory Modeling for Trainee Practice and Model Evaluation in Mental Health Counseling
Jan 12, 2026Huachuan Qiu, Zhaoming Chen, Yuqian Chen +3Large Language Model-Based Role-Play Simulation User Simulation
Westlake University · The University of Utah
MAGneT: Coordinated Multi-Agent Generation of Synthetic Multi-Turn Mental Health Counseling Sessions
Sep 4, 2025Aishik Mandal, Tanmoy Chakraborty, Iryna GurevychMental Health Counseling Mental Health
Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science and Hessian Center for AI (hessian.AI), Technische Universität Darmstadt · National Research Center for Applied Cybersecurity ATHENE, Germany · Department of Electrical Engineering, Indian Institute of Technology Delhi, India +1

Apr 1, 2026J/K move · Enter open · S save

PsychAgent: An Experience-Driven Lifelong Learning Agent for Self-Evolving Psychological Counselor

Yutao Yang, Junsong Li, Qianjun Pan, Jie Zhou+6

School of Computer Science and Technology, East China Normal University · Shanghai AI Laboratory · School of Psychology and Cognitive Science, East China Normal University

Lifelong Learning Agents Agent Skill Learning Self-Improving Agents Mental Health Counseling Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

Existing methods for AI psychological counselors predominantly rely on supervised fine-tuning using static dialogue datasets. However, this contrasts with human experts, who continuously refine their proficiency through clinical practice and accumulated experience. To bridge this gap, we propose an Experience-Driven Lifelong Learning Agent (PsychAgent) for psychological counseling. First, we establish a Memory-Augmented Planning Engine tailored for longitudinal multi-session interactions, which ensures therapeutic continuity through persistent memory and strategic planning. Second, to support self-evolution, we design a Skill Evolution Engine that extracts new practice-grounded skills from historical counseling trajectories. Finally, we introduce a Reinforced Internalization Engine that integrates the evolved skills into the model via rejection fine-tuning, aiming to improve performance across diverse scenarios. Comparative analysis shows that our approach achieves higher scores than strong general LLMs (e.g., GPT-5.4, Gemini-3) and domain-specific baselines across all reported evaluation dimensions. These results suggest that lifelong learning can improve the consistency and overall quality of multi-session counseling responses.

Abstract

LLM Agent Self-Improvement

Explore similar work

PsychAgent: An Experience-Driven Lifelong Learning Agent for Self-Evolving Psychological Counselor

PsyCLIENT: Client Simulation via Conversational Trajectory Modeling for Trainee Practice and Model Evaluation in Mental Health Counseling

MAGneT: Coordinated Multi-Agent Generation of Synthetic Multi-Turn Mental Health Counseling Sessions