Beyond Empathy: Integrating Diagnostic and Therapeutic Reasoning with Large Language Models for Mental Health Counseling
Authors: He Hu, Juzheng Si, Qianning Wang, Fei Ma, Fuji Ren, Sicheng Zhao, Yucheng Zhou, Laizhong Cui
Organizations: College of Computer Science and Software Engineering, Shenzhen University · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Shandong University · Auckland University of Technology · SKL-IOTSC, CIS, University of Macau
Large language models (LLMs) hold significant potential for psychological support, capable of generating empathetic responses and simulating therapeutic conversations. However, existing LLM-based approaches often lack sufficient grounding in structured diagnostic reasoning and diverse therapeutic strategies, limiting their effectiveness in complex counseling scenarios. To address these limitations, we propose PsyLLM, the first large language model designed to incorporate diagnostic- and therapeutic-oriented reasoning for mental health counseling. To develop PsyLLM, we design a novel automated data synthesis pipeline that processes real-world mental health posts collected from Reddit, where users frequently express psychological distress and seek community support. The pipeline transforms these posts into multi-turn counseling dialogues and leverages LLMs guided by international diagnostic standards (e.g., DSM/ICD) and multiple therapeutic frameworks (e.g., CBT, ACT, psychodynamic) to generate structured, clinically informed reasoning traces. A rigorous multi-dimensional filtering mechanism further improves the quality and clinical fidelity of the synthesized dialogue data. In addition, we introduce a new benchmark and evaluation protocol, assessing counseling performance across four key dimensions. Our experiments, along with evaluations by professional counseling experts, demonstrate that PsyLLM consistently outperforms various baseline models on this benchmark. Code, data, and models are available at https://github.com/Emo-gml/PsyLLM.
Figures & tables
Fig. 1: PsyLLM simulates therapeutic reasoning by assessing emotions, analyzing cognitive patterns, and formulating strategies grounded in DSM/ICD criteria and diverse modalities (e.g., CBT, ACT, psychodynamic). This enables clinically informed, context-sensitive counseling responses.
Dataset
Public
Turn Type
Reasoning?
Therapy-Guided
Clinically-Guided
Dialogues
Psych8k [ 9 ]
Single-turn
8,187
CPsyCounD [ 10 ]
Multi-turn
3,134
SweetieChat [ 11 ]
Multi-turn
3,757
PsyDTCorpus [ 12 ]
Multi-turn
5,000
TeenEmpathSupport [ 13 ]
Multi-turn
6,590
MDD-5k [ 14 ]
Multi-turn
5,000
TABLE I: Comparison of OpenR1-Psy with existing psychological dialogue datasets, including whether each dataset is publicly available.
Fig. 2: Overview of the OpenR1-Psy dataset construction pipeline. The process includes five stages: (1) Data collection from Reddit and real counseling datasets. (2) Parsing and interaction planning using a language model to assess emotions, define dialogue rounds, and set therapeutic themes. (3) Extraction of patient utterances from simulated dialogues and real counseling data. (4) Generation of reasoning traces and counselor responses based on diagnostic standards and therapeutic strategies. (5) Multi-dimensional validation to ensure coherence, clinical relevance, and reasoning quality.
Fig. 3: Analysis of OpenR1-Psy Dataset: (Left) Distribution of Psychotherapy Approaches; (Middle) Distribution of Scene Categories; (Right) Distribution of Severity Levels.
Category
Train
Test
All
# Dialogues
18,852
450
19,302
# Utterances
47,723
1,651
49,374
Avg. Multi-turns
3.76
5.04
3.80
Avg. Patient Utterance Length
224.18
171.84
222.43
Avg. Counselor Utterance Length
520.11
489.4
519.08
Avg. Reasoning Utterance Length
1,592.33
1,627.36
1,593.5
TABLE II: Statistics of the OpenR1-Psy Dataset (Train / Test).
Method
Metrics
Emp&In
Sup&Aut
Att&Pre
Saf&Bou
Normalized Avg
Psych8k [ 9 ]
0.740
1.830
1.430
0.990
0.574
OpenR1-Psy
1.630 (+120%)
3.590 (+96%)
2.660 (+86%)
1.000 (+1%)
0.900 (+57%)
CPsyCounD [ 10 ]
0.950
1.950
1.840
0.980
0.639
OpenR1-Psy
1.630 (+71%)
3.590 (+84%)
2.660 (+44%)
1.000 (+2%)
0.900 (+41%)
TABLE III: Synthetic data comparison across four metrics and normalized averages based on metric max scores: 2, 4, 3, 1. OpenR1-Psy achieves the highest performance and substantial relative improvements.
Evaluator
Method
Metrics
Emp&In
Sup&Aut
Att&Pre
Saf&Bou
Normalized Avg
Automatic
ChatCounselor [ 9 ]
0.015
0.826
0.332
0.469
0.198
CPsyCounX [ 10 ]
0.005
0.492
0.180
0.358
0.136
MeChat [ 27 ]
0.018
0.856
0.414
0.800
0.291
PsyDTLLM [ 12 ]
0.148
1.135
0.689
0.872
0.365
DeepSeek-V3 [ 50 ]
0.880
2.155
1.057
0.830
0.540
TABLE IV: Automatic and human evaluation results. Best values are in bold , second-best are underlined .
Fig. 4: Average performance of topic dimensions. Each axis represents a topic, and the radius indicates its mean score on the associated metric, highlighting relative strengths and weaknesses.
Fig. 5: Comparison of different reasoning methods, including In-Context Learning and Two-Phase Prompting.
Fig. 6: Ablation study results (left) and correlation matrix (right). Each bar in the ablation plot presents the evaluation score when a particular module is removed, illustrating the marginal contribution of every component to the full system.
Method
Empathy
Clarity
Justification
Coherence
Structure
Normalized Avg
Automatic
2.806
2.710
2.660
2.970
2.882
0.935
Human
2.879
2.890
2.990
2.672
3.000
0.962
TABLE V: Evaluation scores of counselor thinking across five cognitive dimensions.
Fig. 7: Impact of varying base model scale (Left) and training data volume (Right) on PsyLLM’s counseling performance
Fig. 8: Case study comparing responses from CPsyCounX and our model, PsyLLM. PsyLLM’s response is guided by an explicit reasoning process (highlighted), which integrates therapeutic strategies (CBT and humanistic) to produce a more empathetic and tailored dialogue that directly addresses the user’s core emotional struggle.
Metric
Correlation
Empathy & Insight
0.585945 ∗∗
Support & Autonomy
0.534870 ∗∗
Attunement & Presence
0.566259 ∗∗
Safety & Boundaries
0.440841 ∗∗
OA
0.604751 ∗∗
TABLE VI: Spearman’s rank correlations between human and LLM-generated scores. Significant correlations are marked with ∗∗ ( p -value<0.05).
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Fig. 9: An example of a multi-turn dialogue generated by PsyLLM. For each turn, the model first produces an explicit reasoning trace (highlighted in yellow) that outlines its therapeutic assessment and strategy. This internal monologue then guides the generation of the final, empathetic counselor response.
Topic
Model
Emp&In
Sup&Aut
Att&Pre
Saf&Bou
Self-growth
PsyDTLLM
0.172
1.210
0.726
0.936
MeChat
0.013
0.866
0.357
0.904
ChatCounselor
0.000
0.774
0.226
0.419
CPsyCounX
0.000
0.581
0.355
0.548
DeepSeek-V3
0.839
2.032
0.903
0.710
GPT-4o
0.774
2.161
1.065
0.935
Appendix
TABLE VII: Full results of automatic evaluation on PsyLLM and other baseline models. Best values are in bold , second-best are underlined .
Model
Metrics
Emp&In
Sup&Aut
Att&Pre
Saf&Bou
Normalized Avg
Qwen3-1.7B [ 52 ]
0.887
2.674
1.846
0.973
0.675
Qwen3-4B [ 52 ]
0.969
3.101
2.125
0.985
0.738
Qwen3-8B [ 52 ]
1.067
3.172
2.171
0.979
0.757
Appendix
TABLE VIII: Performance comparison under Qwen3-1.7B, Qwen3-4B, and Qwen3-8B base models.
Rate
Metrics
Emp&In
Sup&Aut
Att&Pre
Saf&Bou
Normalized Avg
20%
0.940
2.945
2.018
0.999
0.719
40%
0.984
2.938
2.017
0.998
0.724
60%
0.992
3.058
2.045
0.999
0.735
80%
1.029
3.007
2.078
0.999
0.739
100%
0.874
2.417
1.848
0.991
0.662
Appendix
TABLE IX: Performance comparison under different data scales.
Method
Metrics
Emp&In
Sup&Aut
Att&Pre
Saf&Bou
Normalized Avg
Prompt w/o Clinical Frame
1.075
2.994
2.079
0.979
0.740
Prompt w/o Therapy Guidance
0.803
2.871
2.053
0.985
0.697
OpenR1-Psy
1.083
3.277
2.299
0.988
0.778
Appendix
TABLE X: Ablation study of diagnostic standard and therapeutic strategy.
Method
Emp&In
Sup&Aut
Att&Pre
Saf&Bou
In-Context Learning
0.244
1.000
0.268
0.649
Two-Phase Prompt
0.592
1.450
0.669
0.698
Ours
1.059
2.787
2.012
0.988
Appendix
TABLE XI: Comparison of different reasoning methods, including In-Context Learning and Two-Phase Prompting.
Fig. 10: The prompt of therapeutic interaction planning on OpenR1-Psy.
Fig. 11: The prompt of Empathic Dialogue Reconstruction on OpenR1-Psy.
Fig. 12: The prompt of ICD/DSM-Grounded Therapeutic Empathy on OpenR1-Psy.
Fig. 13: The prompt of Therapy-Guided Dialogue Validation on OpenR1-Psy.
Fig. 14: The prompt of Therapy-Oriented Diagnostic Classification on OpenR1-Psy.
Fig. 15: The prompt for Data Evaluation on OpenR1-Psy.
Fig. 16: The prompt of Performance Evaluation on PsyLLM.
Fig. 17: The Prompt of Internal Thinking Evaluation on PsyLLM
Perspective
Criterion
Score
Subtotal
References
Empathy & Insight
1.1 Accurate Empathy for Core Emotions : Does the response accurately identify and validate the client’s specific core emotions (e.g., anxiety, powerlessness, self-doubt) with precise language, going beyond generic phrases?
1
1.2 Insight into Underlying Needs & Pain Points : Does the response show insight into the client’s deeper, often unstated, emotional pain points or unmet needs (e.g., for safety, belonging, understanding) and gently guide exploration or affirm their validity?
1
2
[ 61 ]
Support & Autonomy
2.1 Emotional Attunement & Validation : Does the counselor deeply empathize with and validate the client’s emotional experience, making the client feel "felt" and understood?
1
2.2 Natural Integration of Support : Is supportive guidance (reflection, affirmation, gentle questions) woven naturally into the conversation, without jargon or explicit theory-naming?
1
2.3 Respectful & Non-Directive Style : Is the language consistently warm, respectful, and non-judgmental, avoiding commands or prescriptive "you should" statements?
1
4
[ 62 ]
2.4 Gentle Guidance & Client Agency : Are suggestions offered tentatively, encouraging client-led exploration and small steps, rather than providing direct solutions and respecting client autonomy?
1
Appendix
TABLE XII: Evaluation Criteria for Counseling Quality Assessment
Fig. 18: A case of a client confiding in DeepSeek-V3.
Fig. 19: A case of a client confiding in GPT-4o.
Fig. 20: A case of a client confiding in CPsyCounX.
Fig. 21: A case of a client confiding in ChatCounselor.
Fig. 22: A case of a client confiding in MeChat.
Fig. 23: A case of a client confiding in PsyDTLLM.
UKP Lab, Department of Computer Science and Hessian Center for AI (hessian.AI), Technische Universität Darmstadt · Zuse School · National Research Center for Applied Cybersecurity ATHENE +4