Large Language Model (LLM)-based counseling agents can generate fluent and supportive responses, but they often lack the structured, goal-directed progression required to conduct a coherent therapeutic session. We present DeepSAGE (Strategic AI Guidance Engine), a hybrid LLM--Deep Reinforcement Learning (DRL) framework for stage-aware counseling dialogue grounded in the first session of Cognitive Behavioral Therapy (CBT). DeepSAGE represents the session as eleven stages with explicit therapeutic objectives, with an external controller determines stage completion and the DRL model selects therapeutic intentions that guide LLM response generation. We evaluate DeepSAGE against six retrieval-, prompting-, stage-, and policy-based alternatives. DeepSAGE elicits higher simulated client engagement and openness and achieves the strongest balance of stage-goal completion and dialogue efficiency among stage-structured systems. Domain expert review further indicates that the generated conversations exhibit broadly plausible emotional trajectories and recognizable CBT processes. Because the evaluation relies primarily on simulated clients and model-based metrics, these findings demonstrate comparative dialogue-control improvements rather than clinical effectiveness. These results suggest that combining stage-structured dialogue with learned strategy selection is a promising approach for AI counseling, though clinical effectiveness, safety, and real-world utility require further human evaluation.
Figures & tables
Figure 1: Stage-aware dialogue control in DeepSAGE . Given a client response rt at stage Si , the system first evaluates whether the stage goal is satisfied using G(Si,rt) . If so, it transitions to Si+1 . Otherwise, it checks whether the number of dialogue turns reaches the limit Tt≥Tmax(Si) ; if so, it also advances to Si+1 . If neither condition is met, an RL policy π(at∣st) selects a therapeutic intention at based on the state st=(ht,ct) , where ht encodes dialogue history and ct captures stage and turn information. The selected intent guides the LLM to generate a natural language response, steering the conversation toward the stage objective.
Param.
Meaning
Default
N
Number of CBT session stages
11
Tmax,i
Maximum conversation turns in stage i
Varies
α
Weight in reward function (Eq. ( 7 ))
0.5
n
Number of recent utterances encoded in RL state
4
λ
Penalty coefficient for stage length control
0.001
τ
Threshold for stage transition
0.8
Table 1: Key Parameters and Default Values
Scheme
Counseling Prompt
CBT Knowledge
Protocol Visibility
External Stage Controller
Therapeutic-Intent Policy
Learned Policy
Naïve LLM-Bot
General
✗
✗
✗
✗
✗
RAB
General
Retrieved
✗
✗
✗
✗
LLM4CBT
CBT-oriented
Prompt-based
Partial
✗
✗
✗
Full-Protocol Prompt LLM
CBT-oriented
Prompt-based
Full
✗
✗
✗
Stage-Prompt LLM
Stage-specific
Stage-specific
Current stage
✓
✗
✗
DeepSAGE_R
Stage-specific
Stage-specific
Current stage
✓
✓
✗
Table 2: System-design comparison of the six counseling schemes.
AD
MDD
Schemes
UL
SDC
EID
UL
SDC
EID
RAB
0.4573 ± 0.0665
0.4229 ± 0.1063
0.5767 ± 0.1366
0.2888 ± 0.0689
0.2500 ± 0.0781
0.6531 ± 0.2767
Naïve LLM
0.5008 ± 0.0816
0.3646 ± 0.1270
0.5535 ± 0.3998
0.3206 ± 0.0518
0.2646 ± 0.0937
0.9259 ± 0.0811
LLM4CBT
0.5555 ± 0.1221
0.4504 ± 0.1426
0.6985 ± 0.0301
0.3965 ± 0.0868
0.3373 ± 0.1139
0.6418 ± 0.1618
Full-Protocol
0.4997 ± 0.0875
0.4308 ± 0.1543
0.4869 ± 0.4869
0.3578 ± 0.0635
0.2842 ± 0.1179
0.7625 ± 0.0027
Stage-Prompt
0.5461 ± 0.1204
0.4871 ± 0.1671
0.5812 ± 0.4047
0.3529 ± 0.0783
0.3087 ± 0.1296
0.3839 ± 0.0672
Table 3: Engagement and simulated distress-change results for anxiety disorder (AD) and major depressive disorder (MDD) clients. Higher is better.
AD
MDD
Scheme
SSR
# Utt.
SSR
# Utt.
Full-Protocol
0.4792±0.1429
50.00
0.3834±0.0122
50.00
Stage-Prompt
0.7821±0.0465
64.50
0.7652±0.0130
66.00
DeepSAGE_R
0.7657±0.0453
50.85
0.7733±0.0396
51.20
DeepSAGE (Ours)
0.8494±0.0350
46.10
0.8400±0.0345
47.60
Table 4: Session completion and dialogue efficiency for stage-structured schemes. Higher is better for SSR, while lower is better for the number of counselor utterances.
Effective mental health counseling is a complex, theory-driven process requiring the simultaneous integration of psychological frameworks, real-time distress signals, and strategic intervention planning. This level of clinical reasoning is critical for safety and therapeutic effectiveness but is often missing in general-purpose Large Language Models (LLMs). We introduce SAGE (Strategy-Aware Graph-Enhanced), a novel framework designed to bridge the gap between structured clinical knowledge and generative AI. SAGE constructs a heterogeneous graph that unifies conversational dynamics with a psychologically grounded layer, explicitly anchoring interactions in a theory-driven lexicon. Our architecture first employs a Next Strategy Classifier to identify the optimal therapeutic intervention. Subsequently, a Graph-Aware Attention mechanism projects graph-derived structural signals into soft prompts, conditioning the LLM to generate responses that maintain clinical depth. Validated through both automated metrics and expert human evaluation, SAGE outperforms baselines in strategy prediction and recommended response quality. By providing actionable intervention recommendations, SAGE serves as a cutting-edge decision-support tool designed to augment human expertise in high-stakes crisis counseling.
Eliya Naomi Aharon, Meytal Grimland, Avi Segal +4
Ben-Gurion University of the Negev Beer Sheva, Israel · University of Haifa Haifa, Israel · Sahar Tel Aviv, Israel
Cognitive Behavioral Therapy (CBT) aims to identify and restructure automatic negative thoughts pertaining to involuntary interpretations of events, yet existing counseling agents struggle to identify and address them in dialogue settings. To bridge this gap, we introduce STEP, a dataset that models CBT counseling by explicitly reflecting automatic thoughts alongside dynamic, action-level counseling sequences. Using this dataset, we train STEPPER, a counseling agent that proactively elicits automatic thoughts and executes cognitively grounded interventions. To further enhance both decision accuracy and empathic responsiveness, we refine STEPPER through preference learning based on simulated, synthesized counseling sessions. Extensive CBT-aligned evaluations show that STEPPER delivers more clinically grounded, coherent, and personalized counseling compared to other strong baseline models, and achieves higher counselor competence without inducing emotional disruption.
Jihyun Lee, Yejin Min, Yejin Jeon +3
Graduate School of Artificial Intelligence, POSTECH · MILA · McGill University +1
Large language models show potential for scalable mental-health support by simulating Cognitive Behavioral Therapy (CBT) counselors. However, existing methods often rely on static cognitive profiles and omniscient single-agent simulation, failing to capture the dynamic, information-asymmetric nature of real therapy. We introduce CCD-CBT, a multi-agent framework that shifts CBT simulation along two axes: 1) from a static to a dynamically reconstructed Cognitive Conceptualization Diagram (CCD), updated by a dedicated Control Agent, and 2) from omniscient to information-asymmetric interaction, where the Therapist Agent must reason from inferred client states. We release CCDCHAT, a synthetic multi-turn CBT dataset generated under this framework. Evaluations with clinical scales and expert therapists show that models fine-tuned on CCDCHAT outperform strong baselines in both counseling fidelity and positive-affect enhancement, with ablations confirming the necessity of dynamic CCD guidance and asymmetric agent design. Our work offers a new paradigm for building theory-grounded, clinically-plausible conversational agents.
Chang Liu, Changsheng Ma, Yongfeng Tao +2
School of Information Science and Engineering, Lanzhou University