Multi-dimensional Assessment and Explainable Feedback for Counselor Responses to Client Resistance in Text-based Counseling with LLMs

Anqi Li; Ruihan Wang; Zhaoming Chen; Yuqian Chen; Yu Lu; Yi Zhu; Yuan Xie; Zhenzhong Lan

cs.CLFeb 25, 2026

Multi-dimensional Assessment and Explainable Feedback for Counselor Responses to Client Resistance in Text-based Counseling with LLMs

Authors: Anqi Li, Ruihan Wang, Zhaoming Chen, Yuqian Chen, Yu Lu, Yi Zhu, Yuan Xie, Zhenzhong Lan

Organizations: Zhejiang University · Westlake University, Hangzhou, China

Abstract

Effectively addressing client resistance is a sophisticated clinical skill in psychological counseling, yet practitioners often lack timely and scalable supervisory feedback to refine their approaches. Although current NLP research has examined overall counseling quality and general therapeutic skills, it fails to provide granular evaluations of high-stakes moments where clients exhibit resistance. In this work, we present a comprehensive pipeline for the multi-dimensional evaluation of human counselors' interventions specifically targeting client resistance in text-based therapy. We introduce a theory-driven framework that decomposes counselor responses into four distinct communication mechanisms. Leveraging this framework, we curate and share an expert-annotated dataset of real-world counseling excerpts, pairing counselor-client interactions with professional ratings and explanatory rationales. Using this data, we perform full-parameter instruction tuning on a Llama-3.1-8B-Instruct backbone to model fine-grained evaluative judgments of response quality and generate explanations underlying. Experimental results show that our approach can effectively distinguish the quality of different communication mechanisms (77-81% F1), substantially outperforming GPT-4o and Claude-3.5-Sonnet (45-59% F1). Moreover, the model produces high-quality explanations that closely align with expert references and receive near-ceiling ratings from human experts (2.8-2.9/3.0). A controlled experiment with 43 counselors further confirms that receiving these AI-generated feedback significantly improves counselors' ability to respond effectively to client resistance.

Figures & tables

Fig. 1: Overview of our framework for evaluating counselor responses to client resistance. The framework comprises four core communication mechanisms: Respect for Autonomy , Stance Alignment , Emotional Resonance , and Conversational Orientation . For each mechanism, responses are further categorized into three levels of expression: no expression, weak expression, and strong expression. Our computational approach simultaneously identifies the strength of these dimensions along with their underlying explanations.

Dimension	No	Weak	Strong	Total
Respect for Autonomy	561	2761	514	3836
Stance Alignment	445	2654	737	3836
Emotional Resonance	1299	2262	275	3836
Conversation Orientation	1561	1813	462	3836

TABLE I: Statistics of the collected resistance response evaluation dataset.

Model Name	Respect for Autonomy		Stance Alignment		Emotional Resonance		Conversational Orientation
Model Name	F1	Acc.	F1	Acc.	F1	Acc.	F1	Acc.
Claude-3.5-Sonnet	$41.61\pm 0.99$	$57.06\pm 1.56$	$52.59\pm 1.41$	$53.10\pm 1.09$	$55.59\pm 1.44$	$60.77\pm 1.21$	$45.36\pm 1.62$	$45.13\pm 1.97$
GPT-4o	$45.37\pm 1.33$	$53.31\pm 1.38$	$58.61\pm 1.99$	$63.87\pm 1.29$	$53.16\pm 1.05$	$61.52\pm 1.21$	$41.72\pm 0.63$	$44.26\pm 0.48$
Qwen2.5-7B-Instruct	$27.92\pm 1.57$	$34.91\pm 1.56$	$22.97\pm 1.54$	$25.34\pm 1.18$	$29.34\pm 1.35$	$39.49\pm 1.69$	$27.33\pm 1.21$	$30.63\pm 1.61$
Qwen2.5-14B-Instruct	$33.43\pm 1.71$	$32.20\pm 0.64$	$32.97\pm 1.29$	$43.35\pm 0.93$	$30.04\pm 1.38$	$37.38\pm 2.06$	$29.75\pm 1.04$	$29.67\pm 1.17$
Qwen2.5-32B-Instruct	$34.72\pm 1.43$	$44.24\pm 1.83$	$37.73\pm 1.42$	$47.08\pm 1.26$	$33.32\pm 1.43$	$38.35\pm 1.15$	$30.01\pm 0.91$	$29.64\pm 0.81$

TABLE II: Comparison of classification performance between the baseline model and our model on the four communication mechanisms. Results are reported in terms of macro-F1 and accuracy, with the best-performing model in each column shown in bold.

Model Name	Automatic Evaluation					Human Evaluation
Model Name	BLEU-1	BLEU-2	Rouge-1	Rouge-2	Rouge-L	Framework Consistency	Evidence Anchoring	Clarity & Specificity
Claude-3.5-Sonnet	$32.45\pm 0.39$	$20.45\pm 0.18$	$47.58\pm 0.08$	$19.02\pm 0.11$	$29.14\pm 0.05$	$1.94\pm 0.73$	$1.94\pm 0.73$	$2.18\pm 0.48$
GPT-4o	$24.87\pm 0.51$	$14.23\pm 0.26$	$42.46\pm 0.16$	$14.70\pm 0.14$	$25.13\pm 0.09$	$2.04\pm 0.82$	$2.00\pm 0.80$	$2.40\pm 0.49$
Qwen2.5-7B-Instruct	$17.94\pm 0.61$	$9.99\pm 0.31$	$39.20\pm 0.09$	$12.46\pm 0.07$	$21.69\pm 0.15$	$1.90\pm 0.73$	$1.90\pm 0.73$	$2.26\pm 0.48$
Qwen2.5-14B-Instruct	$6.22\pm 0.67$	$3.45\pm 0.37$	$30.82\pm 0.38$	$8.58\pm 0.18$	$15.85\pm 0.24$	$2.08\pm 0.66$	$2.00\pm 0.60$	$2.32\pm 0.47$
Qwen2.5-32B-Instruct	$15.52\pm 0.20$	$9.00\pm 0.11$	$40.28\pm 0.16$	$13.54\pm 0.06$	$22.65\pm 0.07$	$2.22\pm 0.58$	$2.18\pm 0.55$	$2.34\pm 0.47$

TABLE III: Comparison of explanation generation performance between the baseline model and our model. Results are reported in terms of BlEU-1/2, Rouge-1/2/L, and human evaluations, with the best-performing model in each column shown in bold.

Fig. 2: Interaction effects between experimental groups and phases across four dimensions. Solid green lines represent the control group, while dashed orange lines represent the experimental group. Points denote the mean values, and error bars indicate 95% confidence intervals. The results reveal a significant performance surge in the Experimental Group across all dimensions following the intervention, whereas the Control Group maintains a stable baseline.

Explore similar work

Jun 3, 2026

When Clients Stop Following: A Cognitive Conceptualization Diagram-driven Framework for Strategic Counseling

Large Language Models (LLMs) show promise in psychological counseling, yet existing benchmarks rely heavily on highly cooperative simulated clients. We observe a critical counselor-following phenomenon: these clients often rapidly shift from resistance to compliance after only a few turns, creating an illusion of therapeutic progress and inflating scores under current evaluation protocols through superficial empathy. To address this evaluation mismatch, we propose a Cognitive Behavioral Therapy (CBT)-grounded resistance-aware framework. We introduce CARS, a client simulator that explicitly models dynamic resistance via Cognitive Conceptualization Diagrams (CCDs). We present STREAMS, a dual-module framework that decouples strategic reasoning (Thinker) from response generation (Presenter) and optimizes it via reinforcement learning. We further propose EWTS-MI, an entropy-weighted metric for evaluating responsiveness under high-friction interactions. Experiments across resistant and non-resistant counseling settings validate our findings on evaluation mismatch and demonstrate the effectiveness of resistance-aware training for improving strategic robustness under challenging counseling interactions.

Yihao Qin, Junyi Zhao, Changsheng Ma +4

School of Information Science and Engineering, Lanzhou University

Mental Health Counseling Cognitive Modeling Clinical Language Model Evaluation User Simulation Cognitive Behavioral Therapy

Ask PDF

Feb 24, 2026

CARE: An Explainable Computational Framework for Assessing Client-Perceived Therapeutic Alliance Using Large Language Models

Client perceptions of the therapeutic alliance are critical for counseling effectiveness. Accurately capturing these perceptions remains challenging, as traditional post-session questionnaires are burdensome and often delayed, while existing computational approaches produce coarse scores, lack interpretable rationales, and fail to model holistic session context. We present CARE, an LLM-based framework to automatically predict multi-dimensional alliance scores and generate interpretable rationales from counseling transcripts. Built on the CounselingWAI dataset and enriched with 9,516 expert-curated rationales, CARE is fine-tuned using rationale-augmented supervision with the LLaMA-3.1-8B-Instruct backbone. Experiments show that CARE outperforms leading LLMs and substantially reduces the gap between counselor evaluations and client-perceived alliance, achieving over 70% higher Pearson correlation with client ratings. Rationale-augmented supervision further improves predictive accuracy. CARE also produces high-quality, contextually grounded rationales, validated by both automatic and human evaluations. Applied to real-world Chinese online counseling sessions, CARE uncovers common alliance-building challenges, illustrates how interaction patterns shape alliance development, and provides actionable insights, demonstrating its potential as an AI-assisted tool for supporting mental health care.

Anqi Li, Chenxiao Wang, Yu Lu +3

Zhejiang University · Westlake University · Department of Psychology, Jing Hengyi School of Education, Hangzhou Normal University

Mental Health Counseling

Ask PDF

Apr 12, 2026

Beyond Compliance: A Resistance-Informed Motivation Reasoning Framework for Challenging Psychological Client Simulation

Psychological client simulators have emerged as a scalable solution for training and evaluating counselor trainees and psychological LLMs. Yet existing simulators exhibit unrealistic over-compliance, leaving counselors underprepared for the challenging behaviors common in real-world practice. To bridge this gap, we present ResistClient, which systematically models challenging client behaviors grounded in Client Resistance Theory by integrating external behaviors with underlying motivational mechanisms. To this end, we propose Resistance-Informed Motivation Reasoning (RIMR), a two-stage training framework. First, RIMR mitigates compliance bias via supervised fine-tuning on RPC, a large-scale resistance-oriented psychological conversation dataset covering diverse client profiles. Second, beyond surface-level response imitation, RIMR models psychologically coherent motivation reasoning before response generation, jointly optimizing motivation authenticity and response consistency via process-supervised reinforcement learning. Extensive automatic and expert evaluations show that ResistClient substantially outperforms existing simulators in challenge fidelity, behavioral plausibility, and reasoning coherence. Moreover, ResistClient facilities evaluation of psychological LLMs under challenging conditions, offering new optimization directions for mental health dialogue systems.

Danni Liu, Bo Liu, Yuxin Hu +5

Southeast University, Nanjing, China · Purple Mountain Laboratories, Nanjing, China · The Nanjing Derong Wisdom Information Technology Co., Ltd., Nanjing, China

Large Language Model-Based Role-Play Simulation

Ask PDF