Multi-dimensional Assessment and Explainable Feedback for Counselor Responses to Client Resistance in Text-based Counseling with LLMs

Anqi Li; Ruihan Wang; Zhaoming Chen; Yuqian Chen; Yu Lu; Yi Zhu; Yuan Xie; Zhenzhong Lan

cs.CLFeb 25, 2026

Multi-dimensional Assessment and Explainable Feedback for Counselor Responses to Client Resistance in Text-based Counseling with LLMs

Authors: Anqi Li, Ruihan Wang, Zhaoming Chen, Yuqian Chen, Yu Lu, Yi Zhu, Yuan Xie, Zhenzhong Lan

Organizations: Zhejiang University · Westlake University, Hangzhou, China

Abstract

Effectively addressing client resistance is a sophisticated clinical skill in psychological counseling, yet practitioners often lack timely and scalable supervisory feedback to refine their approaches. Although current NLP research has examined overall counseling quality and general therapeutic skills, it fails to provide granular evaluations of high-stakes moments where clients exhibit resistance. In this work, we present a comprehensive pipeline for the multi-dimensional evaluation of human counselors' interventions specifically targeting client resistance in text-based therapy. We introduce a theory-driven framework that decomposes counselor responses into four distinct communication mechanisms. Leveraging this framework, we curate and share an expert-annotated dataset of real-world counseling excerpts, pairing counselor-client interactions with professional ratings and explanatory rationales. Using this data, we perform full-parameter instruction tuning on a Llama-3.1-8B-Instruct backbone to model fine-grained evaluative judgments of response quality and generate explanations underlying. Experimental results show that our approach can effectively distinguish the quality of different communication mechanisms (77-81% F1), substantially outperforming GPT-4o and Claude-3.5-Sonnet (45-59% F1). Moreover, the model produces high-quality explanations that closely align with expert references and receive near-ceiling ratings from human experts (2.8-2.9/3.0). A controlled experiment with 43 counselors further confirms that receiving these AI-generated feedback significantly improves counselors' ability to respond effectively to client resistance.

Figures & tables

Fig. 1: Overview of our framework for evaluating counselor responses to client resistance. The framework comprises four core communication mechanisms: Respect for Autonomy , Stance Alignment , Emotional Resonance , and Conversational Orientation . For each mechanism, responses are further categorized into three levels of expression: no expression, weak expression, and strong expression. Our computational approach simultaneously identifies the strength of these dimensions along with their underlying explanations.

Dimension	No	Weak	Strong	Total
Respect for Autonomy	561	2761	514	3836
Stance Alignment	445	2654	737	3836
Emotional Resonance	1299	2262	275	3836
Conversation Orientation	1561	1813	462	3836

TABLE I: Statistics of the collected resistance response evaluation dataset.

Model Name	Respect for Autonomy		Stance Alignment		Emotional Resonance		Conversational Orientation
Model Name	F1	Acc.	F1	Acc.	F1	Acc.	F1	Acc.
Claude-3.5-Sonnet	$41.61\pm 0.99$	$57.06\pm 1.56$	$52.59\pm 1.41$	$53.10\pm 1.09$	$55.59\pm 1.44$	$60.77\pm 1.21$	$45.36\pm 1.62$	$45.13\pm 1.97$
GPT-4o	$45.37\pm 1.33$	$53.31\pm 1.38$	$58.61\pm 1.99$	$63.87\pm 1.29$	$53.16\pm 1.05$	$61.52\pm 1.21$	$41.72\pm 0.63$	$44.26\pm 0.48$
Qwen2.5-7B-Instruct	$27.92\pm 1.57$	$34.91\pm 1.56$	$22.97\pm 1.54$	$25.34\pm 1.18$	$29.34\pm 1.35$	$39.49\pm 1.69$	$27.33\pm 1.21$	$30.63\pm 1.61$
Qwen2.5-14B-Instruct	$33.43\pm 1.71$	$32.20\pm 0.64$	$32.97\pm 1.29$	$43.35\pm 0.93$	$30.04\pm 1.38$	$37.38\pm 2.06$	$29.75\pm 1.04$	$29.67\pm 1.17$
Qwen2.5-32B-Instruct	$34.72\pm 1.43$	$44.24\pm 1.83$	$37.73\pm 1.42$	$47.08\pm 1.26$	$33.32\pm 1.43$	$38.35\pm 1.15$	$30.01\pm 0.91$	$29.64\pm 0.81$

TABLE II: Comparison of classification performance between the baseline model and our model on the four communication mechanisms. Results are reported in terms of macro-F1 and accuracy, with the best-performing model in each column shown in bold.

Model Name	Automatic Evaluation					Human Evaluation
Model Name	BLEU-1	BLEU-2	Rouge-1	Rouge-2	Rouge-L	Framework Consistency	Evidence Anchoring	Clarity & Specificity
Claude-3.5-Sonnet	$32.45\pm 0.39$	$20.45\pm 0.18$	$47.58\pm 0.08$	$19.02\pm 0.11$	$29.14\pm 0.05$	$1.94\pm 0.73$	$1.94\pm 0.73$	$2.18\pm 0.48$
GPT-4o	$24.87\pm 0.51$	$14.23\pm 0.26$	$42.46\pm 0.16$	$14.70\pm 0.14$	$25.13\pm 0.09$	$2.04\pm 0.82$	$2.00\pm 0.80$	$2.40\pm 0.49$
Qwen2.5-7B-Instruct	$17.94\pm 0.61$	$9.99\pm 0.31$	$39.20\pm 0.09$	$12.46\pm 0.07$	$21.69\pm 0.15$	$1.90\pm 0.73$	$1.90\pm 0.73$	$2.26\pm 0.48$
Qwen2.5-14B-Instruct	$6.22\pm 0.67$	$3.45\pm 0.37$	$30.82\pm 0.38$	$8.58\pm 0.18$	$15.85\pm 0.24$	$2.08\pm 0.66$	$2.00\pm 0.60$	$2.32\pm 0.47$
Qwen2.5-32B-Instruct	$15.52\pm 0.20$	$9.00\pm 0.11$	$40.28\pm 0.16$	$13.54\pm 0.06$	$22.65\pm 0.07$	$2.22\pm 0.58$	$2.18\pm 0.55$	$2.34\pm 0.47$

TABLE III: Comparison of explanation generation performance between the baseline model and our model. Results are reported in terms of BlEU-1/2, Rouge-1/2/L, and human evaluations, with the best-performing model in each column shown in bold.

Fig. 2: Interaction effects between experimental groups and phases across four dimensions. Solid green lines represent the control group, while dashed orange lines represent the experimental group. Points denote the mean values, and error bars indicate 95% confidence intervals. The results reveal a significant performance surge in the Experimental Group across all dimensions following the intervention, whereas the Control Group maintains a stable baseline.

Explore similar work

When Clients Stop Following: A Cognitive Conceptualization Diagram-driven Framework for Strategic Counseling
Jun 3, 2026Yihao Qin, Junyi Zhao, Changsheng Ma +4Mental Health Counseling Cognitive Modeling
School of Information Science and Engineering, Lanzhou University
CARE: An Explainable Computational Framework for Assessing Client-Perceived Therapeutic Alliance Using Large Language Models
Feb 24, 2026Anqi Li, Chenxiao Wang, Yu Lu +3Mental Health Counseling
Zhejiang University · Westlake University · Department of Psychology, Jing Hengyi School of Education, Hangzhou Normal University
Beyond Compliance: A Resistance-Informed Motivation Reasoning Framework for Challenging Psychological Client Simulation
Apr 12, 2026Danni Liu, Bo Liu, Yuxin Hu +5Large Language Model-Based Role-Play Simulation
Southeast University, Nanjing, China · Purple Mountain Laboratories, Nanjing, China · The Nanjing Derong Wisdom Information Technology Co., Ltd., Nanjing, China

Jun 3, 2026J/K move · Enter open · S save

When Clients Stop Following: A Cognitive Conceptualization Diagram-driven Framework for Strategic Counseling

Yihao Qin, Junyi Zhao, Changsheng Ma, Yongfeng Tao+3

School of Information Science and Engineering, Lanzhou University

Mental Health Counseling Cognitive Modeling Clinical Language Model Evaluation User Simulation Cognitive Behavioral Therapy

Open paper AskPDF arXiv

Large Language Models (LLMs) show promise in psychological counseling, yet existing benchmarks rely heavily on highly cooperative simulated clients. We observe a critical counselor-following phenomenon: these clients often rapidly shift from resistance to compliance after only a few turns, creating an illusion of therapeutic progress and inflating scores under current evaluation protocols through superficial empathy. To address this evaluation mismatch, we propose a Cognitive Behavioral Therapy (CBT)-grounded resistance-aware framework. We introduce CARS, a client simulator that explicitly models dynamic resistance via Cognitive Conceptualization Diagrams (CCDs). We present STREAMS, a dual-module framework that decouples strategic reasoning (Thinker) from response generation (Presenter) and optimizes it via reinforcement learning. We further propose EWTS-MI, an entropy-weighted metric for evaluating responsiveness under high-friction interactions. Experiments across resistant and non-resistant counseling settings validate our findings on evaluation mismatch and demonstrate the effectiveness of resistance-aware training for improving strategic robustness under challenging counseling interactions.

Abstract

Explore similar work

When Clients Stop Following: A Cognitive Conceptualization Diagram-driven Framework for Strategic Counseling

CARE: An Explainable Computational Framework for Assessing Client-Perceived Therapeutic Alliance Using Large Language Models

Beyond Compliance: A Resistance-Informed Motivation Reasoning Framework for Challenging Psychological Client Simulation