cs.CLFeb 24, 2026
SaveCARE: An Explainable Computational Framework for Assessing Client-Perceived Therapeutic Alliance Using Large Language Models
Organizations: Zhejiang University · Westlake University · Department of Psychology, Jing Hengyi School of Education, Hangzhou Normal University
Abstract
Client perceptions of the therapeutic alliance are critical for counseling effectiveness. Accurately capturing these perceptions remains challenging, as traditional post-session questionnaires are burdensome and often delayed, while existing computational approaches produce coarse scores, lack interpretable rationales, and fail to model holistic session context. We present CARE, an LLM-based framework to automatically predict multi-dimensional alliance scores and generate interpretable rationales from counseling transcripts. Built on the CounselingWAI dataset and enriched with 9,516 expert-curated rationales, CARE is fine-tuned using rationale-augmented supervision with the LLaMA-3.1-8B-Instruct backbone. Experiments show that CARE outperforms leading LLMs and substantially reduces the gap between counselor evaluations and client-perceived alliance, achieving over 70% higher Pearson correlation with client ratings. Rationale-augmented supervision further improves predictive accuracy. CARE also produces high-quality, contextually grounded rationales, validated by both automatic and human evaluations. Applied to real-world Chinese online counseling sessions, CARE uncovers common alliance-building challenges, illustrates how interaction patterns shape alliance development, and provides actionable insights, demonstrating its potential as an AI-assisted tool for supporting mental health care.
Figures & tables
Figure 1: The CARE model predicts client-perceived fine-grained working alliance scores after counseling sessions by identifying and summarizing explanatory reasons from the dialogue, compared to the traditional method where clients only provide scores through questionnaires.
| Fold | #Client | #Session | Goal | Task | Bond |
| 1 | 15 | 147 | |||
| 2 | 16 | 154 | |||
| 3 | 16 | 157 | |||
| 4 | 18 | 171 | |||
| 5 | 17 | 164 | |||
| Total | 82 | 793 |
Table 1: Summary statistics for each fold, including the number of clients, number of sessions, and the mean with standard deviation of goal, task, and bond scores.
| Model Name | Goal | Task | Bond | ||||||
| Pearson | Spearman | MSE | Pearson | Spearman | MSE | Pearson | Spearman | MSE | |
| Human Counselor | 0.30 | 0.27 | 1.36 | 0.30 | 0.28 | 1.61 | 0.22 | 0.21 | 1.24 |
| GPT-3.5-Turbo | |||||||||
| GPT-4o-mini | |||||||||
| GPT-4o | |||||||||
| Claude-3-Sonnet | |||||||||
Table 2: Performance of different models across the three alliance dimensions, including human counselors ratings, baseline models, and the proposed CARE model, benchmarked against clients’ self-reported ratings. Evaluation metrics include the Pearson’s , Spearman’s , and MSE. / indicates that higher/lower values are better. The best results in each column are shown in bold.
| Metrics | Goal | Task | Bond | |
| Automatic | BLEU | |||
| ROUGE-1 | ||||
| ROUGE-L | ||||
| BERTScore | ||||
| Human | Faithfulness | |||
| Relevance | ||||
Table 3: Comparison of CARE-generated rationales and expert-written rationales in terms of automatic evaluation metrics (BLEU, ROUGE-1, ROUGE-L, and BERTScore) and human assessment dimensions (Faithfulness, Relevance, and Informativeness).
| Architectures | Rationale | Goal | Task | Bond | ||||||
| Pearson | Spearman | MSE | Pearson | Spearman | MSE | Pearson | Spearman | MSE | ||
| Qwen2.5-7B-Instruct | ||||||||||
| Llama-3.1-8B-Instruct | ||||||||||
Table 4: Results of generalization and ablation studies for fine-tuning with versus without rationales, using Qwen2.5-7B-Instruct and LLaMA-3.1-8B-Instruct as backbones. Evaluation metrics include the Pearson’s , Spearman’s , and MSE. / indicates that higher/lower values are better. The best results in each column are shown in bold.
Figure 2: Working alliance scores by dimension for the three most active counselors; dashed lines indicate overall counselor averages.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Distributions of client ratings across the three working alliance dimensions—Goal, Task, and Bond—illustrated using histograms, box plots, and half-violin plots. Within each box, the X denotes the mean. The central line represents the median, the box corresponds to the interquartile range (IQR, 25th–75th percentile), and the whiskers extend to the full range of the data.
| Dim | Length | Lexical Features |
| Goal | 目标/goal (55.32), 努力/effort (29.31), 探讨/explore (27.57), 改善/improve (24.97), 制定/establish (24.38) | |
| Task | 方法/method (41.73), 意识/aware (40.05), 新/new (37.80), 改变/change (29.41) 解决问题/problem-solving (29.70) | |
| Bond | 支持/support (45.51), 关心/care (38.76), 表现/express (37.59), 感受/feeling (37.05), 理解/understand (36.54) |
Table 5: Data characteristics of generated rationales for each working alliance dimension, including the mean with standard deviation of rationale length and key lexical features. The rightmost column presents the five words most strongly associated with each dimension’s rationales, with rounded z-scored log-odds ratios provided in parentheses.
| Conversation | Evaluation Results |
| Counselor: Hello, are you online? 10 minutes later… Client: Sorry, I fell asleep. Counselor: It’s okay, let’s begin now. Client: Thanks for waiting for me. Counseling in progress… Counselor: Do you have any plans for progress? Counselor: Are you still online? Hello? | Dimension : Bond Client : 1.25 CARE : 4.25 Explanations : The counselor patiently waits for the client’s responses, even when the client falls asleep, without showing any impatience. The client also responds in a friendly manner, such as saying "Sorry" and "Thanks". These interactions reflect mutual respect and understanding… |
| Counseling in progress… Client: I’ve been doing counseling for a while, and slowly I’ve started to shift my focus. I’m feeling a bit better now. Counseling in progress… Counselor: You don’t want to have a negative impact on her? Client: Yes. Counselor: You don’t want to put her in a diff- icult position? Client: I guess so. Counseling in progress… | Dimension : Task Client : 1.0 CARE : 4.0 Explanations : The client directly stated that counseling has helped them clarify the areas that need change, and feel better by shifting focus… |
Table 6: Example cases where the model’s predictions significantly deviate from the client’s self-reported scores.
Figure 4: The template prompt for instructing LLMs to predict clients’ perceived working alliance, using an example conversation and questionnaire question (displayed in cadetblue italic text).
| Hyperparameters | Value |
| Per-device Train Batchsize | 1 |
| Gradient Accumulation Steps | 2 |
| Warmup Ratio | 0.1 |
| LR Scheduler Type | cosine |
| Learning Rate | 5e-7 |
| Data Type | bfloat16 |
Table 7: The hyperparameters with values used in our fine-tuning experiments.
| Goal | Task | Bond | |
| Supporting - Positive | 0.17 | 0.84 | 0.62 |
| Challenging - Positive | 2.07*** | 2.64*** | 1.47*** |
| Supporting - Negative | -5.71*** | -5.57*** | -6.36*** |
| Challenging - Negative | -1.70 | -2.74 | -1.77 |
Table 8: Regression coefficients for counselor–client interaction patterns predicting session-level working alliance scores across the three dimensions; *** denotes p < 0.001.
| Dimension | Evaluation | Example |
| Goal | bad | In our conversations, although the counselor tried to guide me in exploring issues and feelings, we did not clearly agree on specific areas for change. For example, when I mentioned how to deal with a difficult roommate, the counselor focused more on understanding my feelings and situation rather than collaboratively determining specific change strategies. My responses, such as “No thanks” and later mentioning “I might not be suited for this,” indicate that my understanding of the counseling process may differ from the counselor’s expectations. As a result, we rarely reached a consensus on the areas where I needed to make efforts for change. |
| good | In the conversation, the counselor and I jointly established counseling goals, such as, “Let’s set the goal to increase daily sleep time to 6 hours.” This indicates that we were indeed actively involved in the goal-setting process, which was a continuous part of the counseling sessions, thus aligning with the description of “always” collaboratively setting goals. | |
| Approach | bad | In the conversation, I expressed my feelings about the counselor’s questions, saying, “To be honest, I speak directly and bluntly, and this is a free service. I shouldn’t have any expectations. I really shouldn’t have too many demands and should be content.” This indicates that I did not feel the counseling provided new perspectives or methods for addressing my relationship issues. The counselor focused more on listening and understanding, whereas I was expecting concrete guidance and solutions, such as, “Shouldn’t you be better and have more ideas?” This shows that I didn’t frequently find new ways to view my problems during the counseling sessions. |
| good | In the conversation, I mentioned trying to make changes but not finding the right approach, which indicates that I recognize the value of the counseling process. The counselor helped me reflect on my concerns about effort and boundaries, for example, “It seems like you feel that the evaluation of your goodness comes from what you do rather than your traits or personal charisma in relationships.” This helped me start to understand my behavioral patterns more deeply. Although I am still exploring, I feel that counseling has indeed guided me toward the changes I want. | |
| Bond | bad | In the conversation, I expressed doubts about the counselor’s professionalism and mentioned, “You gave me the impression of being a bit rushed, not allowing me time to process my changes,” as well as “I have a lot of anger towards you.” This indicates that I did not feel that the counselor was attuned to my personal preferences but rather sensed a mismatch in professional approach. |
| good | During the counseling process, the counselor demonstrated patience and understanding, such as by asking, “Can you describe the sense of presence you have experienced from childhood to now?” This indicates that the counselor actively listens and is concerned about my feelings. Although personal preferences were not directly addressed, the counselor’s positive feedback and in-depth exploration, such as, “The sense of presence you described—when you feel that doing something genuinely brings joy to others—seems to be part of your value system. It looks like you have a direction, but the path is blocked,” show the counselor’s deep understanding and care for me, making me feel valued and supported. |
Table 9: Example explanations generated by our best-performing model.