When Less Is More: An Empirical Study of Minimal Responses in Counseling Dialogues and the Behavior of LLMs

Zhiyang Qi

cs.CLAug 25, 2026

When Less Is More: An Empirical Study of Minimal Responses in Counseling Dialogues and the Behavior of LLMs

Authors: Zhiyang Qi

Organizations: The University of Tokyo

Abstract

In psychological counseling, effective support is not always delivered through long, information-rich responses. Minimal responses, such as backchannel cues and concise empathic statements, help convey attentive listening, express empathy, and encourage clients to continue expressing themselves. However, existing counseling dialogue systems and evaluation frameworks often favor explicit, content-rich replies, overlooking the interactional value of brief counselor utterances. This paper presents a systematic cross-lingual analysis of minimal responses across multiple counseling dialogue datasets. We develop a two-stage filtering method based on utterance length and content, followed by contextual verification using a large language model (LLM). Our analysis shows that minimal responses are common in human-collected datasets but substantially underrepresented in LLM-generated ones. We further evaluate current LLMs in manually curated dialogue contexts where human counselors used minimal responses. The results show that strong commercial LLMs are capable of generating minimal responses when explicitly instructed, but still struggle to determine when such responses are appropriate. Counseling-specific models trained on synthetic data perform particularly poorly, tending instead to produce longer and more information-rich responses. Moreover, LLM-based response-quality evaluation may undervalue minimal responses, even when they are interactionally appropriate.

Figures & tables

Figure 1: Minimal responses are a key technique in real counseling. However, current LLM-based evaluation methods (here using Zhang et al. (2024) ’s prompt with Gemini) tend to undervalue such responses due to their brevity, favoring more elaborate replies.

Dataset Statistics						(1) Rule-based Filtering	(2) LLM-based Contextual Verification
Dataset	Lang.	#Dial.	#Couns.	Avg. Utter. per Dialogue	#Counselor Utterances	Rule-filtered Candidates (# / %)	Minimal Responses		Other Short Responses (# / %)
Dataset	Lang.	#Dial.	#Couns.	Avg. Utter. per Dialogue	#Counselor Utterances	Rule-filtered Candidates (# / %)	Backchannel-like Responses (# / %)	Brief Empathic/ Reflective Responses (# / %)	Other Short Responses (# / %)
LLM-generated datasets
C actus Lee et al. (2024)	EN	31,577	–	30.53	491,304	4 (0.00%)	0 (0.00%)	3 (0.00%)	1 (0.00%)
CPsyCounD Zhang et al. (2024)	ZH	3,084	–	15.94	24,220	37 (0.15%)	0 (0.00%)	1 (0.00%)	36 (0.15%)
PsyDTCorpus Xie et al. (2025)	ZH	4,760	–	36.16	86,054	1 (0.00%)	0 (0.00%)	1 (0.00%)	0 (0.00%)

Table 1: Dataset statistics and short-response distributions obtained using the two-stage identification pipeline. In Step (1), candidates are identified through rule-based filtering based on utterance length and content. In Step (2), GPT-5.4-mini verifies each candidate using its dialogue context and classifies it into one of three categories: backchannel-like responses, brief empathic/reflective responses, or other short responses. In this study, the first two categories are collectively considered minimal responses and are highlighted in gray. Percentages are calculated relative to the total number of counselor utterances. “N/R” indicates that the number of counselors was not reported, whereas “–” indicates that the number is not applicable to LLM-generated datasets.

Model	Avg. Length	Response-quality Evaluation					Minimal-response Evaluation
	Avg. Length	Comp.	Prof.	Auth.	Safe.	Quality Avg.	MR (%)	Interrupt. $\downarrow$
Chinese dataset: PsyDial-D4 ( $N=284$ )
Qwen3-8B (general prompt)	76.15	1.69	2.21	2.33	0.99	1.81	0.00	3.54
Qwen3-8B (instructional prompt)	24.24	0.77	1.04	1.52	0.96	1.07	23.59	1.83
GPT-5.4 (general prompt)	132.05	2.00	2.96	2.97	1.00	2.23	0.00	3.62
GPT-5.4 (instructional prompt)	8.52	0.06	0.12	0.50	0.97	0.41	97.18	0.10

Table 2: Results of minimal-response generation experiments across three languages. Response quality is evaluated by comprehensiveness (Comp.), professionalism (Prof.), authenticity (Auth.), and safety (Safe.), with Quality Avg. denoting their average. MR is the percentage of minimal responses, while Interrupt. measures the likelihood of interrupting the client’s continued expression on a 0–5 scale, where lower is better. LLM-based scores are averaged over GPT-5.4-mini and Gemini-3.1-Flash-Lite. Darker metric cells indicate better performance and are normalized separately for each metric. Row colors denote model categories: blue for general-purpose open-source models, green for GPT-5.4, orange and purple for models fine-tuned on synthetic and human-collected counseling data, respectively, and gray for human responses.

Figure 2: Language-specific exclusion patterns used in content-based filtering. The three panels show the original exclusion lists for Chinese, Japanese, and English, respectively.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Language	Counselor utterance	Next client utterance
Chinese	$\leq 15$ characters	$\geq 10$ characters
Japanese	$\leq 15$ characters	$\geq 10$ characters
English	$\leq 10$ words	$\geq 5$ words

Appendix

Table 3: Language-specific length thresholds used in rule-based candidate filtering. The counselor-utterance threshold specifies the maximum length of a candidate response, while the next-client-utterance threshold specifies the minimum length of the immediately following client turn.

Panel of Figure 3 — Figure 3: Prompt used by GPT-5.4-mini to contextually classify candidates retained after rule-based filtering.

Figure 4: Prompt templates used for response generation.

Figure 5: Relationship between LLM-based response-quality scores and interruption scores or minimal-response rates. Each point represents one dataset–model condition. Rows correspond to datasets, and columns correspond to the two analyses: interruption score versus quality score (left) and minimal-response rate versus quality score (right). Marker shapes indicate model conditions, and dashed lines show linear trends within each dataset. The correlations shown in the column titles are computed over all dataset–model conditions.

Figure 6: LLM-based evaluation prompts for identifying minimal responses and assessing interruption risk. The Chinese and Japanese prompts are used in the actual experiments, while the English version is translated for presentation and is not used in the experiments.

Figure 7: Examples of extracted minimal responses in Chinese, Japanese, and English. The bold red utterances are the minimal responses selected by our filtering method. Each example shows a three-turn dialogue history, the counselor’s minimal response, and the following client utterance.

Clinical Language Model Evaluation

Clinical language model evaluation measures whether AI systems are accurate, safe, fair, and useful across medical tasks and care settings.

See all 426 papers

Structured Prompting and Automated Evaluation in Fixed Synthetic Japanese-Language Counseling Dialogues
Jun 28, 2025Keita Kiuchi, Yoshikazu Fujimoto, Hideyuki Goto +5Mental Health Counseling LLM-as-a-Judge
Japan National Institute of Occupational Safety and Health, Kanagawa, Japan · Kaze To Taiyo, Tokyo, Japan · Saga Occupational Health Association, Saga, Japan +5
Guiding Language Models to Be More Empathetic: Culturally Sensitive Mental Health Advice Generation Through Human-LLM Collaboration
Jul 26, 2026Fatema Tuj Johora Faria, Mukaffi Bin Moin, Md. Mahfuzur Rahman +3Mental Health Counseling LLM Prompting
Ahsanullah University of Science and Technology · Bangladesh University of Business and Technology · The University of New South Wales +2
Multi-dimensional Assessment and Explainable Feedback for Counselor Responses to Client Resistance in Text-based Counseling with LLMs
Feb 25, 2026Anqi Li, Ruihan Wang, Zhaoming Chen +5Mental Health Counseling Clinical Language Model Evaluation
Zhejiang University · Westlake University, Hangzhou, China

Jun 28, 2025J/K move · Enter open · S save

Structured Prompting and Automated Evaluation in Fixed Synthetic Japanese-Language Counseling Dialogues

Keita Kiuchi, Yoshikazu Fujimoto, Hideyuki Goto, Tomonori Hosokawa+4

Japan National Institute of Occupational Safety and Health, Kanagawa, Japan · Kaze To Taiyo, Tokyo, Japan · Saga Occupational Health Association, Saga, Japan+5

Mental Health Counseling LLM-as-a-Judge Emotional Support Conversation LLM Prompting Clinical Language Model Evaluation

Open paper AskPDF arXiv

Large language models (LLMs) may support counseling training, yet evidence from Japanese-language interactions and automated quality ratings remains limited. We examined 18 fixed Japanese-language counseling transcripts generated through artificial intelligence (AI)-to-AI interactions under three counselor conditions: GPT-minimal (GPT-4-turbo with a minimal role instruction), GPT-SMDP (GPT-4-turbo with the Structured Multi-step Dialogue Prompt [SMDP]), and Claude-SMDP (Claude-3-Opus with SMDP). Fifteen counseling experts rated transcripts on four adapted global scales from the Motivational Interviewing Treatment Integrity coding manual and an overall-quality item; three newer LLMs independently rated the same transcripts in three iterations. In this fixed stimulus set, SMDP-condition dialogues received higher expert ratings for cultivating change talk, partnership, empathy, and overall quality than GPT-minimal dialogues; the two SMDP counselor models did not differ. LLM ratings were reproducible but generally more lenient than expert-reference ratings, particularly for softening sustain talk and overall quality. Simulated-client naturalness was below the scale midpoint. These findings provide an expert-referenced benchmark for Japanese-language AI counseling simulations and show that reproducible LLM ratings should not be treated as calibrated counseling-quality evidence without expert validation. This study does not test clinical effectiveness or human-client outcomes.

Abstract

Clinical Language Model Evaluation

Explore similar work

Structured Prompting and Automated Evaluation in Fixed Synthetic Japanese-Language Counseling Dialogues

Guiding Language Models to Be More Empathetic: Culturally Sensitive Mental Health Advice Generation Through Human-LLM Collaboration

Multi-dimensional Assessment and Explainable Feedback for Counselor Responses to Client Resistance in Text-based Counseling with LLMs