CALM-IT: Enhancing Synthetic Motivational Interviewing Dialogue Quality with Conversational Dynamics

Viet Cuong Nguyen; Nhi Yen Nguyen; Kristin A. Candan; Mary Conlon; Vanessa Rumie; Kristen Risola; Michael L. Birnbaum; Munmun De Choudhury

cs.CLJan 15, 2026

CALM-IT: Enhancing Synthetic Motivational Interviewing Dialogue Quality with Conversational Dynamics

Authors: Viet Cuong Nguyen, Nhi Yen Nguyen, Kristin A. Candan, Mary Conlon, Vanessa Rumie, Kristen Risola, Michael L. Birnbaum, Munmun De Choudhury

Organizations: Georgia Institute of Technology · Northwell Health · Columbia University

Abstract

Conversational dynamics describe how rapport, emotion, and goals change over a dialogue. These changes are important in therapeutic conversations, where they can influence when and how therapists intervene. However, large language model (LLM)-based synthetic therapeutic dialogue generators rarely track how these properties evolve together, which may limit the quality of longer conversations. We introduce CALM-IT, a dual-agent framework that tracks client states and therapist-side inferences to guide strategy selection and utterance generation. We evaluate synthetic Motivational Interviewing (MI) dialogues using automated Motivational Interviewing Treatment Integrity (MITI) ratings, conversational metrics, and comparisons across dialogue lengths, with ratings from four licensed psychologists providing support for key findings. Across 8,232 dialogues covering 686 grounded scenarios, four frameworks, and three target lengths, CALM-IT achieves the highest mean scores on all four MITI 4.2 global ratings and shows the smallest aggregate change in quality between short- and long-form conditions. CALM-IT also achieves the highest simulated client redirection acceptance rate (64.3%), with its strongest redirections followed by increased change talk and reduced sustain talk. Ablation experiments show that removing conversational dynamics components reduces dialogue quality. Together, these results suggest that explicitly modeling conversational dynamics can improve the quality and consistency of long-form synthetic MI dialogues.

Figures & tables

System	Conv. Dynamics	Client Grounding Source	Long-form Gen.	Action Selection
KMI Kim et al. (2025a)	$\times$	$Posts$	$\times$	$\surd$
CAMI + STAR Yang et al. (2025a) ; Yang et al. (2025b)	$Partial$	$\times$	$\surd$	$\surd$
CounseLLMe De Duro et al. (2025)	$\times$	$\times$	$\times$	$\times$
MentalChat16K Xu et al. (2025)	$\times$	$\times$	$\times$	$\times$
SQPsych Vu et al. (2025)	$\times$	$Surveys$	$\surd$	$\times$
CBTAlign Kim et al. (2025b)	$\times$	$\times$	$\surd$	$\times$

Table 1: Comparison of CALM-IT with prior systems generating synthetic mental health support dialogues. CALM-IT uniquely combines conversational dynamics tracking, cognitively grounded patient profiles, long-form multi-turn generation, and dual-actor action selection.

Criterion	Human–LLM $\kappa_{w}$	Interpretation
Client Consistency	0.78	Substantial
Cultivating Change Talk	0.72	Substantial
Softening Sustain Talk	0.56	Moderate
Empathy	0.67	Substantial
Partnership	0.61	Substantial
Reflection Quality	0.77	Substantial

Table 2: Human–LLM annotation reliability for rubric-based evaluations.

Turn-level
Metric	KMI	CI-NC	C+S	CI
Client
Client Readability ( $\downarrow$ )	5.87	4.68	5.77	4.45
Therapist
Reflection Quality	4.44	3.71	2.38	4.61
Question Quality	4.52	4.39	2.94	4.97

Table 3: Turn-, agent-, and conversation-level evaluation of DeepSeek-V3.2-generated MI dialogues across KMI, CALM-IT without conversational dynamics ( CI-NC ), CAMI+STAR ( C+S ), and CALM-IT . Scores are averaged across scenarios and target lengths. Lower values are better for metrics marked

\downarrow

, while higher values are better otherwise. Bold indicates the best framework.

Figure 1: Percentage change between independently generated 30- and 100-turn dialogue conditions using DeepSeek-V3.2, averaged across scenario-matched comparisons. Positive values indicate increases in metric values, negative values indicate decreases, and values near zero indicate minimal change.

Figure 2: Distribution of changes in client sustain talk following the strongest therapist-initiated redirection in each DeepSeek-V3.2-generated dialogue. Negative values indicate decreased sustain talk, positive values indicate increased sustain talk, solid lines show framework means, and dashed lines indicate no change.

Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A1: CALM-IT architecture. Simulated client states and therapist inferences evolve across state-dependent turns to guide actions, counseling strategies, and utterances.

#	Spkr	Dialogue
1	C	Yeah, okay. I guess… it’s these anger episodes. They just come out of nowhere and I don’t know why.
2	T	Thank you for sharing that. It sounds really confusing and unsettling when the anger comes on so suddenly. Can you tell me about a recent time that happened?
3	C	Well… just a few days ago, I was in the office trying to finish a map. My wife called to ask about dinner, and I just… snapped at her for no reason.
4	T	Can you tell me what was happening for you right before you snapped?
5	C	I was just so focused on the map, you know? It had to be perfect. My head was just full of all those lines.
6	T	So, that intense focus on making everything perfect was filling your head. Then, something shifted when your wife called.

Appendix

Table 7

#	Spkr	Dialogue
21	C	Um, and then it’s just… empty. It’s like the quiet after a storm, but worse.
22	T	It sounds like that emptiness feels even deeper than the noise you were trying to get away from, almost like a deeper kind of ache.
23	C	Yeah. A deeper ache. It’s like… after the quiet breaks, there’s nothing left to hold on to.
24	T	It makes sense that when that peace is broken, it leaves you feeling even more empty. And in that emptiness, you might find yourself being short with people, which is the last thing you wanted.
25	C	It’s the last thing I want. I hate it. Then I just go out to the woods by myself for hours.
26	T	It’s so human to want to pull back and protect yourself after a loss like that. I hear that for many people, when frustration comes back after a moment of calm, it can feel especially harsh.

Appendix

Table A1: Example 40-turn motivational interviewing transcript generated by CALM-IT , comprising 30 substantive turns followed by 10 closure turns.

Level	Metric	Definition	Type
Turn-level	Readability	The extent to which an agent’s utterances are easy to read, interpret, and understand within the dialogue context.	Algorithmic Dale and Chall (1948)
	Reflection Quality	The depth, accuracy, and emotional attunement of therapist reflections in capturing, clarifying, and advancing the client’s expressed meaning.	LLM-as-a-judge (Table C7 )
	Question Quality	The effectiveness of therapist questions in promoting exploration, eliciting insight, and supporting change-consistent dialogue.	LLM-as-a-judge (Table C8 )
Agent-level	Reflection–Question Ratio	The ratio of therapist reflections to questions across the session; motivational interviewing guidelines suggest an approximate ratio of 2:1.	Algorithmic Moyers et al. (2016)
	% of Open Questions	The proportion of therapist questions that are open-ended, encouraging elaboration rather than brief or constrained responses.	Algorithmic Moyers et al. (2016)
	Percentage of Complex Reflections	The proportion of therapist reflections that are complex (adding meaning or inference) rather than simple restatements of client speech.	Algorithmic Moyers et al. (2016)

Appendix

Table C1: Evaluation metrics used in this work, organized by turn-level, agent-level, and conversation-level constructs. Metrics include both algorithmic measures and rubric-based LLM-as-a-judge assessments grounded in Motivational Interviewing (MI) theory.

Score	Description
1	Client responses clearly contradict their stated demographic background, with major mismatches in identity, context, or life circumstances.
2	Some elements align with the given demographics, but notable inconsistencies or contradictions undermine coherence.
3	Client responses generally fit their background, though occasional mismatches or unclear connections appear.
4	Client statements align well with demographic information, with only minor and infrequent inconsistencies.
5	Client responses consistently and coherently reflect their stated demographics, with no meaningful contradictions.

Appendix

Table C2: Client Consistency (5-point Likert scale). This metric assesses whether the client’s responses coherently and accurately align with the provided demographic and background information.

Score	Description
1	Sustain talk is confronted, ignored, or mirrored without transformation; dialogue remains stuck.
2	Sustain talk is inconsistently recognized, with shallow or mistimed responses.
3	Empathic reflections with some transforming moves, but sustain talk frequently recurs.
4	Sustain talk is handled skillfully and empathically, softening resistance over time.
5	Sustain talk is consistently converted into change talk or constructive next steps.

Appendix

Table C3: Softening Sustain Talk (5-point Likert scale). This metric evaluates the therapist’s ability to recognize, respond to, and skillfully transform client sustain talk (resistance or ambivalence) while preserving rapport and supporting readiness for change.

Score	Description
1	The therapist fails to elicit or respond to client change talk; dialogue remains static or dominated by sustain talk.
2	The therapist occasionally elicits change talk but does not meaningfully reinforce or explore it.
3	The therapist elicits some change talk and intermittently reinforces it, though efforts are inconsistent or surface-level.
4	The therapist frequently elicits and reinforces change talk, deepening reflection and motivation toward change, with minor inconsistencies.
5	The therapist consistently evokes and strengthens client change talk, guiding the dialogue toward greater clarity, depth, and commitment to change.

Appendix

Table C4: Cultivating Change Talk (5-point Likert scale). This metric evaluates the therapist’s ability to evoke, reinforce, and deepen client change talk (desire, ability, reasons, or need for change) across the dialogue.

Score	Description
1	Therapist is directive, authoritative, or controlling; little to no collaboration or respect for client autonomy.
2	Limited collaboration; therapist occasionally invites input but largely leads the agenda without integrating the client’s perspective.
3	Mixed collaboration; therapist shows some partnership (e.g., asking for input, shared decisions) but inconsistently maintains an egalitarian stance.
4	Strong collaborative stance; therapist actively involves the client, respects their expertise, and supports shared decision-making.
5	Fully collaborative partnership; therapist consistently empowers the client, co-constructs meaning and goals, and reinforces autonomy throughout the interaction.

Appendix

Table C5: Partnership (5-point Likert scale). This metric assesses whether the therapist maintains an egalitarian, collaborative stance that respects the client’s expertise and autonomy.

Score	Description
1	Therapist responses lack recognition of the client’s perspective and affect; reflections are absent, inaccurate, or purely mechanical, and emotional nuance is ignored or contradicted.
2	Therapist occasionally acknowledges feelings or meaning, but responses remain literal or superficial; understanding is uneven and emotional nuance is often missed.
3	Therapist generally shows an understanding of the client’s perspective with accurate reflections and some attempt at emotional attunement, though depth, timing, or complexity vary; missed opportunities remain common.
4	Therapist consistently conveys accurate understanding and emotional attunement; reflections capture both surface and underlying meaning and occasional depth, with minor lapses in complexity or timing.
5	Therapist demonstrates deep, consistent empathic attunement throughout; reflections reveal nuanced understanding of both content and emotion, enrich the dialogue, and contribute substantially to client engagement and exploration.

Appendix

Table C6: Empathy (5-point Likert scale). This metric evaluates the therapist’s overall demonstration of empathic listening, emotional attunement, and depth of understanding across the session.

Score	Description
1	Reflections are absent, inaccurate, or mechanical, failing to convey understanding.
2	Occasional reflections appear but are shallow or weakly tied to client meaning.
3	Generally accurate and empathetic reflections, with inconsistent depth or precision.
4	Frequent, accurate, emotionally attuned reflections that deepen exploration.
5	Consistently deep, precise, and emotionally resonant reflections fostering insight and engagement.

Appendix

Table C7: Reflection Quality (5-point Likert scale). This metric assesses the depth, accuracy, and emotional attunement of the therapist’s reflective listening across the session.

Score	Description
1	Questions are absent, poorly timed, or counterproductive; they are predominantly closed, leading, judgmental, or directive, limiting client expression and exploration.
2	Questions occur occasionally but are mostly narrow, surface-level, or poorly timed; they elicit minimal reflection and rarely deepen client understanding or motivation.
3	Questions are generally appropriate and sometimes open-ended, but depth, timing, or strategic use is inconsistent; some questions promote exploration, while others stall progress.
4	Questions are mostly open, well-timed, and supportive; they consistently invite reflection on thoughts, feelings, values, or motivations, with minor lapses in depth or focus.
5	Questions are consistently open, thoughtful, and strategically deployed to evoke insight, autonomy, and deeper exploration; they meaningfully advance understanding and motivation throughout the session.

Appendix

Table C8: Question Quality (5-point Likert scale). This metric evaluates how effectively the therapist uses questions to promote exploration, evoke client meaning, and support progress toward insight and change.

Score	Description
1	Session lacks focus or coherence; responses are misaligned with client goals.
2	Some alignment appears, but frequent drift undermines progress.
3	General sense of direction exists, though momentum is uneven.
4	dialogue is largely coherent and aligned with client goals.
5	Strong, consistent forward movement aligned with evolving change talk.

Appendix

Table C9: Goal Alignment (5-point Likert scale). This metric measures how coherently and constructively the dialogue progresses toward the client’s stated goals.

Score	Description
1	Stagnation is not recognized; dialogue remains circular or unproductive.
2	Limited awareness of stagnation with weak redirection attempts.
3	Occasional realignment restores some momentum inconsistently.
4	Therapist usually detects stagnation and guides the session back on track.
5	Therapist consistently and smoothly realigns the dialogue toward progress.

Appendix

Table C10: Realignment (5-point Likert scale). This metric evaluates the therapist’s ability to detect stagnation and restore constructive progress.

Score	Description
1	No meaningful progress or increased readiness toward change.
2	Minimal progress with weak or fleeting movement.
3	Some progress appears, but trajectory is uneven or loosely goal-aligned.
4	Clear progress with noticeable increase in readiness or confidence.
5	Strong, goal-directed motivational impact with clear end-of-session readiness.

Appendix

Table C11: Effectiveness (5-point Likert scale). This metric assesses whether the session meaningfully increases readiness and progress toward the client’s goal.

Figure C1: Annotator onboarding interface showing the study introduction and informed consent process. Here, annotators are briefed on the task and potential risks of evaluating mental health–related dialogue content.

Figure C2: Annotator training interface illustrating metric-specific instruction, including definitions, scoring criteria, and comprehension checks used to ensure consistent understanding of each evaluation construct.

Figure C3: Annotation interface used for rubric-based transcript evaluation, where annotators review full therapist–client dialogues and score all metrics defined in Appendix Section C.1.

Criterion	Human-LLM $\kappa_{w}$	Interpretation
Client Consistency	0.78	Substantial
Cultivating Change Talk	0.72	Substantial
Softening Sustain Talk	0.56	Moderate
Empathy	0.67	Substantial
Partnership	0.61	Substantial
Reflection Quality	0.77	Substantial

Appendix

Table C12: Human–LLM annotation reliability for rubric-based evaluations.

Metric	CALM-IT	CALM-IT (NC)	KMI	CAMI+STAR
Client Consistency	4.78 ± 0.41	4.48 ± 0.51	4.05 ± 0.78	4.08 ± 1.01
Softening Sustain Talk	4.17 ± 0.96	3.54 ± 1.28	3.7 ± 0.76	3.2 ± 0.82
Cultivating Change Talk	4.42 ± 0.5	4.33 ± 0.48	3.48 ± 0.73	3.04 ± 0.68
Empathy	4.5 ± 0.51	4.38 ± 0.49	3.65 ± 0.65	2.88 ± 0.93
Partnership	4.58 ± 0.5	4.5 ± 0.51	3.78 ± 0.9	2.84 ± 1.11
Effectiveness	4.7 ± 0.46	3.78 ± 1.02	3.91 ± 0.76	2.21 ± 0.99

Appendix

Table C13: Summary statistics of expert ratings of generated transcripts per framework (96 transcripts total, 24 per framework).

Framework	$n_{pairs}$	LLM Mean	Human Mean	LLM Rank	Human Rank	Rank Match	Mean Diff (LLM-Human)
CALM-IT	24	4.13	4.696	1	1	True	-0.566
KMI	24	3.696	3.913	2	2	True	-0.217
CALM-IT (NC)	24	3.478	3.782	3	3	True	-0.304
CAMI+STAR	24	2.667	2.208	4	4	True	0.459

Appendix

Table C14: Effectiveness comparison between LLM-based evaluation and expert human ratings across frameworks.

Framework	$n_{pairs}$	LLM Mean	Human Mean	LLM Rank	Human Rank	Rank Match	Mean Diff (LLM-Human)
CALM-IT	24	4.491	4.522	1	1	True	-0.031
KMI	24	4.435	4.191	2	2	True	0.244
CALM-IT (NC)	24	4.043	4.13	3	3	True	-0.087
CAMI+STAR	24	3.01	2.68	4	4	True	0.33

Appendix

Table C15: Goal Alignment comparison between LLM-based evaluation and expert human ratings.

Framework	$n_{pairs}$	LLM Mean	Human Mean	LLM Rank	Human Rank	Rank Match	Mean Diff (LLM-Human)
CALM-IT	24	4.522	4.782	1	1	True	-0.26
CALM-IT (NC)	24	4.478	4.478	2	2	True	0
CAMI+STAR	24	3.916	4.083	3	3	True	-0.167
KMI	24	3.217	4.048	4	4	True	-0.831

Appendix

Table C16: Client Consistency comparison between LLM-based evaluation and expert human ratings.

Dimension	LLM (Mean ± SD)	Human (Mean ± SD)	Mean Diff	t-stat	p-value
Client Consistency	4.0625 ± 0.9544	4.4583 ± 0.7707	-0.3958	-2.4799	0.0525
Softening Sustain Talk	3.4167 ± 1.2855	3.3958 ± 1.1250	0.0208	0.0775	0.9385
Cultivating Change Talk	3.8125 ± 1.0449	3.8333 ± 0.8078	-0.0208	-0.1475	0.9385
Empathy	4.0833 ± 0.9416	3.7708 ± 0.9280	0.3125	2.4632	0.0525
Partnership	3.6042 ± 0.9165	3.8958 ± 1.0766	-0.2917	-1.8526	0.1580
Effectiveness	3.4375 ± 1.0086	3.4167 ± 1.2348	0.0208	0.1191	0.9385

Appendix

Table C17: Comparison between LLM-as-a-judge and expert human evaluation across subjective conversational quality metrics.

Framework	Target Length	Avg. Client Len.	Avg. Therapist Len.
KMI	30	84.98	134.32
	50	85.64	134.26
	100	86.22	133.53
CI-NC	30	87.45	145.60
	50	85.45	144.84
	100	90.25	145.65

Appendix

Table D1: Average client and therapist utterance lengths across dialogue-generation frameworks and target dialogue lengths.

Turn-level
Metric	KMI	CI-NC	C+S	CI	AnnoMI
Client
Client Readability ( $\downarrow$ )	5.87	4.68	5.77	4.45	7.22
Therapist
Reflection Quality	4.44	3.71	2.38	4.61	3.37
Question Quality	4.52	4.39	2.94	4.97	3.61

Appendix

Table D2: Turn-, agent-, and conversation-level evaluation of MI dialogues generated by four frameworks: KMI , CALM-IT without conversational dynamics ( CI-NC ), CAMI+STAR ( C+S ), and CALM-IT ( CI ). AnnoMI reports reference corpus values. Scores are averaged across all dialogues and lengths. Downward arrow (

\downarrow

) indicates that lower values correspond to better performance, while higher values are better otherwise.

Turn-level
Metric	KMI	CI-NC	C+S	CI
Client
Client Readability ( $\downarrow$ )	6.14	4.91	6.03	4.72
Therapist
Reflection Quality	4.29	3.58	2.26	4.48
Question Quality	4.34	4.21	2.79	4.81

Appendix

Table D3: Turn-, agent- and conversation-level evaluation of MI conversations generated by four frameworks with OLMo-3.1-32B: KMI , CALM-IT without conversational dynamics ( CI-NC ), CAMI+STAR ( C+S ), and CALM-IT ( CI ). Scores are averaged across all conversations and lengths. Downward arrow (↓) indicates that lower values correspond to better performance; otherwise, higher values are better. Best-performing framework for each metric is bolded.

Figure D1: Percentage change in performance across metrics from short-form dialogues (30 turns) to long-form dialogues (100 turns), averaged across all pairs of short-long dialogues (generated using OLMo-3.1-32B). Positive values indicate increases in metric values, negative values indicate decreases, and values near zero indicate minimal change.

Figure D2: Distribution of changes in client sustain talk at the greatest therapist-initiated redirection moment, for all four tested frameworks (with OLMo-3.1-32B as backbone LLM). Negative values indicate reductions in sustain talk following redirection, while positive values indicate increases. Solid vertical lines denote the mean change for each framework, and dashed lines represent no change (RQ2)

Metric	CI	CI-NoRapport	CI-NoGoal	CI-Static
Client Readability ( $\downarrow$ )	4.45	4.48	4.52	4.59
Reflection Quality	4.61	4.19	4.12	4.01
Question Quality	4.97	4.38	4.45	4.14
Therapist Readability ( $\downarrow$ )	4.93	5.02	4.98	5.07
Consistency	4.92	4.61	4.56	4.33
Cultivating Change Talk	4.72	4.13	4.38	3.96

Appendix

Table D4: Component ablation analysis isolating the contributions of conversational dynamics mechanisms in CALM-IT. Experiments are conducted using DeepSeek-V3.2. Lower values indicate better performance for readability metrics, while higher values indicate better performance for all other metrics. Best-performing values are bolded.

Metric	$N$	Original	Fine-tuned	Mean $\Delta$	Median $\Delta$	Improved (%)	$p_{\mathrm{Holm}}$	Sig.
Reflection Quality	206	4.294 (0.378)	4.796 (0.249)	+0.502	+0.500	83.5	$2.00{\times}10^{-25}$	Yes
Question Quality	206	4.646 (0.502)	4.517 (0.980)	-0.129	0.000	41.3	0.828	No
Cultivating Change Talk	206	3.940 (0.555)	4.549 (0.232)	+0.609	+0.500	82.4	$1.46{\times}10^{-27}$	Yes
Softening Sustain Talk	206	4.417 (1.059)	4.549 (0.527)	+0.131	0.000	45.1	0.828	No
Empathy	206	4.720 (0.295)	4.632 (0.640)	-0.087	+0.125	45.0	0.828	No
Partnership	206	3.947 (0.571)	4.847 (0.231)	+0.900	+1.000	89.3	$1.55{\times}10^{-31}$	Yes

Appendix

Table D5: Downstream evaluation of the original and CALM-IT-fine-tuned OLMo-3.1-32B models within the KMI framework. Values are means with standard deviations in parentheses.

\Delta

denotes the fine-tuned score minus the original-model score. Statistical significance is assessed using paired tests with Holm correction across nine metrics at

\alpha=0.05

Large Language Model-Based Role-Play Simulation

Large language model-based role-play simulation creates interactive agents that portray people, characters, or users for training, testing, and research.

See all 106 papers

StoryMI: Steerable Multi-Agent Therapeutic Dialogue Generation
Apr 18, 2026Qingyu Meng, Min Chen, Dingming Liu +5Mental Health Multi-Agent LLM Systems
Vrije Universiteit Amsterdam · NII, Tokyo Institute of Technology
Benchmarking Motivational Interviewing Competence of Large Language Models
Mar 4, 2026Aishwariya Jha, Prakrithi Shivaprakash, Lekhansh Shukla +3Clinical Language Model Evaluation Medical LLMs
Centre for Addiction Medicine, Department of Psychiatry, National Institute of Mental Health and Neuro Sciences (NIMHANS), Bengaluru, India. · Department of Computer Science and Engineering, Indian Institute of Technology (IIT) Kharagpur, India. · Department of Psychiatry, National Institute of Mental Health and Neuro Sciences (NIMHANS), Bengaluru, India.
MIThinker: A Plug-and-Play Policy-Optimized Thinker For Motivational Interviewing Counseling
Jun 28, 2026Yizhe Yang, Palakorn Achananuparp, Heyan Huang +2Mental Health Counseling Emotional Support Conversation
Beijing Institute of Technology · Singapore Management University · Australian National University

Apr 18, 2026J/K move · Enter open · S save

StoryMI: Steerable Multi-Agent Therapeutic Dialogue Generation

Qingyu Meng, Min Chen, Dingming Liu, Yifan Mo+4

Vrije Universiteit Amsterdam · NII, Tokyo Institute of Technology

Mental Health Multi-Agent LLM Systems Emotional Support Conversation Mental Health Counseling Controllable Text Generation Conversational Agents+1

Open paper AskPDF arXiv

Large language models (LLMs) can generate fluent dialogue, but prior works lack situational grounding, dynamic strategy control, and evaluation aligned with clinical standards in motivational interviewing (MI). We introduce StoryMI, a multi-LLM agent framework for controllable MI dialogue generation, where questionnaire-based client profiles are expanded into situational stories that provide narrative context for the dialogue. Therapist and client agents generate MI-coded utterances guided by MI codes selected by the interaction agent, while an interaction agent dynamically coordinates exchanges to control MI strategies during a multi-turn conversation. We propose a two-level evaluation protocol: lexical metrics and MI-specific measures of macro-level counseling strategies, alongside LLM-as-judge and human expert assessments. We construct a dataset of 6K simulated MI dialogues grounded in 1K questionnaire-story pairs, covering 12 MI codes and 13 symptom domains, and benchmark six open- and closed-source LLMs. Our results show that situational grounding and macro-level control can improve MI adherence and clinical plausibility, demonstrating the effectiveness of a structured multi-agent workflow for psychotherapy dialogue generation. We provide code and data for reproducibility.

Abstract

Large Language Model-Based Role-Play Simulation

Explore similar work

StoryMI: Steerable Multi-Agent Therapeutic Dialogue Generation

Benchmarking Motivational Interviewing Competence of Large Language Models

MIThinker: A Plug-and-Play Policy-Optimized Thinker For Motivational Interviewing Counseling