Multi-Objective Aligned Small Language Model Framework for SUD Patient Dialogue Generation

Thushara Manjari Naduvilakandy; Hyeju Jang; Mohammad Al Hasan

cs.CLOct 6, 2026

Multi-Objective Aligned Small Language Model Framework for SUD Patient Dialogue Generation

Authors: Thushara Manjari Naduvilakandy, Hyeju Jang, Mohammad Al Hasan

Organizations: Department of Computer Science, Indiana University Indianapolis, Indianapolis, IN, USA

Abstract

Substance Use Disorder (SUD) counseling requires patient responses that reflect underlying cognitive states such as beliefs, coping strategies, and readiness for change. Although large language models (LLMs) can generate fluent text, they often fail to produce cognitively coherent and clinically realistic patient behavior, especially under ethical and data-scarce clinical settings. Moreover, deploying frontier-scale LLMs in healthcare applications presents practical challenges including high computational cost, latency, privacy concerns, and limited deployability in resource-constrained environments, motivating the need for cognitively aligned small language models (SLMs). We propose a cognitively grounded framework for SUD patient dialogue generation that explicitly models and aligns latent cognitive components with patient histories and counselor questions. Our pipeline consists of two stages: cognitive component detection and cognitive component-aligned dialogue generation. To enable effective learning with smaller models, we combine knowledge distillation from high-capacity teacher models, preference optimization from human-annotations, and attention-guided reward shaping. Extensive evaluations using automatic scores like BERTScore, ROUGE, METEOR and BLEU, and LLM-as-judge hit-metrics against both human and teacher-model references show that cognitively informed fine-tuning substantially improves cognitive realization and alignment over a generic instruction-tuned baselines and mental health domain specific SLMs, with particularly strong gains for open-ended cognitive components.

Figures & tables

Fig. 1: Framework for Cognitive Components-Aligned SUD Patient Dialogue Generation in LLMs

Generations	Reference Labels
Generations	Human						GPT-5
	BERT-F1	R-1	R-2	R-L	METEOR	BLEU	BERT-F1	R-1	R-2	R-L	METEOR	BLEU
Llama3-8B-instruct	0.805	0.0289	0.0074	0.0258	2.31	0.10	0.813	0.0666	0.0191	0.0515	6.81	0.35
MentaLLaMA7B	0.812	0.0259	0.0047	0.0221	1.90	0.09	0.819	0.0617	0.0150	0.0463	5.49	0.30
MentaLLaMA13B	0.815	0.0243	0.0037	0.0210	1.61	0.09	0.819	0.0576	0.0136	0.0450	4.84	0.35
MentaLLaMA13B-Fewshot	0.828	0.0678	0.0069	0.0647	3.46	0.78	0.830	0.1018	0.0185	0.0923	6.94	1.33

TABLE I: Cognitive Component Detection (CCD) performance across BERTScore, ROUGE, METEOR, and BLEU.

Gen vs Ref	Ref-Hit ratio (PM / FM / NM %)
CCD vs Expert	48.3 / 45.4 / 51.7
CCD vs GPT-5	60.0 / 57.4 / 40.0
GPT vs Expert	61.5 / 57.8 / 38.5

TABLE II: Cognitive Component Detection (CCD) LLM-Judge Ref-Hit Ratio.

CC	$\boldsymbol{\Delta R}$	CC	$\boldsymbol{\Delta R}$
Intermediate belief	$-0.1034$	DSM-5 traits	$0.1034$
Legal issue	$-0.0690$	Physical health issues	$0.1379$
Automatic thought	$-0.0690$	Primary core belief	$0.1552$
Core belief	$-0.0632$	Social issue	$0.1552$
Emotion	$-0.0517$	Drugs used	$0.2011$
Financial issue	$-0.0345$	SUD stage	$0.3103$

TABLE III: Per–cognitive component difference in reference hit ratio between GPT-5 and CCD when evaluated against human annotations (

\Delta R=R_{\text{GPT}}-R_{\text{CCD}}

). Positive values indicate higher GPT coverage, while negative values indicate higher CCD coverage.

Generations	F1	P	R
Llama3-8B-instruct	$0.829\pm 0.013$	$0.807\pm 0.016$	$0.853\pm 0.014$
MentalLlama7B	$\textbf{0.856}\pm\textbf{0.016}$	$\textbf{0.851}\pm\textbf{0.020}$	$0.862\pm 0.020$
MentalLlama13B	$0.850\pm 0.024$	$0.838\pm 0.034$	$0.864\pm 0.019$
MentalLlama13B-Fewshot	$0.850\pm 0.024$	$0.840\pm 0.034$	$0.864\pm 0.020$
CADG	$0.844\pm 0.018$	$0.826\pm 0.027$	$\textbf{0.865}\pm\textbf{0.015}$

TABLE IV: Cognitive Component Aligned Dialogue Generation (CADG) BERTScores.

Generated Model	GPT-5 Reference(370)	Expert Reference(249)	CCD Reference(350)
	PM / FM / NM (%)	PM / FM / NM (%)	PM / FM / NM (%)
CADG (Ours)	56.8 / 55.4 / 43.2	49.3 / 48.1 / 50.7	97.1 /80.9 /2.9
Llama8b-instruct	54.0/ 52.9 / 46.0	45.4 / 44.4 /54.6	–
MentaLlama-7B	47.1 / 46.2 /52.9	39.4/38.1 /60.6	–
MentaLlama-13B	49.9 / 48.9 / 50.1	42.4 / 41.8/57.6	–
MentaLlama-13B+FS	52.7 / 51.9 / 47.3	46.2 /44.4/ 53.8	–

TABLE V: Cognitive Component Aligned Dialogue Generation (CADG) performance across various reference labels and baseline models. PM / FM / NM (%) represents Partial Match, Full Match, and No Match ratios respectively.

Model	ROUGE (R-1 / R-2 / R-L)	BLEU	METEOR
Llama8b-instruct	0.012/0.003 /0.010	0.03	1.55
Mentallama-7B	0.024 /0.004 /0.019	0.07	2.29
Mentallama-13B	0.025 / 0.005 / 0.020	0.08	2.39
Mentallama-13B+FS	0.021 / 0.003 / 0.017	0.06	2.02
CADG	0.018 / 0.004 / 0.016	0.04	1.67

TABLE VI: Automatic evaluation scores with ROUGE,BLEU and METEOR for human reference cognitive alignment analysis. Showing our CADG got low rouge score than other baselines except Llama 8B instruct and still achieves high LLM judge alignment score indicating model did not copy paste the CC but it paraphrased for alignment.

Model	Cognitive Component			Counselor Question			Length
Model	ROUGE (R1/R2/RL)	BLEU	METEOR	ROUGE (R1/R2/RL)	BLEU	METEOR	#words $<40$ /#words $>$ 40
CADG	0.056 / 0.034 / 0.050	1.05	6.75	0.093/0.030/0.068	0.93	14.63	0.0% / 100.0 %
CConlyAttendRwd	0.532 / 0.485 / 0.533	20.49	28.76	0.028/0.003/0.025	0.25	1.36	98.4%/ 1.6%
CQonlyAttendRwd	0.023/0.001/0.020	0.15	1.15	0.647/0.554/0.611	33.45	48.40	94.7% / 5.3%
lengthOnlyRwd	0.047/ 0.024/ 0.038	0.52	5.86	0.081/ 0.022/ 0.058	0.54	13.63	0.0% / 100.0%

TABLE VII: Analysis on GRPO reward ablation study on Cognitive Component attention Counselor Question attention and Length reward (reports average response length)

Fig. 2: Training reward progress of preference optimization (DPO) for chosen and rejected in cognitive aligned dialogue generator.

Fig. 3: Progress of attention- and length-based rewards during GRPO training. The attention-based reward shows how the model progressively increases its attention toward both the cognitive component and counselor question segments, indicating improved focus and alignment over training steps. The length-based reward shows how the model learns to produce responses within the desired length range over training steps.

Fig. 4: Harmonic mean of length and attention reward for different weight distributions.

Cohen’s Kappa ( $\kappa$ ): 0.5421 (Moderate agreement)
GPT-5 $\backslash$ GPT-5 Mini	Worse	Better	Same	Row Total
Worse	20	2	5	27
Better	3	32	4	39
Same	2	9	416	427
Column Total	25	43	425	493
Spearman Correlation ( $\rho$ ): 0.5528 (Moderate, $p<1e{-40}$ )

TABLE VIII: Confusion matrix comparing GPT-5 and GPT-5 Mini predictions across three classes (CADG worse, CADG better, CADG same). Diagonal entries (bold) indicate agreement. Agreement metrics show moderate categorical agreement (Cohen’s

\kappa

) and strong ranking consistency (Spearman

\rho

Explore similar work

Understanding Clinical Cognitive Dialogues Using Large Language Models
Sep 28, 2026Vishalakshi Arumugam, Dan Schumacher, Veronica Rammouz +3Cognitive Diagnosis Clinical Reasoning Training
College of AI, Cyber and Computing University of Texas at San Antonio, USA
Graph2Counsel: Clinically Grounded Synthetic Counseling Dialogue Generation from Client Psychological Graphs
Apr 22, 2026Aishik Mandal, Hiba Arnaout, Clarissa W. Ong +5Counseling Synthetic Data
UKP Lab, Department of Computer Science and Hessian Center for AI (hessian.AI), Technische Universität Darmstadt · Zuse School · National Research Center for Applied Cybersecurity ATHENE +4
CARE: Counselor-Aligned Response Engine for Online Mental-Health Support
Apr 23, 2026Hagai Astrin, Ayal Swaid, Avi Segal +1Counseling Emotional Support Conversation Task
Ben-Gurion University · School of Informatics, University of Edinburgh Edinburgh, UK

Sep 28, 2026, cs.CLJ/K move · Enter open · S save

Understanding Clinical Cognitive Dialogues Using Large Language Models

Vishalakshi Arumugam, Dan Schumacher, Veronica Rammouz, Enrique Gonzalez Guerrero+2

College of AI, Cyber and Computing University of Texas at San Antonio, USA

Cognitive Diagnosis Clinical Reasoning Training Fine-Grained Actions

Open paper AskPDF arXiv

In-person cognitive assessment is both a test and an interaction. Clinicians explain tasks, repair misunderstandings, and adapt to patient responses, while patients may hesitate, seek clarification, or disengage. Yet clinical dialogue resources rarely label the interaction structure needed to study these behaviors at scale. We present an de-identified corpus of 33 cognitive assessment conversations with 8,250 utterances annotated for three speaker roles and 56 dialogue acts. We use this corpus to benchmark large language models on fine-grained dialogue-act classification and next-patient-utterance generation. We also test whether out-of-domain instruction data and explanation-augmented training transfer to this clinical setting. Instruction tuning produces the strongest patient-utterance reference matching and improves classification accuracy. Reasoning-aware fine-tuning produces the strongest classification results among the LLaMA-3.1-8B variants. However, even the best models struggle to separate closely related dialogue acts, showing that broad conversational intent is easier to recognize than fine-grained communicative function. The corpus and benchmark make interaction structure measurable in cognitive assessments and support follow-up work on conversational markers, clinician education, and carefully validated simulated patients. This work does not make diagnostic claims. Instead, it provides the data and evaluation framework needed to study these applications.

Abstract

Explore similar work

Understanding Clinical Cognitive Dialogues Using Large Language Models

Graph2Counsel: Clinically Grounded Synthetic Counseling Dialogue Generation from Client Psychological Graphs

CARE: Counselor-Aligned Response Engine for Online Mental-Health Support