From Speech to Profile: A Protocol-Driven LLM Agent for Psychological Profile Generation

Xingjian Yang; Yudong Yang; Zhixing Guo; Yongjie Zhou; Nan Yan; Lan Wang

cs.SDApr 11, 2026

From Speech to Profile: A Protocol-Driven LLM Agent for Psychological Profile Generation

Authors: Xingjian Yang, Yudong Yang, Zhixing Guo, Yongjie Zhou, Nan Yan, Lan Wang

Organizations: School of Science&Engineering, South China University of Technology, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, China · Key Laboratory of Biomedical Imaging Science and System, Chinese Academy of Sciences, China · Shenzhen Mental Health Center&Shenzhen Kangning Hospital, China

Abstract

The psychological profile that structurally documents the case of a depression patient is essential for psychotherapy. Large language models can be applied to summarize the profiles from counseling speech, however, it may suffer from long-context forgetting and produce unverifiable hallucinations, due to overlong length of speech, multi-party interactions and unstructured chatting. Hereby, we propose a StreamProfile, a streaming framework that processes counseling speech incrementally, extracts evidences grounded from ASR transcriptions by storing it in a Hierarchical Evidence Memory, and then performs a Chain-of-Thought pipeline according to PM+ psychological intervention for clinical reasoning. The final profile is synthesized strictly from those evidences, making every claim traceable. Experiments on real-world teenager counseling speech have shown that the proposed StreamProfile system can accurately generate the profiles and prevent hallucination.

Figures & tables

Figure 1: The StreamProfile framework processes streaming audio, performs PM+-guided CoT reasoning, and accumulates evidence in Hierarchical Evidence Memory for final profile generation.

Module	ID	Sub-Dimensions
Background	A	Lifestyle (A1), Education (A2)
Health History	B	Mental Health Hx (B1), Self-harm Hx (B2)
Social Relations	C	Family (C1), Peers (C2), Social (C3), Academic (C4)
Self-Cognition	D	Self-Perception (D1), Self-Worth (D2)
Current State	E	Emotion (E1), Recent Events (E2), Depression (E3), Trust (E4), Anxiety (E5), Distress (E6), Bullying (E7)

Table 1: Clinical schema

\Omega

of the psychological profile

Figure 2: t-SNE projection of profile embeddings generated by StreamProfile (DeepSeek-CoT).

Figure 3: Ablation Study: Based on SBERT-Scores distribution across sessions for four conditions, grouped by patient type (Normal vs. depression).

Audio LLM
Model	SBERT $\uparrow$	ROUGE-L $\uparrow$	BERTScore-F $\uparrow$	Average $\uparrow$
Qwen2-Audio [ 23 ]	0.323	0.056	0.481	0.287
Text LLM (No COT)
Qwen3-252B [ 24 ] †	0.592	0.132	0.648	0.457
LLaMA-3.1-70B [ 25 ]	0.477	0.089	0.605	0.391
DeepSeek-V3 [ 26 ] †	0.605	0.136	0.648	0.463

Table 2: Profile similarity comparison between different models and our proposed system against human-annotated reference profiles, where † denotes the baseline

Type	Model	LLM-As-Judge
Type	Model	Hallucination	Coverage	Consistency	Average
Audio	Qwen2-Audio [ 23 ]	2.02	1.15	1.56	1.58
Text	LLaMA-3.1-70B [ 25 ]	2.46	1.21	1.89	1.86
	Qwen3-252B [ 24 ]	2.38	2.38	2.71	2.49
	DeepSeek-V3 [ 26 ]	2.43	2.52	2.84	2.60
Ours	StreamProfile (Qwen3)	2.54	2.53	2.86	2.64

Table 3: LLM-as-judge evaluation of three dimensions (Hallucination, Coverage, Consistency) on a 1–5 scale.

LLM Hallucination Mitigation

Large language model hallucination mitigation develops ways to detect and reduce unsupported, fabricated, or inconsistent outputs.

See all 178 papers

Synthetic or Authentic? Building Mental Patient Simulators from Longitudinal Evidence
Mar 24, 2026Baihan Li, Bingrui Jin, Kunyao Lan +2Large Language Model-Based Role-Play Simulation Memory-Augmented Language Model Agents
SJTU Paris Elite Institute of Technology, Shanghai Jiao Tong University, China · X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · School of Computer Science and Engineering, Northeastern University, China +1
Psy-Chronicle: Synthesizing Campus Counseling Grounded in a Student's Semester Timeline
May 21, 2026Chaogui GouMental Health Counseling Emotional Support Conversation
University of Science and Technology Beijing
Graph2Counsel: Clinically Grounded Synthetic Counseling Dialogue Generation from Client Psychological Graphs
Apr 22, 2026Aishik Mandal, Hiba Arnaout, Clarissa W. Ong +5Mental Health Counseling Mental Health
UKP Lab, Department of Computer Science and Hessian Center for AI (hessian.AI), Technische Universität Darmstadt · Zuse School · National Research Center for Applied Cybersecurity ATHENE +4

Mar 24, 2026J/K move · Enter open · S save

Synthetic or Authentic? Building Mental Patient Simulators from Longitudinal Evidence

Baihan Li, Bingrui Jin, Kunyao Lan, Ming Wang+1

SJTU Paris Elite Institute of Technology, Shanghai Jiao Tong University, China · X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · School of Computer Science and Engineering, Northeastern University, China+1

Large Language Model-Based Role-Play Simulation Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

Patient simulation is essential for developing and evaluating mental health dialogue systems. As most existing approaches rely on snapshot-style prompts with limited profile information, homogeneous behaviors and incoherent disease progression in multi-turn interactions have become key chellenges. In this work, we propose DEPROFILE, a data-grounded patient simulation framework that constructs unified, multi-source patient profiles by integrating demographic attributes, standardized clinical symptoms, counseling dialogues, and longitudinal life-event histories from real-world data. We further introduce a Chain-of-Change agent to transform noisy longitudinal records into structured, temporally grounded memory representations for simulation. Experiments across multiple large language model (LLM) backbones show that with more comprehensive profile constructed by DEPROFILE, the dialogue realism, behavioral diversity, and event richness have consistently improved and exceed state-of-the-art baselines, highlighting the importance of grounding patient simulation in verifiable longitudinal evidence.

Abstract

LLM Hallucination Mitigation

Explore similar work

Synthetic or Authentic? Building Mental Patient Simulators from Longitudinal Evidence

Psy-Chronicle: Synthesizing Campus Counseling Grounded in a Student's Semester Timeline

Graph2Counsel: Clinically Grounded Synthetic Counseling Dialogue Generation from Client Psychological Graphs