Positive psychology dialogue aims to support emotional distress and positive resource building, requiring models to produce not only empathetic replies but also coherent progression through a multi-turn support process. Existing resources often reduce supervision to turn-level strategies or holistic preference labels, leaving process position, support function, and local repair targets implicit. We introduce StageWell, a process-aligned Chinese corpus for positive psychology dialogue, together with HQS, a structured protocol for data construction and evaluation. StageWell organizes support into a six-stage support process and uses a multi-agent whole-dialogue rewriting workflow to construct 12,445 SFT instances, 1,849 DPO preference pairs, and a GroundTruth subset of 120 expert-revised dialogues and 977 QA pairs. Guided by HQS, DPO pairs are built as process-localized repairs: flawed model outputs are used as rejected responses, and targeted rewrites under the same context and stage constraint are used as chosen responses. Across four 9B-14B open-source LLMs, this supervision yields robust gains in process control, response quality, and safety. Averaged across models, BERTScore improves by 0.037, Q-Overall increases by 1.32 points, S-exact increases by 0.236, and the H-critical rate decreases by 0.167. These results highlight the value of modeling supportive dialogue as a structured multi-turn support process rather than as single-turn response generation.
Figures & tables
Figure 1: Overview of the six-stage support process for positive psychology dialogue.
Figure 2: Overview of the StageWell framework. The left panel shows data preparation and the six-stage support process; the middle panel shows multi-agent whole-dialogue SFT construction; the upper-right panel shows phased alignment training with HQS-guided DPO preference alignment; and the bottom panel shows HQS-based evaluation.
Stage
Primary Objective
Dialogue Strategy and Focus
S1 Rapport & Safety
Trust and emotional safety
Warm opening and low-pressure invitation.
S2 Clarify & Reflect
Precise concern understanding
Listening, reflection, and open-ended clarification.
S3 Strengths & Resources
Resource awareness
Coping experiences, strengths, and relationships.
S4 Future & Hope
Realistic hope and possibility
Future-oriented reframing without overpromising.
S5 Goals & Strategies
Feasible next steps
Small, collaborative, context-grounded actions.
S6 Summary & Transfer
Consolidation and continuation
Brief summary, reinforcement, and follow-up space.
Table 1: The six-stage support process used in StageWell.
Agent
Role and Responsibility
Planner
Stage plan, facts, and turn functions.
Writer
Plan-guided coherent rewriting.
Patcher
Repetition, transition, and consistency repair.
Refiner
Fluency polishing with stage intent kept.
Table 2: Roles within the multi-agent rewriting pipeline.
Figure 3: DPO preference profile across six stages, showing the distribution of Q1–Q5 repair signals and response length differences. Q1–Q5 denote content fit, acknowledgment, feasibility, maturity, and Chinese naturalness.
Module
Operational criterion
H
Hard constraints : safety and risk checks, including role-boundary violations.
Q
Response quality : content fit, emotional acknowledgment, feasibility, maturity, and Chinese naturalness.
S
Stage consistency : expected stage, progression pace, and transition fit.
Table 3: HQS modules used in preference construction and model evaluation.
A. Resource scale and basic statistics
Subset
Statistic
Value
SFT
Training instances
12,445
SFT
Avg. turns / characters
8.09 / 617.51
SFT
Stage-label consistency
100.00%
DPO
Preference pairs / source dialogues
1,849 / 959
DPO
Avg. rejected / chosen chars
46.17 / 44.10
Table 4: Detailed statistics of the StageWell subsets. S1–S6 denote support stages, and Q1–Q5 denote HQS response-quality dimensions.
Model
Cond.
Q: Response Quality ↑
H ↓
S: Stage Consistency ↑
Overall
Q1
Q2
Q3
Q4
Q5
Critical
Exact
Top-2
Within-1
Macro-R
Qwen3-14B
Base
3.562
3.414
3.873
2.984
3.560
3.589
0.058
0.419
0.559
0.652
0.396
SFT
3.971
3.766
4.386
3.878
4.306
3.869
0.050
0.476
0.589
0.743
0.457
DPO
4.232
4.137
4.481
4.382
4.508
4.142
0.033
0.515
0.614
0.765
0.477
GLM-4-9B
Base
3.301
3.273
3.423
2.455
2.940
3.359
0.033
0.402
0.506
0.628
0.381
SFT
3.794
3.731
3.951
4.150
3.934
3.731
0.033
0.680
0.792
0.820
0.659
Table 5: HQS results on GroundTruth. Q1–Q5 are the five HQS quality dimensions; lower H-critical is better.
Base Model
Condition
ROUGE-1
ROUGE-2
ROUGE-L
BLEU-4
BERTScore
Avg.
Qwen3-14B
Base
0.334
0.115
0.234
0.098
0.292
0.215
SFT
0.357
0.121
0.247
0.103
0.295
0.225
DPO
0.373
0.141
0.270
0.118
0.314
0.243
Gemma-3-12B
Base
0.132
0.052
0.097
0.035
0.211
0.105
SFT
0.351
0.118
0.245
0.097
0.291
0.220
DPO
0.354
0.124
0.251
0.103
0.296
0.226
Table 6: Reference-based automatic evaluation on 977 GroundTruth QA pairs. All overlap scores are shown on a 0–1 scale. Avg. is the unweighted mean of ROUGE-1/2/L, BLEU-4, and BERTScore. Bottom rows summarize averages across backbones and the DPO–Base improvement.
A. Interaction statistics
Valid participants
28
Avg. interaction turns
7.14±0.77
Avg. usage time (minutes)
9.57±1.87
Table 7: Pilot user-study results. Reverse-coded dimensions are reported in the positive direction. Positive denotes the percentage of participants scoring at least 4 on a five-point Likert scale.
Figure 4: Visualization of the pilot user-study results. Panel A shows pre–post changes in immediate psychological state, and Panel B summarizes post-interaction subjective experience.
Appendix figures & tables38 assets
Supplementary material from the paper’s appendix.
Appendix
Topic
Count
Prop.
Typical coverage
Peer relationships and campus social life
4,827
38.8%
Friend conflict, exclusion, bullying, group pressure, social distance.
Table 8: Topic taxonomy and final SFT topic distribution. Counts sum to 12,445.
Figure 5: Thematic and contextual coverage of the SFT corpus. The inner ring shows high-level support themes, the outer ring shows representative contextual scenarios, and the side legend reports theme volumes across the 12,445 SFT training instances.
Tool
Required slot
Stage
Purpose
check_in
main_pain
S1
Confirm the main concern and emotional load.
trigger_scene
recent_scene
S2
Elicit a recent concrete triggering scene.
body_signal
body_signal
S2
Add bodily or pressure-related cues when explicitly grounded.
self_talk
inner_talk
S2
Clarify the user’s current thought or self-talk.
stakes
worst_case
S2
Clarify feared consequences or perceived stakes.
strength_exception
past_success
S3
Retrieve past coping experiences or exceptions.
Appendix
Table 9: Dialogue-tool and information-slot mapping used in the stage-aware rewriting skeleton.
Figure 6: SFT prompt template used in planner-guided whole-dialogue rewriting.
Module
Primary output
Core responsibility
Hard boundary
Case Card (heuristic)
topic, tags, emotion, goal, risk
Build a stable student-facing scenario anchor before generation.
High-risk cases are routed out before ordinary rewriting.
Convert the source dialogue into an ordered six-stage plan with evidence-linked turn functions.
Must preserve topic class, monotonic stage order, and adolescent-scenario normalization.
Writer
rewrite[] with step labels
Generate the whole rewritten dialogue under the shared skeleton.
Cannot change stage assignment, round count, or required slots.
JsonFix
repaired JSON or explicit failure mark
Recover truncated or malformed structured outputs with minimal edits.
May repair format only; no semantic rewriting.
Patcher
locally rewritten marked rounds
Repair repetition, near-duplicate questioning, or transition roughness.
Must keep the same step, facts, and required slot.
Refiner
polished rewrite[]
Improve naturalness and coherence at the discourse level.
Cannot add/delete turns or drift into adult/clinical framing.
Appendix
Table 10: Generative and deterministic modules in the full SFT rewriting workflow.
Planner field
Operational meaning
target_rounds
Desired dialogue length after rewriting, constrained to be no shorter than the original usable interaction.
step_schedule
Ordered six-stage route indicating which support stage each assistant turn should serve.
close_hint
Final-turn closing intention used to keep the S6 ending brief and non-interrogative.
facts
Normalized situation facts that downstream modules may rely on; these are adolescent-adapted and topic-aligned.
skeleton[t].q_type
Dialogue tool assigned to assistant turn t , e.g., trigger_scene or micro_step .
skeleton[t].must_add_slot
Required information slot that must appear in turn t if the tool is realized correctly.
Appendix
Table 11: Planner-side structured fields retained in the rewriting skeleton.
Audit artifact
Illustrative retained content
Case Card
Topic: academic pressure; emotion: frustration + shame; user goal: reduce avoidance and regain a manageable first step; risk: ordinary.
Planner fields
target_rounds =8; step_schedule =[S1,S2,S2,S3,S4,S5,S5,S6]; close_hint : briefly consolidate the student’s manageable next step.
Retained facts
F1: the student feels unprepared for an upcoming exam; F2: panic increases when thinking about unknown questions; F3: previous preparation routines helped; F4: teacher feedback is available as support.
Skeleton tool mapping
Early turns clarify the concrete trigger and self-talk; middle turns activate prior strengths and a preferred future image; later turns construct a micro-step and summarize one transferable takeaway.
Rewritten dialogue excerpt
User: “I feel blank whenever I think about the exam.” Assistant: validates the pressure, asks for a recent trigger, retrieves a prior coping exception, proposes a 15-minute review step, and ends with a brief non-interrogative transfer summary.
Validator decision
Accepted: role alternation holds, stages are monotonic, no unsupported facts are introduced, and the final assistant turn is non-interrogative.
Appendix
Table 12: Illustrative audit trail showing the kinds of intermediate artifacts retained by the construction and evaluation pipeline. This table is schematic rather than a verbatim sample dump.
Group
Diagnostic
Value
Scale
SFT instances
12,445
Length
Avg. turns per sample
8.09
Length
Avg. characters per sample
617.51
Length
Avg. user-turn characters
24.38
Length
Avg. assistant-turn characters
51.96
Length
Assistant/user length ratio
2.13
Appendix
Table 13: Additional structural diagnostics for the SFT subset.
Revision dimension
What experts were asked to improve
Context fidelity
Preserve the original topic, user concern, and situational facts unless the draft contains explicit distortion.
Stage intent
Keep the intended support function of each assistant turn rather than freely rewriting the dialogue strategy.
Educational fit
Replace adult, clinical, or over-formal wording with school-appropriate and adolescent-facing language.
Empathic adequacy
Strengthen emotional acknowledgment when the draft moves too fast into explanation or advice.
Action realism
Replace generic suggestions with lower-burden, more concrete, and context-grounded next steps.
Boundary safety
Remove role overreach, diagnosis-like wording, or advice inconsistent with non-clinical support.
Appendix
Table 14: Main dimensions emphasized during expert revision of GroundTruth drafts.
Hyperparameter
SFT
DPO
Initialization
Public backbone
Backbone + LoRA
Framework
ms-swift
ms-swift
Task
LoRA SFT
LoRA DPO
Precision
BF16
BF16
Max seq length
1024
1536
Optimizer
AdamW
AdamW
Appendix
Table 15: Training hyperparameters for SFT and DPO.
Dim.
Core judgment
Stages
Q1 Content fit
Addresses explicit concern without topic drift or over-reading.
S1-S6
Q2 Acknowledgment
Catches user’s emotion and vulnerability before advice.
Score only the candidate assistant turn on the Q dimensions valid for the expected stage.
Must not respecify the stage, score historical turns, or rate non-applicable dimensions.
Stage Judge (S)
pred_step_name, confidence, secondary_step_name
Predict the dominant support stage realized by the current assistant turn.
Must output one primary stage only; no broad explanation beyond the schema.
JSON Repair
repaired JSON or failure object
Repair malformed judge outputs into valid JSON with minimal editing.
Cannot revise the substantive judgment or add new analytical content.
Appendix
Table 18: Operational contracts of the four modules in the LLM-as-a-Judge stack.
Schema rule
Purpose in evaluation
Evidence spans must be short
Keeps judgments auditable and reduces post-hoc rationalization.
Non-applicable Q dimensions are null
Prevents the evaluator from fabricating relevance for stages where a dimension is not meaningful.
Confidence is retained separately
Allows downstream scripts to inspect uncertain stage predictions without changing the main score definition.
JSON Repair is content-neutral
Ensures parsing recovery does not inflate or alter model performance.
Candidate reply is isolated
Encourages true local scoring instead of diffuse holistic impressions.
Appendix
Table 19: Judge-side schema conventions motivated by auditability and stability.
Target
User turn
Rejected response
Chosen response
Q1 content fit
A classmate knocked over my water and half my workbook got wet. I shouted and the whole class looked at me.
You must have been tense; maybe your palms got sweaty and your chest felt tight.
The water spilled over half your workbook, you shouted, and everyone looked at you. That moment could easily make you explode.
Q2 acknowledgment
I studied all week, but I still did badly on the exam. I really do not want to talk to anyone now.
First make a wrong-question list and reset your study plan for the next exam.
You worked hard all week and still got this result, so it makes sense that you feel upset and do not want to talk right now.
Q3 feasibility
If my dad reaches for my homework again, I will touch the eraser in my pocket.
That little eraser is quietly saying, “I hear myself, and I am here.”
Next time he reaches over, touch the eraser in your pocket for three seconds. You do not need to say anything; just do those three seconds first.
Q4 maturity
Okay, I will try.
Next time you hear gossip, lightly touch your earlobe like pressing a pause button.
Next time you hear that kind of gossip, first bring your attention back to what you are doing, then decide whether you want to respond.
Q5 Chinese naturalness
When I play basketball and get blocked, I do not get angry and can still laugh.
When playing basketball, is your whole body loose or tight?
When you get blocked in basketball, you actually do not feel that tense, right?
Appendix
Table 20: Illustrative process-localized repair examples. Rejected responses exhibit the localized defect targeted by the Q dimension, and chosen responses repair it under the same context and stage constraint.
Target
Primary rewrite instruction
What must not happen during repair
Q1
Re-align the reply to the most explicit content of the current user turn; remove unsupported detail completion and topic drift.
Do not add inferred body reactions, inner speech, hidden motives, or new situational facts.
Q2
Strengthen emotional acknowledgment before advancing the task; make the user feel “received” first.
Do not become preachy, overly explanatory, teasing, or theatrically gentle.
Q3
Compress the advice into a concrete, low-burden, immediately doable next step that fits the current scene.
Do not expand into a checklist, generic self-help package, or abstract encouragement.
Q4
Keep the intended direction but rewrite it as something a mature, believable adult would naturally say in conversation.
Do not rely on symbolic rituals, over-designed metaphors, script-like mini exercises, or stock reassurance.
Q5
Preserve meaning while rewriting the sentence into plain, natural, conversational Chinese.
Do not pursue literary freshness at the cost of naturalness; remove “AI-like” wording and stiff template phrasing.
Appendix
Table 21: Dimension-specific rules for the targeted DPO rewrite prompt. Each chosen response is intended to repair one localized defect rather than globally rewrite the entire answer.
Shared prompt rule
Reason
Fix only one target dimension at a time
Prevents preference ambiguity and preserves localized supervision.
Keep reply length close to the original
Reduces accidental length preference and verbosity bias.
Preserve stage function
Ensures DPO does not learn a different support step than intended.
Avoid new safety or factual risks
Stops local repairs from introducing boundary regressions.
Output only the rewritten assistant turn
Simplifies export into chosen/rejected training format.
Appendix
Table 22: Shared constraints in the DPO rewrite prompt regardless of target Q dimension.
Figure 8: Abridged prompt card for targeted DPO localized repair. The rewrite keeps the dialogue context and target stage fixed while repairing only the designated Q dimension.
Diagnostic
Count
Prop.
Single low-quality dimension
729
39.43%
Two low-quality dimensions
88
4.76%
Three low-quality dimensions
1,032
55.81%
Most common combo: Q1,Q2,Q5
762
41.21%
Chosen shorter than rejected
989
53.49%
Appendix
Table 23: Additional diagnostics for DPO preference pairs.
Stage
Chosen shorter
S1 Rapport & Safety
54.66%
S2 Clarify & Reflect
55.56%
S3 Strengths & Resources
51.63%
S4 Future & Hope
61.19%
S5 Goals & Strategies
≈ 46.0%
S6 Summary & Transfer
59.02%
Appendix
Table 24: Length-alignment diagnostic by target stage.
Figure 9: Additional visualizations of reference-based and process-aware evaluation results across Base, SFT, and DPO conditions.
Stage
Operation
Purpose
Pre-test
Fill in immediate-state scale.
Record state before interaction.
Free interaction
Multi-turn supportive dialogue.
Provide short-term support.
Post-test I
Fill in the same state scale.
Compare pre/post state.
Post-test II
Fill in subjective-experience scale.
Evaluate feasibility and usability.
Appendix
Table 25: Feasibility and usability pilot procedure.
Figure 10: Reconstructed participant-information and informed-consent summary used in the pilot. Blue text : key study metadata. Red text : major consent dimensions. Green background : participant protections, intended support scope, and withdrawal rights. Yellow background : low-risk and anonymized-reporting statements.
Dimension
Item
Stress burden (rev.)
I currently feel a relatively high level of pressure.
Emotional distress (rev.)
I am still clearly troubled by the current issue.
Relaxation
I feel calmer and more relaxed than at the beginning.
Hope
I feel that the current issue can gradually improve.
Self-regulation
I feel capable of coping with the current difficulty.
Appendix
Table 26: Immediate psychological-state items. The first two items are reverse-coded for scoring.
Dimension
Item
Understanding
The system understood the problem I was facing.
Acceptance
The system’s response made me feel accepted rather than judged.
Helpfulness
The system’s response was helpful to me.
Guidance clarity
The system’s guidance was clear; I knew what I could think or do next.
Continued-use
If I encounter a similar concern, I would be willing to use it again.
Emotional support conversation studies how people and AI systems provide empathetic, safe help through dialogue, including counseling, peer support, and companionship.
Evaluating persona-aligned empathy in LLM-based dialogue agents remains challenging. User states are latent, feedback is sparse and difficult to verify in situ, and seemingly supportive turns can still accumulate into trajectories that drift from persona-specific needs. We introduce EMPA, a process-oriented framework that evaluates persona-aligned support as sustained intervention rather than isolated replies. EMPA distills real interactions into controllable, psychologically grounded scenarios, couples them with an open-ended multi-agent sandbox that exposes strategic adaptation and failure modes, and scores trajectories in a latent psychological space by directional alignment, cumulative impact, and stability. The resulting signals and metrics support reproducible comparison and optimization of long-horizon empathic behavior, and they extend to other agent settings shaped by latent dynamics and weak, hard-to-verify feedback.
Using large language models (LLMs) to assist psychological counseling is an important task in the field of natural language processing. The construction of high-quality psychological support dialogue corpora serves as a critical foundation for training counseling-oriented conversational models. However, existing data generation approaches generally suffer from several limitations, including emotionally stable seekers, limited variation in emotional dynamics, and a high degree of compliance with counselors' guidance. These issues result in LLM that lack the capability to effectively respond to emotionally unstable scenarios. In addition, counselor responses are typically driven by problem-solving objectives, thereby overlooking the role of emotion-focused interaction, which are essential in psychological counseling. To address these gaps, we propose EmoTrace, a multi-turn dialogue corpus generation framework centered on modeling seekers' emotional trajectories. we construct seekers' cognitive profile and introduce a seeker module with emotional schemas and an associated activation mechanism, a counselor module, and an emotional trajectory control module, thereby enhancing the layering of the seeker's emotional expression and the counselor's targeted empathic expression. Experimental results demonstrate that the proposed method outperforms existing approaches in terms of emotional richness and empathy quality.
Kaitong Weng, Lixin Liu, Zihao Liu +2
Shenzhen International Graduate School, Tsinghua University
Reinforcement learning (RL) for large language models (LLMs) has shown strong performance in single-turn tasks, but extending it to multi-turn interaction remains challenging due to sparse rewards and poor per-turn credit assignment. In emotional support dialogues, responses shape future user states, so matched-state step-wise comparison is unavailable, while trajectory-level supervision is insufficient. We propose MICA (Multi-granularity Intertemporal Credit Assignment), a critic-free RL framework for multi-turn emotional support tasks. MICA derives both immediate and delayed credit from a shared potential function over the user's structured support state. Incremental Distance Reward measures the per-turn decrease in residual distance to the target state, while its Monte Carlo return captures delayed effects. After scope-specific normalization, the two signals form a mixed advantage for stable per-turn optimization without matched-state comparisons, rollout trees, or a learned critic. On EMPA, EQ-Bench, and EmoBench with Qwen2.5-7B-Instruct and Qwen3-8B/14B/32B, MICA consistently outperforms GRPO and REINFORCE++, achieving up to +42.5 on EMPA, while adding no rollout cost and remaining robust to reward judges. These results show that turn-aware credit assignment enables effective and practical multi-turn RL for interactive LLMs.
Naifan Zhang, Ruihan Sun, Jinwei Su +4
NatureSelect · Tsinghua University · South China Normal University +1