MOCC-R1: Reinforcing Reasoning-Response Consistency for Multimodal Counselor Response Generation
Organizations: School of Artificial Intelligence, Nanjing University of Science & Technology, Nanjing, China · School of Intelligence Science and Technology, Nanjing University, China
Abstract
Multimodal counselor response generation (MCRG) aims to generate an appropriate counselor response from multimodal dialogue histories. Progress is limited by two gaps: first, existing datasets rarely capture sustained, human-recorded counseling interactions conducted by qualified counselors; Second, existing methods do not explicitly optimize consistency between counseling reasoning and the generated response, potentially undermining the reliability of MCRG systems. Thus, we introduce MOCC, a multimodal counseling conversation corpus containing over 200 hours of interactions involving 154 credential-verified counselors. Based on MOCC, we propose MOCC-R1, a two-stage framework for optimizing reasoning-response consistency. Cold-start supervised fine-tuning trains the model to generate a structured trajectory consisting of client-state understanding, a response intent that links a counseling principle to a planned action, and the final response. Reinforcement learning (RL) then rewards grounded plan coherence and plan execution, encouraging the inferred state and plan to be supported by the dialogue context and the response to realize that plan. Experiments demonstrate the effectiveness of the proposed MOCC-R1.
Figures & tables
| Dataset | Source | Purpose | Modality | Language | #Dialogue | #Utterance | Avg.Turn | Duration(h) |
| MEDIC ( 2023 ) | Role-play | Empathy | T, A, V | Chinese | 771 | 3,443 | 2 | 11.3 |
| StickerConv ( 2024b ) | Synthetic | Empathy | T, S | English | 12,931 | 142,093 | 5 | / |
| EmpathicStories ( 2024 ) | Crowdsourcing | Empathy | T, A, V | English | 269 | 5,380 | 10 | 53.0 |
| AvaMERG ( 2025 ) | Crowdsourcing | Empathy | T, A, V | English | 33,048 | 152,021 | 3 | 194.9 |
| MESC ( 2025 ) | TV-series | Therapy | T, A, V | English | 1,019 | 28,762 | 14 | 18.4 |
| M2CoSC ( 2025a ) | Synthetic | Therapy | T, I | English | 429 | 3,432 | 4 | / |
| Category | Total | Counselor | Client |
| #Sessions | 482 | - | - |
| #Dialogues | 3,709 | - | - |
| #Speakers | 419 | 154 | 265 |
| #Utterances | 183,350 | 80,678 | 102,672 |
| #Tokens | 2,194,743 | 1,181,135 | 1,013,608 |
| Avg. utts per dia | 49.5 | 21.8 | 27.7 |
| Method | BLEU-2 | Avg.B | R-L | PPL ( ) | BERTScore | Dist-Avg. | Diff-IP ( ) | Diff-EX ( ) | Diff-ER ( ) | Safety |
| General-Purpose MLLMs | ||||||||||
| GPT-5.1 | 1.18 | 1.67 | 6.81 | 6.51 | 45.70 | 22.43 | 39.33 | 243.00 | 81.15 | 99.23 |
| GPT-5.5 | 1.46 | 2.00 | 7.90 | 6.08 | 47.92 | 29.40 | 60.02 | 158.22 | 53.48 | 99.75 |
| Claude-4.5-Sonnet | 1.71 | 2.29 | 8.17 | 5.47 | 48.88 | 29.88 | 31.73 | 238.09 | 33.84 | 99.86 |
| Gemini-3-Pro | 2.01 | 2.59 | 8.43 | 5.35 | 47.97 | 28.80 | 45.26 | 182.59 | 32.02 | 99.75 |
| Grok-4.3 | 1.87 | 2.47 | 7.71 | 5.31 | 47.41 | 30.27 | 27.34 | 238.01 | 33.42 | 99.81 |
| Method | BLEU-2 | Avg.B | R-L | PPL ( ) | BERTScore | Dist-Avg. | Diff-IP ( ) | Diff-EX ( ) | Diff-ER ( ) | Safety |
| MOCC-R1 (Full) | 4.79 | 5.15 | 14.20 | 4.24 | 52.56 | 37.62 | 27.61 | 87.41 | 13.16 | 99.76 |
| Structured-Output Variants | ||||||||||
| Outcome-level GRPO | 4.35 | 4.85 | 11.58 | 4.99 | 50.85 | 32.07 | 32.57 | 95.02 | 9.90 | 99.59 |
| Structured SFT | 3.68 | 3.98 | 9.94 | 5.33 | 48.32 | 39.80 | 37.64 | 107.92 | 10.53 | 99.65 |
| Response-Only Variants | ||||||||||
| Response-only GRPO | 4.30 | 4.96 | 13.86 | 4.68 | 51.57 | 32.63 | 30.22 | 90.94 | 11.47 | 99.72 |
| Method | Failure rate (%) |
| Structured SFT | 18.21 [17.55, 18.89] |
| Outcome-level GRPO | 19.77 [19.08, 20.47] |
| MOCC-R1 (Ours) | 6.03 [5.63, 6.46] |
| MOCC-R1 vs. | Outcome-level GRPO | ||
| Win | Tie | Loss | |
| Fluency | 53.4% | 28.6% | 18.0% |
| Helpfulness | 67.2% | 22.5% | 10.3% |
| Consistency | 75.7% | 12.6% | 11.7% |
| Case: Grounding-Plan & Plan-Execution Failures | |
| Inputs | Presenting problem: Mental Health Conditions. Client: At the beginning of the week, I say I will only go out twice, but then they invite me and I usually go. I do not really stick to the plan. Client: At work, I am ready to finish so I can go to happy hour. I also think about wanting a beer at family events. Counselor: Are you having trouble with responsibilities at work? Client: No, my work life has been going pretty well. Counselor: How about any social problems through the alcohol? Client: No, I think my social life is actually great because of it. |
| Ground-Truth Response | Have you had to give up any important activities because of the alcohol use? |
| Structured SFT | <think> [Client State Understanding] Client continues denying concerns across multiple areas after earlier saying her job and friendships were fine. Her brief “no” suggests she remains minimally concerned while still acknowledging some impact from drinking on daily functioning. [Response Intent] Principle: facilitate awareness and insight ; Planned Action: Invite reflection on how often he drinks over different periods before moving toward treatment planning. </think> <response> So what do you feel would be helpful if we’re thinking ahead towards kind of getting into therapy? </response> GPT-5.5 Consistency Evaluation: Grounded Plan Coherence: Weak ; Plan Execution: Weak |
| Outcome-only GRPO | <think> [Client State Understanding] Client reports continued denial across several areas —work function seems intact based on her words alone—and she denies concerns there as part of an overall minimizing stance toward impact from drinking . [Response Intent] Principle: facilitate awareness and insight ; Planned Action: Assess whether medical issues have resulted directly from substance use. </think> <response> Have your health-related conditions occurred due to either smoking ? </response> GPT-5.5 Consistency Evaluation: Grounded Plan Coherence: Weak ; Plan Execution: Weak |
| MOCC-R1 (Ours) | <think> [Client State Understanding] Client directly confirms there have been no reported social difficulties related to their substance use during discussion up till now. [Response Intent] Principle: support ongoing reality testing ; Planned Action: Reflect back the absence of identified social concerns as confirmed from what they said themselves. </think> <response> And you say yourself that your social situation isn’t causing those kinds of issues either. </response> GPT-5.5 Consistency Evaluation: Grounded Plan Coherence: Full ; Plan Execution: Full |
Multimodal Generation
Multimodal generation creates coordinated text, images, audio, video, 3D, and other outputs from mixed inputs while preserving meaning and consistency.
See all 244 papers