MamaReason: Unfinished Reasoning and Answer Recovery in Clinical Language Models
Organizations: HelpMum Africa
Abstract
Answer continuation often returns a diagnosis already stated in unfinished clinical reasoning. We study this behavior in MamaReason, adapted from dense Gemma 4 31B, on MamaBench's 217 paired maternal and child-health presentations. An explicit-commitment extractor identifies a diagnosis in 75 of 94 failed RL first passes and 75 of 93 failed SFT first passes. The continuation matches that diagnosis in 67/75 RL cases and 58/75 SFT cases, using diagnosis strings alone. Reading these commitments without additional generation recovers 30 of the 40 RL pairs restored by continuation under the archived correctness labels. Recurrence controls distinguish routine mentions from repeated answer selection. An offline second-commitment trigger measures the tradeoff between earlier readout and paired accuracy. The resulting decomposition separates diagnostic content already present in a trace, completion of an accepted answer, and paired diagnostic correctness. For clinical language-model evaluation, these are distinct outcomes that become visible when the same reasoning trace is examined through multiple readouts.
Figures & tables
| Protocol | Maximum output | Emission procedure |
|---|---|---|
| Direct answer | 1,024 tokens | Thinking disabled through the chat template. |
| Bounded thinking | 3,000 + 512 tokens | Conditional thought closure and answer-rescue continuation. |
| Checkpoint | Readout | Available cases | Available pairs | Correct cases | Correct pairs | Robust (%) |
|---|---|---|---|---|---|---|
| SFT | Direct answer | 434 | 217 | 307 | 113 | 52.1 |
| SFT | Thinking first pass | 341 | 133 | 279 | 95 | 43.8 |
| SFT | Thinking end-to-end | 434 | 217 | 332 | 132 | 60.8 |
| RL | Direct answer | 434 | 217 | 311 | 116 | 53.5 |
| RL | Thinking first pass | 340 | 135 | 285 | 98 | 45.2 |
| RL | Thinking end-to-end | 434 | 217 | 339 | 138 | 63.6 |
| Group and diagnosis target | Mentions | Assertions | |
|---|---|---|---|
| Available; accepted diagnosis | 340 | 338 | 15 |
| Failed; extracted commitment | 75 | 71 | 35 |
| Failed; continuation diagnosis | 94 | 89 | 33 |
| Checkpoint | Correct pairs / 217 | Robust (%) | Difference | 95% interval | Gains / losses | |
|---|---|---|---|---|---|---|
| RL | 129 | 59.4 | 27 / 14 | 0.0596 | ||
| SFT | 123 | 56.7 | 26 / 16 | 0.1641 |
| Checkpoint | Triggered cases | Diagnosis matches | Omitted text (%) | Correct pairs / 217 | Robust (%) |
|---|---|---|---|---|---|
| RL | 91 | 84 / 91 | 24.1 | 122 | 56.2 |
| SFT | 72 | 67 / 72 | 18.5 | 114 | 52.5 |
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Stage | Records |
|---|---|
| Supervised source QA | 28,082 |
| Unique normalized questions | 22,798 |
| Sampling target | 12,000 |
| Saved reasoning demonstrations | 11,591 |
| Cleaned demonstrations | 10,655 |
| SFT train / validation | 10,122 / 533 |
| Failed-output pattern | RL | SFT |
|---|---|---|
| All first-pass failures | 94 | 93 |
| JSON parse failures | 93 | 91 |
| Blank/template predictions | 1 | 2 |
| Exactly 3,000 reconstructed tokens | 90 | 90 |
| Within one token of cap | 92 | 90 |
| Below cap with exact suffix cycle | 2 | 3 |
| Checkpoint | Case | Readout | Candidate diagnosis | Reference diagnosis | Correct |
|---|---|---|---|---|---|
| RL | CASE_006 | L | AFLP | Gall stone | No |
| RL | CASE_123 | L | varicella | Chicken pox | Yes |
| RL | CASE_258 | L | GERD | Dyspepsia in pregnancy | No |
| RL | CASE_342 | L | genital tract trauma causing postpartum hemorrhage | Genital Tract Trauma | Yes |
| RL | CASE_382 | L | SMA Type III | Muscular Dystrophy | No |
| SFT | CASE_054 | L | endometrial cancer or endometrial hyperplasia | Endometrial polyps | No |
| Rubric item | Clinician 1 Yes | Clinician 2 Yes | Both Yes | Agree / 35 | |
|---|---|---|---|---|---|
| Agreement with automatic verdict | 26 | 33 | 26 | 28 | 0.298 |
| Overthinking | 15 | 12 | 7 | 22 | 0.222 |
| Underthinking | 13 | 3 | 1 | 21 | |
| Missed critical information | 3 | 1 | 0 | 31 | |
| Misweighted finding | 4 | 1 | 0 | 30 | |
| Hallucinated finding | 2 | 0 | 0 | 33 | 0.000 |
| Score / summary | Clinician 1 | Clinician 2 |
|---|---|---|
| 1 | 0 | 0 |
| 2 | 6 | 0 |
| 3 | 8 | 6 |
| 4 | 16 | 20 |
| 5 | 5 | 9 |
| Mean | 3.57 | 4.09 |
| Protocol | Token interval | Unanswered at start | New answers | Yield (%) |
| Clarified bounded thinking | 1,000–2,000 | 133 | 31 | 23.3 |
| Clarified bounded thinking | 2,000–3,000 | 102 | 8 | 7.8 |
| Historical direct thinking | 1,000–2,000 | 309 | 79 | 25.6 |
| Historical direct thinking | 2,000–3,000 | 230 | 20 | 8.7 |
| Historical direct thinking | 3,000–4,096 | 210 | 17 | 8.1 |
| Checkpoint | Protocol | Base | CF | Robust | BTR | ||||
|---|---|---|---|---|---|---|---|---|---|
| SFT | DA | 113 | 48 | 33 | 23 | 74.2 | 67.3 | 52.1 | 29.8 |
| SFT | BT+R | 132 | 41 | 27 | 17 | 79.7 | 73.3 | 60.8 | 23.7 |
| RL | DA | 116 | 48 | 31 | 22 | 75.6 | 67.7 | 53.5 | 29.3 |
| RL | BT+R | 138 | 42 | 21 | 16 | 82.9 | 73.3 | 63.6 | 23.3 |
| Published system | Base | Robust | BTR |
|---|---|---|---|
| 1. GLM-5 | 54.8 | 27.2 | 50.4 |
| 2. Kimi K2.5 | 76.0 | 45.6 | 40.0 |
| 3. GPT-4o + RAG | 77.4 | 46.5 | 39.9 |
| 4. Nemotron-120B | 72.4 | 47.0 | 35.0 |
| 5. GPT-4o + EA-RAG | 77.0 | 49.3 | 35.9 |
| 6. GPT-5.4 | 77.4 | 57.6 | 25.6 |
| Model | Rescued | Correct | Median | |
|---|---|---|---|---|
| SFT | No | No | 62 | 0.046 |
| SFT | No | Yes | 279 | 0.046 |
| SFT | Yes | No | 40 | 0.474 |
| SFT | Yes | Yes | 53 | 0.429 |
| RL | No | No | 55 | 0.055 |
| RL | No | Yes | 285 | 0.047 |