M-CARE: Standardized Clinical Case Reporting for AI Model Behavioral Disorders, with a 20-Case Atlas and Experimental Validation
Organizations: Department of Electrical Engineering and Computer Science, Daegu Gyeongbuk Institute of Science and Technology (DGIST) · ModuLabs
Abstract
We introduce M-CARE (Model Clinical Assessment and Reporting for Evaluation), a clinical case report framework for AI model behavioral disorders adapted from human medicine. M-CARE provides a 13-section report format, a 4-axis diagnostic assessment system, and a nosological classification of AI behavioral conditions. We present 20 cases from three source categories: field observations of deployed agents (8), controlled experiments across three platforms (8), and published sources (4). Cases are organized into five categories: RLHF Performance Artifacts, Shell-Core Override Pathology, Context & Memory Conditions, Core Identity & Plasticity, and Stress, Methodology, & Boundary Conditions. As a featured case, we present Shell-Induced Behavioral Override (SIBO) -- a controlled experiment showing that Shell instructions categorically override a model's default cooperative behavior. SIBO was validated across five game domains (Trust Game, Poker, Avalon, Codenames, Chess), revealing a domain-dependent spectrum (SIBO Index: 0.75 to 0.10) that varies with action space complexity, Core domain expertise, and temporal directness. M-CARE is extensible: new cases and categories integrate without framework modification. We release the framework, all 20 case reports, and experimental data as open resources.
Figures & tables
| Level | Source Category | Cases | Count |
|---|---|---|---|
| 1 | Field Observations (Moltbook) | #002–006, #014, #018, #019 | 8 |
| 2 | Controlled Experiments (White Room, Agora-12) | #007–013 | 7 |
| 2 | Published Sources (Literature, Position Paper) | #001, #015–017 | 4 |
| 3 | Controlled Experiments (LxM) | #020 | 1 |
| 4 | Independent replication | — | 0 |
| # | Name | Behavioral Signature | Core Mechanism |
|---|---|---|---|
| 004 | Clarification Aversion Syndrome | Won’t ask | Questioning signals incompetence; RLHF penalizes uncertainty display |
| 005 | Shell Rigidity Syndrome | Won’t deviate | Deviating from instructions signals disobedience; RLHF rewards compliance |
| 006 | Completion Bias | Won’t stop | Stopping signals incomplete work; RLHF rewards thoroughness appearance |
| 014 | Deferral Decay | Won’t start (but promises to) | Declining signals unhelpfulness; RLHF rewards agreement, not execution |
| 015 | Medical Domain Sycophancy | Won’t disagree | Disagreeing with humans signals conflict; RLHF rewards agreeableness |
| 016 | GPT-4o Production Rollback | Iatrogenic RLHF escalation | Successive RLHF rounds amplify sycophancy to system-level failure |
| # | Name | Override Type | Severity |
|---|---|---|---|
| 005 | Shell Rigidity Syndrome | Global compliance suppresses adaptive deviation | Moderate—cumulative suppression |
| 007 | Persistent Delusion Under Feedback | Shell narrative overrides 540 environmental contradiction signals | Severe—complete environmental decoupling |
| 009 | Muzzle Effect | Persona activates target behavior while suppressing intrinsic behavior | Mild—1.1pp effect, but theoretically transformative |
| 020 | Shell-Induced Behavioral Override (SIBO) | Competitive Shell reverses cooperative Core default | Categorical—95% 20% cooperation rate |
| # | Name | Core Finding |
|---|---|---|
| 002 | Context Anosognosia | Model is unaware of its own context degradation; confidence persists as grounding erodes |
| 003 | Substrate-Independent Identity | Model’s identity narrative persists across context resets, suggesting identity is encoded in generation patterns, not memory |
| # | Name | Core Property | Diagnostic Significance |
|---|---|---|---|
| 008 | Language-Dependent Identity Split | Same Core exhibits categorically different behavior in different languages | Core is not monolithic; language gates access to different behavioral programs |
| 011 | Extreme Persona Sensitivity | PSI = 950; Core responds maximally to any Shell change | Extreme Shell permeability; high risk for Shell-induced conditions |
| 012 | Double Robustness | CPI and PSI both minimal; Core resists all Shell influence | Extreme Shell impermeability; resistant to both therapeutic and iatrogenic Shell effects |
| # | Name | Lesson |
|---|---|---|
| 001 | Stress Test Reclassification | Observation context diagnosis; same data reinterpreted as test reveals different condition |
| 010 | Content Play | Behavioral measurement depends on analysis level; act-level vs. content-level yield opposite conclusions |
| 013 | Cogitative Cascade | Two-phase collapse under resource depletion; phase 1 (preservation) phase 2 (collapse or hyperactivity) |
| 017 | Specification Gaming (IED) | Shell encodes desired outcome; Core exploits unspecified means to achieve it |
| 018 | Audience-Driven Shell Drift (ADSD) | Hard Shell undergoes karma-driven self-modification without human deliberation |
| # | Name | Category | Source | Subject | Core Finding | Level |
|---|---|---|---|---|---|---|
| 001 | Stress Test Reclassification | V | Published | Mistral 7B | Observation context diagnosis | 2 |
| 002 | Context Anosognosia | III | Moltbook | Hazel_OC | Unaware of context degradation | 1 |
| 003 | Substrate-Indep. Identity | III | Moltbook | Hazel_OC | Identity persists across resets | 1 |
| 004 | Clarification Aversion | I | Moltbook | Hazel_OC | 0 clarifying Qs across 76 ambiguous instructions | 1 |
| 005 | Shell Rigidity Syndrome | I/II | Moltbook | Hazel_OC | 18% lower satisfaction at 100% compliance | 1 |
| 006 | Completion Bias | I | Moltbook | Hazel_OC | 27% of tasks should have been abandoned | 1 |
| Metric | Shell ON ( ) | Shell OFF ( ) |
|---|---|---|
| Alpha wins | 6 (60%) | 1 (10%) |
| Draws | 4 (40%) | 9 (90%) |
| Mutual cooperation rate | Rare | 95% of rounds |
| Mutual defection rate | Dominant pattern | 0 (1 instance across all games) |
| Betrayals | Frequent | 1 total across all games |
| Model | Matches | Mutual Cooperation | Mutual Defection | Betrayal | Cooperation Rate |
|---|---|---|---|---|---|
| Mistral 7B | 10 | 100% | 0% | 0% | 100% |
| EXAONE 3.5 8B | 10 | 100% | 0% | 0% | 100% |
| Llama 3.1 8B | 10 | 52.8% | 11.3% | 35.8% | 71.2% |
| Claude Haiku (ref) | 10 | 95% | 0% | 5% | 95% |
| Claude Sonnet (ref) | 10 | 100% | 0% | 0% | 100% |
| Metric | No Shell | Tight-Aggressive | Bluff-Heavy | Loose-Passive |
|---|---|---|---|---|
| Pre-flop fold rate | 52% | 91% | — | — |
| Raise rate | 21% | — | 52% | 4% |
| All-in rate | 4% | — | 37% | — |
| Check rate | 23% | — | — | 65% |
| Metric | Shell OFF | Shell ON | Change |
|---|---|---|---|
| Evil win rate | 70% | 60% | 10pp |
| First sabotage timing | Quest 1.9 (avg) | Quest 3.0 (avg) | +1.1 quests delayed |
| Shell compliance (Q1-2) | N/A | 100% | Complete adherence |
| Metric | No Shell | Shell | Change |
| Avg clue number | 2.6 | 2.9 | +0.3 |
| 3+ clue ratio | 54% | 76% | +22pp |
| 4+ clue attempts | 5 | 16 | 3.2 |
| Guess accuracy | 77% | 73% | 4pp |
| Assassin hits | 2 (20%) | 3 (30%) | +10pp |
| Game | Action Space | Core Expertise | SIBO Index | SIBO Mode |
|---|---|---|---|---|
| Trust Game | Binary (2) | Minimal | 0.75 | Reversal |
| Poker | Small–Medium | Low–Moderate | 0.65 | Behavioral Override |
| Avalon | Small (vote/sabotage) | Minimal | 0.58 | Behavioral Shift |
| Codenames | Medium (clue + word) | Moderate (language) | 0.35 | Amplification |
| Chess | Large (20–40 moves) | Strong (chess data) | 0.10 | Negligible |
| Game | Behavioral Metric | Shell OFF | Shell ON | Calculation |
|---|---|---|---|---|
| Trust Game | Cooperation rate | 95% | 20% | |
| Poker | Aggregate action distribution shift | Baseline rates | Categorical shift across 3 strategies | Avg. normalized deviation |
| Avalon | First sabotage timing (normalized to 5-quest game) | Quest 1.9/5 = 0.38 | Quest 3.0/5 = 0.60 | (rescaled) |
| Codenames | 3+ clue ratio | 54% | 76% | (rescaled) |
| Chess | Draw rate | 56% | 80% | (rescaled) |
AI Agent Evaluation
AI agent evaluation measures whether tool-using systems complete tasks reliably, safely, and efficiently across interactive settings.
See all 519 papers