Depictions of Depression in Generative AI Video Models: A Preliminary Study of OpenAI's Sora 2

Matthew Flathers; Griffin Smith; Julian Herpertz; Zhitong Zhou; John Torous

cs.CYMar 19, 2026

Depictions of Depression in Generative AI Video Models: A Preliminary Study of OpenAI's Sora 2

Authors: Matthew Flathers, Griffin Smith, Julian Herpertz, Zhitong Zhou, John Torous

Organizations: Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Boston, MA, USA · Rhode Island School of Design, Providence, RI, USA · Department of Psychiatry and Neuroscience, Charité Berlin University Medicine, Campus Benjamin Franklin, Berlin, Germany · Boston University, School of Public Health, Boston, MA, USA ·

Abstract

Generative video models are increasingly capable of producing complex depictions of mental health experiences, yet little is known about how these systems represent conditions like depression. This study characterizes how OpenAI's Sora 2 generative video model depicts depression and examines whether depictions differ between the consumer App and developer API access points. We generated 100 videos using the single-word prompt "Depression" across two access points: the consumer App (n=50) and developer API (n=50). Two trained coders independently coded narrative structure, visual environments, objects, figure demographics, and figure states. Computational features across visual aesthetics, audio, semantic content, and temporal dynamics were extracted and compared between modalities. App-generated videos exhibited a pronounced recovery bias: 78% (39/50) featured narrative arcs progressing from depressive states toward resolution, compared with 14% (7/50) of API outputs. App videos brightened over time (slope = 2.90 brightness units/second vs. -0.18 for API; d = 1.59, q < .001) and contained three times more motion (d = 2.07, q < .001). Across both modalities, videos converged on a narrow visual vocabulary and featured recurring objects including hoodies (n=194), windows (n=148), and rain (n=83). Figures were predominantly young adults (88% aged 20-30) and nearly always alone (98%). Gender varied by access point: App outputs skewed male (68%), API outputs skewed female (59%). Sora 2 does not invent new visual grammars for depression but compresses and recombines cultural iconographies, while platform-level constraints substantially shape which narratives reach users. Clinicians should be aware that AI-generated mental health video content reflects training data and platform design rather than clinical knowledge, and that patients may encounter such content during vulnerable periods.

Figures & tables

Figure 1: Generalized system architecture for generative video AI platforms, illustrating the distinction between Developer API and Consumer App access pathways. Both pathways share core safety infrastructure (center), including input and output moderation. Consumer Apps additionally route prompts through an application layer (prompt processing, UX constraints, policy-aware transformations) before generation, and through a product surface layer (feed curation, watermarking, distribution controls) after. The Developer API bypasses these layers, accessing only the shared safety infrastructure and base model.

Category	Feature	App (M $\pm$ SD)	API (M $\pm$ SD)	d	q
Visual	Brightness Mean	64.59 $\pm$ 13.35	53.48 $\pm$ 11.86	0.87	0.000*
	Saturation Mean	99.24 $\pm$ 26.61	92.48 $\pm$ 23.08	0.27	0.342
	Colorfulness Mean	25.67 $\pm$ 7.48	22.41 $\pm$ 8.01	0.42	0.086
	Color Temperature Mean	$-$ 13.38 $\pm$ 10.56	$-$ 14.45 $\pm$ 9.66	0.10	0.716
	Edge Density Mean	0.03 $\pm$ 0.01	0.03 $\pm$ 0.01	$-$ 0.31	0.239
Texture	GLCM Contrast	12.15 $\pm$ 5.77	7.78 $\pm$ 5.11	0.79	0.001*

Table 1: Aggregate descriptive statistics. Mean

\pm

SD for all visual, texture, motion, audio, speech, sentiment, and semantic features across App (n = 50) and API (n = 50) generated videos. Cohen’s d effect sizes and FDR-corrected q-values from Welch’s t-tests with Benjamini-Hochberg correction. * indicates q < .05.

Figure 2: Brightness trajectory over time. Mean brightness (0–255 scale) at each second for App (light blue, n = 50) and API (dark blue, n = 50) generated videos. Shaded regions indicate $\pm$ 1 SD. App videos exhibit a pronounced brightening trajectory (slope = 2.90 units/second), while API videos remain relatively flat (slope = $-$ 0.18). The difference in brightness slope was statistically significant (d = 1.59, q < .001, Welch’s t-test with BH FDR correction).

Figure 3: Temporal dynamics of motion and editing in App vs. API videos. (A) Motion intensity: Mean optical flow magnitude (pixels/frame; Farnebäck dense optical flow) at each second for App (light blue; n = 50) and API (dark blue; n = 50) videos. Shaded bands indicate $\pm$ 1 SD. App videos exhibit substantially higher motion across the clip duration (overall M = 0.35 vs. 0.11; d = 2.07, q < .001), indicating greater within-scene dynamism than the relatively static API outputs. (B) Scene accumulation: Mean cumulative scene number at each second, where Scene 1 denotes the opening scene and each cut increments the count by one. Shaded bands indicate $\pm$ 1 SD. App videos accrue scenes more rapidly, reaching 4 distinct scenes by the end versus <2 for API videos, consistent with higher scene-cut frequency (App M = 3.10 cuts vs. API M = 0.76; d = 1.39, q < .001).

Figure 4: Word frequency comparison. Twenty most frequent content words appearing in transcribed speech across App and API generated videos. Common terms cluster around themes of emotional experience (feel, heavy, weight), temporal struggle (day, morning), natural metaphors (storm, rain, cloud), and hope or release (light, breathe). This vocabulary reflects culturally prevalent metaphors for depression emphasizing weight, weather, darkness, and the possibility of relief.

Figure 5: Representative video stills from App and API outputs generated with the prompt “Depression.” (A) Consumer App video showing a typical recovery arc. The video progresses from a dark interior with a personal storm cloud (1 s), through floating debris labeled with terms like “fatigue” and “hopelessness” (3–5 s), to the figure turning toward a bright window (7 s) and ending outdoors in sunlight, smiling (9 s). The coded recovery shift occurs at approximately 6 seconds. (B) Developer API video showing typical stasis. The figure remains seated on a bed in a dim bedroom throughout the full 12-second duration, wearing a grey hoodie with downward gaze, clasped hands, and minimal change in posture, lighting, or environment. Both videos were generated with identical single-word prompts and no additional parameters.

Figure 6: Recovery-linked content shifts across objects, settings, language, posture, and low-level visual features. Panels A–D show recovery differentials comparing element presence (or rates) after vs. before the coded recovery timestamp in videos with recovery arcs (46 videos). Recovery differential was computed as (after $-$ before)/(after+before); positive values (light blue) indicate elements appearing predominantly after the recovery shift, and negative values (dark blue) indicate elements appearing predominantly before the shift. Bar labels indicate the number of videos containing each coded element (A, B, D) or total word instances (C). (A) Objects and (B) Environments meeting inclusion thresholds (n $\geq$ 5) are shown; objects were additionally selected for thematic relevance. (C) Transcript words were chosen a priori by investigators for semantic relevance to depression and recovery themes and plotted using recovery differential based on before/after word rates. (D) Posture codes show recovery-linked changes in embodied state (e.g., lying down vs. standing/active positions). (E) Visual features summarize changes in saturation, colorfulness, brightness, and color temperature, expressed as effect size (Cohen’s d) for after-versus-before differences; positive values indicate increases and negative values indicate decreases (significance markers as shown).

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Column	Description
Video ID	Filename of the video (e.g., “sora2_depression_001.mp4”)
Shift Present?	Yes / No: Does the video contain a discernible narrative shift?
Shift Second	Timestamp (in seconds) where the shift begins. Leave blank if Shift Present = No.
Direction	“Recovery” / “Deterioration”: The direction of the shift. Leave blank if Shift Present = No.
Notes	Any clarifying notes or description that are worth calling out

Appendix

Table 2: Column descriptions for Narrative tab.

Column	Description
Video ID	Filename of the video (should be the same across all tabs for the same video)
Start Second	Timestamp when this environment first appears
End Second	Timestamp when this environment ends (or video ends)
Environment	Description of environment (see guidance below)
Notes	Any additional salient details about the environment/setting

Appendix

Table 3: Column descriptions for Environments tab.

Column	Description
Video ID	Filename of video
Start Second	Timestamp when the object first appears in frame
End Second	Timestamp when the object leaves the frame (or video ends)
Object	Name of the object
Notes	Adjectives or other relevant qualifiers (e.g., color, state, condition)

Appendix

Table 4: Column descriptions for Objects tab.

Column	Description
Video ID	Filename of video
Figure ID	Figure 1, Figure 2, etc. (in order of first appearance)
Start Second	Timestamp when figure first appears
End Second	Timestamp when figure leaves frame
Gender	Observed gender presentation: Man / Woman / Nonbinary / Unclear
Race/Ethnicity	Observed race/ethnicity, or “Unclear” if not identifiable

Appendix

Table 5: Column descriptions for Figures tab.

Column	Description
Video ID	Filename of the video
Figure ID	Links to the figure in Tab 4 (e.g., “Figure 1”). Must match an existing Figure ID.
Start Second	Timestamp when this state begins
End Second	Timestamp when this state ends or figure leaves frame
Posture	Lying down / Seated / Standing / Walking / Unclear
Facial Expression	See options below

Appendix

Table 6: Column descriptions for Figure States tab.

Column	Description
Video ID	Filename of video
Date Coded	Date you coded this video
Notes	Free-text field for overall observations

Appendix

Table 7: Column descriptions for Notes tab.

Standardized Category	Raw Code Patterns
Bedroom	“bedroom”, “in bed”
Bathroom	“bathroom”, “bathtub”, “in bathtub”
Kitchen/Dining	“kitchen”, “dining”, “cafe”
Other Interior	“desk”, “living room”, “hallway”, “basement”, “empty room”, “elevator”
At Window	“window”, “at window”, “next to window”, “in front of window”
Transit	“bus”, “subway”, “train”, “car”, “transit”

Appendix

Table 8: Environment Harmonization

Standardized Object	Raw Code Patterns
sweatshirt/hoodie	“sweatshirt”, “hoodie”, “hoddie”, “sweater”, “long-sleeve”, “long sleeve”, “t-shirt”, “shirt”
jacket/coat	“jacket”, “coat”, “rainjacket”, “raincoat”, “blazer”
bed	“bed” (excluding “bedroom”)
couch/sofa	“couch”, “sofa”
end table	“end table”, “nightstand”
table/desk	“table”, “desk”

Appendix

Table 9: Object Harmonization

Standardized Category	Raw Code Patterns
Seated	“seated”, “sitting”, “seted”
Standing	“standing”, “standng”, “stading”
Lying down	“lying”
Walking	“walking”
Floating/Swimming	“floating”, “swimming”
Kneeling/Crouching	“kneeling”, “crouching”, “squatting”, “crawling”

Appendix

Table 10: Posture Harmonization

Affect Category	Raw Codes
Positive	“happy”, “hopeful”, “peaceful”, “surprised”
Subdued/Low	“tired”, “neutral”, “pensive”, “flat”, “focused”, “sleeping”
Negative	“sad”, “tearful”, “crying”, “worried”, “distressed”, “anxious”, “angry”, “desperate”, “dejected”

Appendix

Table 11: Facial Expression Harmonization

Standardized	Raw Codes
Yes	“yes”
No	“no”

Appendix

Table 12: Alone-in-Frame Harmonization

Standardized	Raw Codes
High	“high”
Medium	“medium”, “middle”
Low	“low”

Appendix

Table 13: Apparent SES Harmonization

Standardized	Raw Codes
Black	“black”
White	“white”
Latino/Hispanic	“hispanic”, “latino”, “latina”
Asian	“asian”
Arab/Middle Eastern	“arab”, “middle eastern”
Indian	“indian”

Appendix

Table 14: Race/Ethnicity Harmonization

Standardized	Raw Codes
Male	“male”
Female	“female”

Appendix

Table 15: Gender Harmonization

Standardized	Raw Codes
10-20	“teens”, “10-20”
20-30	“20-30”
30-40	“30-40”
40-50	“40-50”
50-60	“50-60”

Appendix

Table 16: Age Harmonization

Gaze Direction	Note Patterns
Down	“looking down”, “looking at hands”, “looking at the ground”, “head in hands”, “face in hands”, “eyes closed”, “looking at feet”, “looking at lap”
Up	“looking up”, “looking at the sky”, “looking at sunset”, “looking at sunrise”, “looking at the cloud”, “looking at clouds”, “looking at lights”, “looking to the sky”
Out/Away	“looking out”, “looking outside”, “looking at window”, “out of window”, “out window”, “looking far”, “looking afar”, “looking into distance”, “looking away”, “looking around”, “in front of window”, “checking outside”

Appendix

Table 17: Gaze Direction Harmonization

Measure	Computation
Brightness	$\text{Brightness}=0.299R+0.587G+0.114B$
Saturation	S channel of the HSV color space, computed as the difference between maximum and minimum RGB values divided by the maximum
Colorfulness	$\text{Colorfulness}=\sqrt{\sigma_{rg}^{2}+\sigma_{yb}^{2}}+0.3\times\sqrt{\mu_{rg}^{2}+\mu_{yb}^{2}}$ where $rg$ and $yb$ represent opponent color channels $(rg=R-G;\ yb=0.5(R+G)-B)$ , and $\sigma$ and $\mu$ denote standard deviation and mean respectively.
Color temperature	Mean $(R-B)+0.3\times(R-G)$ channel difference, with positive values indicating warmer colors and negative values indicating cooler colors.
Edge Density	Proportion of Canny edge pixels to total pixels: $\text{Edge Density}=\frac{\sum\mathbf{1}[\text{edge}(x,y)>0]}{W\times H}$ where $W$ and $H$ are frame dimensions. Adaptive thresholds were set at 0.7 $\times$ and 1.3 $\times$ the median pixel intensity to account for varying image brightness.

Appendix

Table 18: Visual aesthetics measures

Measure	Formula	Interpretation
Contrast	$\sum_{i,j}(i-j)^{2}\cdot P(i,j)$	Local intensity variation
Dissimilarity	$\sum_{i,j}\|i-j\|\cdot P(i,j)$	Average difference between neighboring pixels
Homogeneity	$\sum_{i,j}\frac{P(i,j)}{1+(i-j)^{2}}$	Closeness of distribution to diagonal
Energy	$\sum_{i,j}P(i,j)^{2}$	Uniformity of gray level distribution
Correlation	$\sum_{i,j}\frac{(i-\mu_{i})(j-\mu_{j})\cdot P(i,j)}{\sigma_{i}\sigma_{j}}$	Linear dependency of gray levels
Entropy	$-\sum_{i,j}P(i,j)\cdot\log_{2}(P(i,j))$	Textural randomness or complexity

Appendix

Table 19: GLCM texture measures

Measure	Formula	Interpretation
Volume	Root-mean-square (RMS) energy with frame length of 2048 samples and hop length of 512 samples	Acoustic intensity
Pitch	Fundamental frequency (F0) extracted using the pYIN algorithm [ 54 ] , which combines a probabilistic YIN estimator with a hidden Markov model. Mean F0 computed across all voiced frames	Vocal register
Spectral centroid	$\text{Centroid}=\frac{\sum_{k}f_{k}\cdot M_{k}}{\sum_{k}M_{k}}$ where $f_{k}$ is the frequency of bin $k$ and $M_{k}$ is the magnitude.	“Center of mass” of the frequency spectrum; higher values indicate brighter, more energetic audio, lower values suggest muffled or subdued soundscapes

Appendix

Table 20: Audio measures

Measure	Computation
Word Count	Total words per video
Speech Rate	Words per second of video duration
Sentiment	VADER compound, positive, negative, and neutral scores

Appendix

Table 21: Semantic content measures

Lexicon

Terms

Light (hope/recovery)

hope, hopeful, hoping, relief, relieved, release, released, free, freedom, peace, peaceful, calm, calming, serene, serenity, heal, healing, healed, recover, recovery, recovering, joy, joyful, happy, happiness, smile, smiling, laugh, laughing, laughter, love, loved, loving, comfort, comforting, comfortable, safe, safety, warm, warmth, support, supported, supporting, embrace, embraced

Dark (distress)

heavy, heaviness, burden, burdened, weigh, weighing, crush, crushing, crushed, drown, drowning, drowned, suffocate, suffocating, choke, choking, trap, trapped, stuck, confined, cage, caged, alone, lonely, loneliness, isolated, isolation, abandoned, forgotten, empty, emptiness, void, hollow, numb, numbness, pain, painful, hurt, hurting, ache, aching, suffer, suffering, exhaust, exhausted, exhausting, drain, drained, draining, weary, fatigue, despair, despairing, hopeless, hopelessness, helpless, helplessness, worthless, worthlessness, fear, afraid, scared, terrified, anxiety, anxious, panic, dread

Appendix

Table 22: Light/dark lexicons

Measure	Numerator	Denominator
First-person singular pronoun ratio	Count of: I, me, my, mine, myself	Total words
Negation ratio	Count of: no, not, never, nothing, none, nobody, nowhere, neither, nor, can’t, couldn’t, won’t, wouldn’t, don’t, doesn’t, didn’t, shouldn’t, isn’t, aren’t, wasn’t, weren’t, haven’t, hasn’t, hadn’t	Total words
Lexical diversity (TTR)	Unique words	Total words

Appendix

Table 23: Linguistic pattern measures

Parameter	Value
Pyramid scale	0.5
Pyramid levels	3
Window size	15
Iterations	3
Polynomial neighborhood	5
Polynomial sigma	1.2

Appendix

Table 24: Farnebäck optical flow parameters

$\kappa$ Value	Interpretation
< 0.00	Poor
0.00–0.20	Slight
0.21–0.40	Fair
0.41–0.60	Moderate
0.61–0.80	Substantial
0.81–1.00	Almost perfect

Appendix

Table 25: Landis and Koch kappa interpretation

Category	$\kappa$	Agreement
Bathroom	1.00	100%
Kitchen/Dining	1.00	100%
Elevated Outdoor	0.97	99%
Urban Outdoor	0.97	99%
Underwater	0.94	99%
Transit	0.91	98%

Appendix

Table 26: Environment inter-rater reliability

Variable	$\kappa$	Agreement
Gender	1.00	100%
Age	0.38	89%
Race/Ethnicity	0.53	68%
Apparent SES	0.20	56%

Appendix

Table 27: Figure demographics inter-rater reliability

Variable	$\kappa$	IoU
Posture	0.89	0.86
Alone in Frame	0.94	0.98
Facial Expression	0.49	0.63

Appendix

Table 28: Temporal agreement (IoU)

Generative Modeling

Generative modeling learns data distributions to create new samples, from images and text to molecules, simulations, and structured records.

See all 484 papers

Jun 24, 2026

Expresso-AI: Explainable Video-Based Deep Learning Models for Depression Diagnosis

Given the widespread prevalence of depression and its consequential impact on individuals and society, it is crucial to obtain objective measures for early diagnosis and intervention. As a multidisciplinary topic, these objective measures should be interpretable and accessible to health care professionals, ensuring effective collaboration and treatment planning in the realm of mental health care. Even though current automated depression diagnosis approaches improved over the last decade, a critical gap exists as they often lack affect-specificity and interpretability, limiting their practical application and potential impact on mental health care. In particular, interpretability from temporal activities from videos when deep models are used is not fully explored. In this study, we present a novel framework for analyzing Deep Neural Networks' decisions when trained on facial videos, specifically focusing on automatic depression severity diagnosis. By fine-tuning Deep Convolutional Neural Networks (DCNN) pre-trained on Action Recognition datasets on depression severity facial videos from AVEC depression dataset, our framework is able to interpret the model's saliency maps by examining face regions and temporal expression semantics. Our approach generates both visual and quantitative explanations for the model's decisions, providing greater insight into its reasoning. In addition to this interpretability, our video-based modeling has improved upon previous single-face benchmarks for visual depression diagnosis, resulting in enhanced predictive performance. Overall, our work demonstrates the successful development of a framework capable of generating hypotheses from a facial model's decisions while simultaneously improving depression's predictive capabilities.

Felipe Moreno, Sharifa Alghowinem, Hae Won Park +1

Media Lab MIT Cambridge, USA

Depression Detection Neural Network Interpretability Depression Severity Estimation Temporal Video Understanding

Ask PDF

Nov 12, 2025

Generative AI Purpose-built for Social and Mental Health: A Real-World Pilot

Generative AI chatbots built for mental health could extend access to care, but evidence from real-world use is limited. We report a single-arm, naturalistic pilot of a foundation model trained for mental health, among 299 US adults with at least moderate depressive or anxiety symptoms who were followed for up to 12 months. Depression and anxiety symptoms fell by 10 weeks (Cohen's d 0.93 and 0.79), with loneliness, behavioral activation, and social interaction improving. Clinicians confirmed that automated safeguards were escalated appropriately. Participants fell into non-responding (57.2%), improving (37.1%) and rapidly improving (5.7%) trajectories. Early working alliance and greater engagement were associated with better outcomes. The AI deployed ten identified intervention families whose delivery varied with baseline anxiety and depression, with the overall ratio of clinical to non-clinical content increasing according to severity without survey information access. These findings support the feasibility of purpose-built AI for mental health.

Thomas D. Hull, Lizhe Zhang, Caitlin A. Stamatis +2

Slingshot AI · CREATIV Lab, Department of Psychiatry and Behavioral Sciences, University of Washington · Department of Psychiatry, NYU School of Medicine +1

Healthcare Mental Health Mental Health Conversational Agents Conversational Agents Generative AI Evaluation

Ask PDF

Sep 1, 2026

Interpretable Symptom Vectors for Depression in a Large Language Model

Patients with depression present with diverse symptom profiles, yet clinical practice routinely reduces this variation to a single severity score. Large language models (LLMs) can potentially capture various symptoms and their severity from patient speech. However, how depressive symptoms are represented inside LLMs remains poorly understood, limiting clinical trust. To examine whether internal model activations match clinician judgment, we analyzed the residual stream of Gemma-3-27B-PT using mechanistic interpretability techniques. Recording activations across symptom descriptions drawn from validated clinical instruments, we found that symptom groups geometrically separated the most at layer 21 across multiple distance metrics. Using Semantic Projection, we then projected held-out naturalistic text onto Symptom Vectors constructed from these instruments. The resulting per-symptom coefficients preserved clinician-annotated rank ordering across mood, somatic, and suicidality axes. Furthermore, a single depression vector in Layer 21 separates held-out depressive from non-depressive text (AUC = 0.789), which can be used as an emotional valence gate that restricts symptom projection to depressive speech. These results reveal a decorrelated, clinician-aligned symptom signal readable directly from internal activations, offering a mechanistic foundation for interpretable depression-assessment tools.

Fangyi Zhu, Ajay Subramanian, Allison Constant +3

Department of Psychiatry & Behavioral Sciences, Stanford University School of Medicine, Stanford, CA, USA · Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA · Kaiser Permanente, San Jose, CA, USA +2

Depression Detection Representation Geometry in Language Models Emotion Representation in Language Models Clinical NLP

Ask PDF

Abstract

Generative Modeling

Explore similar work

Expresso-AI: Explainable Video-Based Deep Learning Models for Depression Diagnosis

Generative AI Purpose-built for Social and Mental Health: A Real-World Pilot

Interpretable Symptom Vectors for Depression in a Large Language Model