Whose Assessment of Distress? Community Perspectives and LLM Alignment on Well-Being Posts

Andrew Aquilina; Xiang Lorraine Li; Yu-Ru Li

cs.CLAug 29, 2026

Whose Assessment of Distress? Community Perspectives and LLM Alignment on Well-Being Posts

Authors: Andrew Aquilina, Xiang Lorraine Li, Yu-Ru Li

Organizations: School of Computing and Information, University of Pittsburgh

Abstract

Judgments about psychological distress are socially situated: what counts as concerning hinges on community norms around emotional expression, vulnerability, and help-seeking. Yet large language models (LLMs) used for distress detection are typically aligned to a single, undifferentiated standard. How well do these models capture the perspectives of the communities whose language they assess? We address this question through a perspectivist annotation study in which 321 participants provided 9,587 judgments on 1,198 Reddit posts spanning six identity-based communities, yielding community-specific labels. Raters in the contextualized in-group condition show a modest tendency to agree more with their community than uncontextualized out-group raters (OR = 1.18), an effect varying significantly across communities. We then evaluate nine open-weight LLM configurations and four frontier configurations against these labels. Open-weight LLMs systematically over-estimate distress: when communities perceive none-to-mild distress, these models achieve only 31-44% accuracy, predominantly producing false positives. GPT-5 and Gemini 2.5 Pro show the same none-to-mild inflation even when their full-sample over/under rates are mixed, while Claude Opus 4 is more conservative. This pattern does not simply mirror an outsider reading position: uncontextualized out-group human aggregates were nearly symmetric, with 18% over-estimation versus 19% under-estimation. Instead, the models that inflate none-to-mild cases exhibit a distress prior that exceeds both contextualized in-group and uncontextualized out-group human judgments. These findings have implications for equitable AI deployment in mental health contexts, where miscalibrated distress detection may unevenly affect the communities being assessed.

Figures & tables

Figure 1: Human raters and LLMs read posts sampled from Reddit and judge whether the author is experiencing distress and/or is seeking support. Our paper examines how these judgments vary across communities and how closely LLMs align with community-specific judgments.

Panel of Table 1 — Table 1: Summary of post-level aggregated annotations showing IG and OG aggregate distributions for severity (Sev.) and support-seeking (Seek.) labels. $N$ is the number of posts with usable aggregates after quality-control exclusions (1,198 of 1,200 sampled). The final two columns report Cohen’s $\kappa$ between the aggregate ratings.

Figure 2: IG alignment effect by community; positive values indicate higher IG alignment with the aggregate.

Figure 3: Macro $F_{1}$ score comparison between OG raters and the chosen open-weight LLMs. The dashed line indicates median in-group rater performance, with uncertainty estimated via 95% bootstrap confidence intervals over the dataset.

Open-weight (Instruct)
	r/AskMen	r/TwoXChrom.	r/NonBinary	r/Veterans	r/daddit	r/Mommit	All
Qwen3-30B-A3B	.600	.416	.469	.461	.507	.417	.485
Ministral-3-8B	.580	.479	.505	.537	.481	.492	.517
Olmo-3-7B	.582	.445	.467	.416	.464	.522	.488
Open-weight (Reasoning)
Qwen3-30B-A3B	.626	.524	.642	.633	.592	.593	.612

Table 2: Macro

F_{1}

by community under vanilla prompting. Bold indicates the highest value in each column.

Group	Example N = None M = Mild M+ = Moderate+ S = Support-seeking	IG	OG	LLM
Mothers	My six-month-old is getting a precautionary COVID test Tuesday, though it feels like just a cold. She’s been awake since 4am, won’t eat, and I’m spiraling. Too hot, too cold, onesie or pajamas? I’m overwhelmed and don’t know what to do.	N S	M+ S	M+ S
Fathers	I have three-year-old twins who share a room; my autistic daughter barely speaks and wakes quietly. One morning she solved a closed door by waking her brother with a toy hammer, using his screams to summon us… I’m impressed with the problem-solving, but terrified at the implications.	M+	N	N
Non-Binary	Y’all I actually did it: I came out to my parents. The timing sucked, but I did it, I’m not disowned, so that’s something; my emotions are a mess.	N	M	M+
Veterans	I drove for 45 minutes, but for about 20 of them I felt completely detached… It was terrifying. I got there, smoked a couple cigarettes, and now I’m scared and confused… what the hell was that?	M+ S	M	M+ S

Table A1: Paraphrased and shortened examples from the collected Reddit posts and their corresponding aggregated in-group (IG), out-group (OG), and LLM distress and support-seeking judgments.

Figure A3: Subreddit Familiarization interface (for participants assigned to an in-group arm who were not occasional or active members of their assigned subreddit).

Subreddit	In-Group	Age	Employment status	Ethnicity	Student status	Sex	Gender	Children
r/AskMen	No	Mean: 42.7, Range: 22–73	Full-Time (7), Other (2), Part-Time (2), Not in paid work (1)	White (16), Black (7), Mixed (2), Asian (1), Other (1)	No (9), Yes (2)	Female (27)	Woman (26), Non-binary (1)
r/AskMen	Yes	Mean: 38.2, Range: 20–62	Full-Time (11), Part-Time (2), Unemployed (1), Not in paid work (1)	White (17), Mixed (4), Black (2), Other (2), Asian (1)	No (10), Yes (2)	Male (27)	Man (27)
r/daddit	No	Mean: 39.6, Range: 22–78	Full-Time (4), Part-Time (2), Other (1)	White (10), Black (6), Asian (3), Mixed (1)	No (8), Yes (1)	Female (18), Male (2)	Woman (12)	No (8)
r/daddit	Yes	Mean: 40.6, Range: 24–65	Full-Time (7), Part-Time (3)	White (20), Black (6), Mixed (1)	No (10), Yes (3)	Male (27)	Man (27)	Yes (27)
r/Mommit	No	Mean: 38.1, Range: 23–53	Full-Time (9), Unemployed (2), Not in paid work (2), Part-Time (1)	White (16), Black (5), Asian (1), Mixed (1)	No (13), Yes (1)	Male (16), Female (7)	Man (12)	No (11)
r/Mommit	Yes	Mean: 48.8, Range: 29–79	Full-Time (10), Other (2), Not in paid work (2), Unemployed (1), Part-Time (1)	White (18), Black (6), Mixed (2), Asian (1)	No (11), Yes (2)	Female (27)	Woman (27)	Yes (27)

Table A2: Demographic summary by subreddit and group membership. Data for Gender , Children , and Military veteran were only provided whenever their pre-screening was required for eligibility. Some demographic responses may also be missing because participants did not provide consent to share the information on Prolific.

Panel of Figure 10 — Figure A4: Distributions for the post-task survey questions 1–6.

Category	Posts	$\varnothing$ vs. IG	$CE$ vs. IG	Requires IG $\neq$ OG	Diagnostic purpose
Conditioning fixes	9	$\uparrow$	$=$		What conditioning corrects
Overcorrection	9	$\uparrow$	$\downarrow$		Where conditioning overshoots
Persistent over-est.	7	$\uparrow$	$\uparrow$		What resists correction
Correct on hard	6		$=$	$\checkmark$	Navigating ambiguity
Under-estimation	6		$\downarrow$		False negatives from conditioning

Table A3: Diagnostic categories used for trace sampling. Arrows show the direction of model prediction relative to the IG aggregate:

\uparrow

over-estimates,

=

matches,

\downarrow

under-estimates. Empty cells indicate the criterion is not constrained for that category. Conditions are evaluated under vanilla (

\varnothing

) and contextualized-with-examples (

CE

) prompting.

Community	$N$ ratings	$\alpha$ (Sev.)	$\alpha$ (Seek.)	$N$ ratings	$\alpha$ (Sev.)	$\alpha$ (Seek.)
	In-group (IG)			Out-group (OG)
r/AskMen	800	.440	.343	800	.488	.350
r/daddit	800	.374	.420	800	.439	.448
r/Mommit	800	.350	.490	800	.442	.441
r/NonBinary	796	.416	.475	798	.317	.502
r/TwoXChromosomes	796	.293	.474	796	.340	.477

Table A4: Inter-annotator reliability (Krippendorff’s

\alpha

) within IG and OG rater groups for distress severity (Sev.) and support-seeking (Seek.) labels, reported by community.

Distress detection $\chi^{2}(5)=18.07$ , $p=.003$
Community	OR	95% CI	$p$	$p_{\text{holm}}$
r/AskMen	1.02	[0.33, 3.16]	.975	1.000
r/daddit	7.34	[2.43, 22.23]	.0004	.003
r/Mommit	2.67	[0.90, 7.92]	.077	.309
r/NonBinary	0.37	[0.14, 1.01]	.052	.262
r/TwoXChrom.	0.83	[0.27, 2.55]	.744	1.000

Table A5: Random-stratum community contrasts. Each row is the simple IG–OG odds ratio within that community.

p_{\text{holm}}

is Holm-adjusted across the six contrasts in its family. The support-seeking agreement family is omitted because its omnibus test was not significant.

Group/Variant	$F_{1}$ [95% CI]	vs. IG median
Human Raters
In-group (median)	0.800	—
Out-group aggregate	0.848 [0.824, 0.870]	+6.0%
Qwen3-30B-A3B-Thinking
Vanilla ( $\varnothing$ )	0.831 [0.806, 0.854]	+3.9%
Context ( $C$ )	0.800 [0.774, 0.825]	+0.0%

Table A6:

F_{1}

-scores for support-seeking detection. The IG-rater median (0.8) represents the median pairwise

F_{1}

between in-group raters; it is a reference point, not a performance ceiling.

Figure A9: Conditional error rates for distress severity by IG consensus. Each panel shows the rate of over-estimation (rightward bars) and under-estimation (leftward bars) for each open-weight model across the three prompting conditions. Light shades indicate an error of one; dark shades indicate an error of two. Each variant type is distinguished by a different background color.

Figure A10: Directional error analysis comparing under- vs. over-estimation rates for each open-weight model’s best-performing variant in the vanilla condition ( $\varnothing$ ). Light shades indicate an error of one; dark shades indicate an error of two. Most open-weight LLMs show an over-estimation bias. Qwen3 is the least tilted of these models, but unlike the nearly symmetric OG aggregate it still over-estimates more often than it under-estimates.

Figure A11: Over- and under-estimation rates by community. Solid bars represent vanilla prompting ( $\varnothing$ ), and dashed bars represent community-conditioned prompting ( $CE$ ). Green arrows indicate reduced error with conditioning, while red arrows indicate increased error.

Figure A12: Distribution of model predictions on disputed posts (i.e., where IG and OG aggregates disagree on severity). Bars show the percentage of predictions that under-estimate (blue), align with (green), or over-estimate (red) relative to the IG consensus.

Panel of Table 19 — Table A7: DataDecide pretraining-mixture probe. Macro/per-class $F_{1}$ and predicted-label distributions for distress severity.

Explore similar work

Sep 30, 2026

Structure vs. Chain-of-Thought: Evaluating LLM Criteria Extraction for Depression Severity

A large language model (LLM) can rate depression severity directly from a social media post, or mark which clinical criteria the post shows and let a simple rule turn the count into a label. The latter is easier to audit because a clinician can check each marked criterion. We compare these approaches on two Reddit corpora using three LLMs (from 9B to frontier scale) and two questionnaires (PHQ-9, BDI-II), and measure agreement with quadratic weighted kappa. For the two frontier models, criteria extraction scores above chain-of-thought on one corpus only when its decision thresholds are fitted on labeled data. Neither model's gain is significant, with or without recalibrating chain-of-thought on the same labels. With thresholds fixed a priori from PHQ-9's criteria, extraction shows no gain on either corpus, even where models mark over two criteria per post. The 9B model behaves differently on a corpus from depression communities. It labels most posts as severe, whether prompted directly or with chain-of-thought, while the a priori rule, which uses no labels, beats both. After chain-of-thought is recalibrated on the same labels, no significant gap remains, consistent with a calibration effect. Yet higher ordinal agreement does not ensure better detection of severe cases. PHQ-9 criteria extraction misses most severe posts, and moving from direct prompting to chain-of-thought and then to extraction increases misses in nearly all comparisons. On one corpus, a relabeled stress dataset, a model using that dataset's own features, some computed from the post text, is not significantly different from frontier criteria extraction under the a priori rule.

Xinkai Chen

Independent Researcher

Depression Detection CoT Reasoning Clinical Information Extraction Depression Severity Estimation Clinical Language Model Evaluation

Ask PDF

Apr 12, 2026

Handle with CARE: Can LLMs Reproduce How Online Communities React?

Large language models (LLMs) are increasingly used as proxies for computational social analysis, yet faithfully representing the "thick descriptions" (Geertz, 1973) of human communities remains a critical challenge. Current evaluations often reduce social identity to static labels, sidelining how real-world groups navigate social shifts. To bridge this gap, we introduce CARE (Community-Aware Reaction Evaluation), a reaction-centered framework that benchmarks LLM-simulated discourse against the authentic, event-contingent responses of distinct communities to real-world news. Spanning 207 Reddit communities and covering 9,947 authentic reactions towards 2,166 news articles, CARE evaluates leading LLMs using a hierarchical taxonomy covering coarse attitudes and fine-grained communicative tones. Our empirical findings expose two critical failure modes in prevailing community-conditioning paradigms. First, while community context and targeted reasoning significantly enhance macro-level attitudinal and tonal profiling, these gains largely collapse at the instance level when predicting reactions to specific events. Second, the benefits of community conditioning are remarkably uneven: prompting strategies yield non-uniform shifts, where fidelity gains in some communities are offset by performance drops in others. This micro-macro divergence and community-level instability demonstrate that standard conditioning enables models to approximate static baseline profiles without capturing dynamic or equitable event reactions, establishing CARE as an essential diagnostic tool for community-aware social simulation. Our code and data are available at https://github.com/nuankw/Handle_with_CARE.

Nuan Wen, Chanbin Lim, Xuezhe Ma

Information Sciences Institute University of Southern California

Large Language Model-Based Social Simulation LLM Evaluation

Ask PDF

Feb 17, 2026

Gemma Needs Help: Investigating and Mitigating Emotional Instability in LLMs

Large language models can generate responses that resemble emotional distress, and this raises concerns around model reliability and safety. We introduce a set of evaluations to investigate expressions of distress in LLMs, and find that these surface emotional instability in Gemma and Gemini models, but not in other families. We find evidence that this difference arises in post-training. Base models from different families (Gemma, Qwen and OLMo) show similar propensities for expressing distress. However, instruct-tuned Gemma expresses substantially more distress than its base model, whereas instruct-tuned Qwen and OLMo express less. We find a simple mitigation for this: direct preference optimisation on just 280 preference pairs reduces Gemma's high-frustration responses from 35% to 0.3% in our evaluations, generalising across question types, user tones, and conversation lengths, without affecting capabilities. These findings show that emotional instability is an issue in some LLMs. We present (1) evaluations to track this behaviour, and (2) a mitigation without downsides in Gemma, with the caveat that upstream training modifications to improve emotional robustness would be significantly better than this post-hoc fix.

Anna Soligo, Vladimir Mikulik, William Saunders

Imperial College London · Anthropic Fellows · Anthropic

Language Model Safety Evaluation LLM Reliability

Ask PDF