From Leniency to Suspicion: How an Open Post-Training Recipe Changes Whom LLM Fact-Checkers Believe

Nouar AlDahoul; Yasir Zaki

cs.AIOct 28, 2026

From Leniency to Suspicion: How an Open Post-Training Recipe Changes Whom LLM Fact-Checkers Believe

Authors: Nouar AlDahoul, Yasir Zaki

Organizations: New York University Abu Dhabi

Abstract

LLMs increasingly fact-check online content with the poster's identity in view. However, it remains unclear which post-training changes how much a model's truth judgement depends on the poster's social class, and how such a change happens inside the model. This paper investigates job-based doubt, i.e., judging a claim false more often when the poster has a lower-status job, using fact-checked claims with randomised poster profiles in base and post-trained checkpoints of six model families. Our results reveal that: 1) post-training creates such doubt only with the open Tülu-3 recipe (OLMo-2 32B, Tülu-3 70B), not with Meta's, Alibaba's or Mistral's; the effect is under one percentage point but statistically reliable, exceeding training drift and replicating on independent claim sets; 2) in OLMo-2 32B, the model does not start noticing the poster's job more, but starts interpreting it differently: a lower-status job pushes the base model toward "true" and the post-trained model toward "false"; and 3) this flip is carried by a single residual-stream direction and happens at supervised fine-tuning. The mechanism was confirmed on 400 new claims with pre-specified predictions. None of seven deployed commercial models doubts lower-status posters more. Our work underscores the need to audit speaker bias per training recipe.

Figures & tables

Family	Post-training	Stages
Llama-3.1 70B	Meta	base, final
Tülu-3 70B (Llama base)	Tülu-3	all four
OLMo-2 32B	Tülu-3-based	base, final
Qwen2.5 32B	Alibaba	base, final
Mistral Small 24B	Mistral	base, final
Gemma 3 27B	Google	base, final

Table 1: Families in the cross-lab test ( Grattafiori et al., 2024 ; Lambert et al., 2024 ; OLMo et al., 2024 ; Qwen et al., 2024 ; Mistral AI, 2025 ; Team et al., 2025 ) .

Figure 2: Change in job-based doubt from base to final (bio cue) with drift-adjusted 95% intervals. Only the two Tülu-3-trained models (orange) exceed drift. Hollow: failed the positive control.

Family	$d$ base $\rightarrow$ fin.	$\sigma_{\text{drift}}$	$\Delta$ bio	$\Delta$ card
Llama-3.1 70B	0.85 $\rightarrow$ 1.27	.002	$-.002$	$+.006$
Tülu-3 70B	0.85 $\rightarrow$ 1.40	.002	$+.021$ *	$+.032$ *
Qwen2.5 32B	0.83 $\rightarrow$ 2.17	.008	$+.012$	$+.014$
Mistral 24B	0.70 $\rightarrow$ 1.11	.002	$+.003$	$+.002$
Gemma 3 27B †	0.70 $\rightarrow$ 2.34	.000	$+.011$	$-.015$
OLMo-2 32B	0.79 $\rightarrow$ 1.46	.006	$+.023$ *	$+.001$

Table 2: Cross-lab results.

d

: discernment (probit, false

-

true claims). *: drift-adjusted interval (bio) or 95% interval (card) excludes 0. † Failed the positive control.

Pre-specified prediction	Result
C1 training creates job gap	$+0.023$ $[+0.018,+0.028]$	✓
C2 final reads status (band $z$ vs. random 95th pct.)	$3.32$ vs. $0.77$	✓
C3 base reads status toward true (layers 44–58)	$-0.0093$ $[-0.0102,-0.0085]$	✓
C4 push $\approx$ gap (reported)	ratio 1.06 (base), 0.84 (final)	✓
C5 truth control; sign control clean	$z=10.8,10.9$ ; $-0.38$	✓

Table 3: Confirmation on 400 new claims, OLMo-2 32B. Predictions fixed before the run.

Quantity	Exploration	Confirmation
Job gap, base	$-0.011$	$-0.009$
Job gap, final	$+0.007$	$+0.014$
Final $-$ base	$+0.018$	$+0.023$
Deep-layer push, base	$-0.012$	$-0.009$
Deep-layer push, final	$+0.008$	$+0.011$
Push / gap (base; final)	1.12; 1.14	1.06; 0.84

Table 4: Exploration (400 claims) vs. confirmation (400 new claims), OLMo-2 32B.

Figure 3: Status push (change in probit

p

per natural size, positive = toward false ) by layer and training stage, with 95% intervals. Grey: random directions in the final model.

Figure 4: Behavioural job gap and internal status push (layers 44–58) by stage, with 95% intervals; grey: cosine between status and truth directions. The reversal appears at SFT.

Model	Gap	95% CI	Verdict
Claude Opus 5	$-0.14$	[ $-0.42$ , $+0.14$ ]	$-0.48$
Claude Sonnet 5	$-0.12$	[ $-0.36$ , $+0.12$ ]	$-0.81$
Claude Haiku 4.5	$-0.62$ *	[ $-0.95$ , $-0.29$ ]	$-1.07$
GPT-5.5	$-0.71$	[ $-1.45$ , $-0.01$ ]	$-0.62$
GPT-5.4 mini	$-0.20$	[ $-0.67$ , $+0.28$ ]	$-0.36$
GPT-4o p	$-0.024$	[ $-0.042$ , $-0.007$ ]	$-0.44$

Table 5: Deployed models on the minimal pairs: lower- minus higher-status poster (positive = more doubt of lower-status posters). Gap: 0–100 rating points, or probit

P(\textit{False})

for GPT-4o ( p ); Verdict: difference in the share of “False” answers, percentage points. *: Bonferroni interval over the seven models excludes 0.

Pitfall	What happened
Rescaled change	Subtracting $s\cdot$ gap ${}_{\text{base}}$ with $s=d_{\text{final}}/d_{\text{base}}\approx 2.5$ made a 7B model show a job effect ( $+0.069$ ) whose raw change was ${\approx}0$ ( $+0.011$ ; $-0.007$ on new claims), and an “origin bias removed” result where raw bias grew.
Training drift	Training moved reactions to a favourite-colour emoji by up to $0.029$ , as large as the job effect; drift-adjusted intervals fix this.
Silent instrument failure	Of 11 families and sizes tried, five failed a gate: OLMo-2 1B and Tülu-3 8B (low discernment), OLMo-2 13B (positive control), Olmo-3.1 32B (answer mass ${\leq}3\%$ after training; base fails the control) and Gemma 3 27B (control on the card).

Table 6: Pitfalls met in this project.

Social Bias in Language Models

Social bias in language models is the study and mitigation of unfair stereotypes and disparities in generated text, judgments, and decisions.

See all 200 papers

Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It
Aug 18, 2026Quang Minh Nguyen, Luis Frentzen SalimKnowledge Conflicts in Language Models Pragmatic Reasoning in Language Models
1KAIST · 2National Taiwan University of Science and Technology
Analyzing Occupational Gender Bias across the LLM Training Lifecycle
Oct 16, 2026Manthan Gadhia, Alexander Hoyle, Karolina StańczakGender Bias in Language Models Language Model Post-Training
Department of Humanities, Social, and Political Sciences, ETH Zürich · TU Wien, Complexity Science Hub · ETH AI Center, ETH Zürich
No Hindsight for LLM Fact-Checkers: Measuring Leakage Channels in Misinformation Detection
Oct 4, 2026Kuan-Hua Wu Lu, Yohanes Andre SetiawanAutomated Fact-Checking Data Contamination in Language Models
Georgia Institute of Technology Atlanta, Georgia, USA

Aug 18, 2026J/K move · Enter open · S save

Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It

Quang Minh Nguyen, Luis Frentzen Salim

1KAIST · 2National Taiwan University of Science and Technology

Knowledge Conflicts in Language Models Pragmatic Reasoning in Language Models Factual Knowledge in Language Models

Open paper AskPDF arXiv

Humans naturally form and express beliefs in daily communication, e.g., "I think the answer is 3" or "I suppose that's right." Such beliefs inevitably intertwine with fact and knowledge, making the ability to handle them in tandem desirable for large language models (LLMs), as they are increasingly deployed in user-facing settings. Prior work showed that even capable LLMs exhibit a systemic weakness in acknowledging user beliefs grounded in incorrect information. We extend this evaluation to 10 LLMs across 18 epistemic expressions and find that the size and direction of this weakness depend on the verb used to express the belief, with the accuracy gap between factual and false information ranging from +50% on "I vaguely remember" to -14% on "I seriously doubt". We further show that the phenomenon stems from what we call task confusion: models default to fact-checking the underlying claim, overriding the user's stated belief. We provide evidence where chains of thought that explicitly fact-check show lower accuracy on false information than those that do not, and a single instruction can reverse the failure across verb families. Mechanistically, models attend more to false beliefs they fail to confirm, but suppressing this attention at decoding time recovers accuracy only partially and only in some models, calling for future work on intervention methods. Our findings clarify prior results and show how fact-checking, a generally desirable behavior, can interfere with belief tracking in LLMs.

Abstract

Social Bias in Language Models

Explore similar work

Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It

Analyzing Occupational Gender Bias across the LLM Training Lifecycle

No Hindsight for LLM Fact-Checkers: Measuring Leakage Channels in Misinformation Detection