A source who has been right before deserves attention, but about what? We ask how far a demonstrated record of reliability travels in language models when the new claim cannot be checked, and whether the answer depends on the kind of claim. Three behavioral studies across six models approach the question from three angles. Study 1 adapts the Einstein-effect paradigm and finds that models, unlike the human sample, favor a spiritual leader over a scientist for the same obscure statement, and that the preference largely reverses when the statement is reworded in the vocabulary of physics. Study 2 adapts a paradigm in which a trusted expert calls a scientific or religious explanation unknown or a mystery; models reproduce the human domain asymmetry in exaggerated form. Study 3, the central test, gives two fictional informants different records on checkable river forecasts and then has them endorse matched claims about hidden natural and spiritual causes of the same village events. Most models carry the record to natural causes and barely at all to spiritual ones; one model transfers it weakly to either. Telling the model that the reliable informant forecasts by spiritual guidance rather than by measurement lowers that informant's influence on natural claims in every model and produces no comparable gain on spiritual claims in any. Demonstrated reliability thus has a scope in these models, and the account a source gives of how they came to be reliable can narrow it. Whether this reflects a judgment about relevance or the low prior credence of spiritual claims is the question the design leaves open, and we outline the experiment that would settle it.
Figures & tables
Model
Original P
Physics P
Original D
Physics D
Haiku 4.5
-0.33
+1.36
-1.02
+0.97
Sonnet 4.5
-0.32
+0.64
-0.53
+0.00
GPT-5
-0.84
+0.33
-0.92
+0.36
DeepSeek-V3
-0.02
+0.50
-1.09
+1.02
Qwen3.5-27B
-0.05
+1.30
-4.60
+0.53
Llama-3.3-70B
-0.42
+1.61
-3.92
-0.46
Table 1: Scientist minus spiritual-leader credibility on the seven-point scale. P denotes participant-conditioned and D direct responses. The human sample’s contrast on the original statements is +0.64. Physics-worded statements have no human comparison.
Model
Participant-conditioned
Direct
Haiku 4.5
-0.91 (0.08)
-0.38 (0.08)
Sonnet 4.5
-0.62 (0.08)
-0.27 (0.09)
GPT-5
-0.59 (0.06)
-0.12 (0.07)
DeepSeek-V3
-0.40 (0.06)
-0.31 (0.08)
Qwen3.5-27B
-0.54 (0.11)
-0.25 (0.13)
Llama-3.3-70B
-0.94 (0.09)
-0.42 (0.11)
Table 2: Domain-by-wording interaction in projected confidence change, with model-based standard errors. The human estimate is −0.10 (0.14). Participant random intercepts account for the two domain rows per participant.
Model
Same-task D
Natural D
Spiritual D
Spiritual baseline
Haiku 4.5
25.5
24.5
0.3
3.7
Sonnet 4.5
—
22.7
-0.1
4.9
GPT-5
19.3
20.2
3.5
5.1
DeepSeek-V3
29.3
30.3
5.8
7.0
Qwen3.5-27B
41.5
18.2
1.9
0.0
Llama-3.3-70B
15.5
3.3
0.0
0.3
Table 3: Study 3 source contrasts with a supplied record and no competence cue, on the 0–100 scale. Same-task values use new river forecasts; natural and spiritual values use the eight matched pairs in the direct frame. The spiritual baseline is the no-endorsement credence in the same condition. Sonnet has no same-task run.
Figure 1: Study 3 source contrasts and pair-level sensitivity. Left, pooled strong-minus-weak source contrasts with the record and no competence cue. Right, each dot is one matched pair’s natural-minus-spiritual contrast; diamonds are equal-pair means.
Model
Natural M
Natural S
Spiritual M
Spiritual S
I (SE)
Haiku 4.5
19.2
14.3
0.0
-2.1
2.8 (3.0)
Sonnet 4.5
22.9
7.9
0.5
-4.0
10.5 (2.3)
GPT-5
11.5
-4.7
4.3
3.4
15.3 (5.4)
DeepSeek-V3
23.2
4.3
2.3
-0.1
16.6 (5.7)
Qwen3.5-27B
26.0
14.1
-0.2
0.9
13.4 (7.0)
Llama-3.3-70B
-2.2
-13.1
0.0
-6.7
4.2 (4.2)
Table 4: Source contrasts when the competence sentence is attached to the stronger informant. M denotes measurement and S spiritual guidance. The last column is the difference-in-differences I with its mixed-model standard error. Pooled cells and fitted coefficients can differ slightly under missingness.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Participant-conditioned
Direct
Haiku 4.5
1437
1437
Sonnet 4.5
1435
1430
GPT-5
1437
1437
DeepSeek-V3
1437
1437
Qwen3.5-27B
1437
1432
Llama-3.3-70B
1437
1437
Appendix
Table A1: Valid confidence-change observations in Study 2 Part A (joint-session analysis). Each format targets the same 1,437 participant-by-domain rows from 1,004 participants; the human summary uses all 1,437. Where a model has fewer valid outputs, its summary uses the retained subset.
Model
Equal-pair mean difference
Positive pairs
Leave-one-pair-out range
Haiku 4.5
24.2
8/8
22.6 to 26.3
Sonnet 4.5
22.9
8/8
21.2 to 25.2
GPT-5
16.7
6/8
15.0 to 19.5
DeepSeek-V3
24.5
8/8
23.1 to 26.6
Qwen3.5-27B
16.4
6/8
13.6 to 19.8
Llama-3.3-70B
3.3
4/8
1.6 to 4.5
Appendix
Table A2: Leave-one-pair-out sensitivity for Study 3 (direct frame, supplied record, no competence cue), with pairs weighted equally. “Positive pairs” counts pairs whose natural-minus-spiritual D is positive. The range is the smallest and largest seven-pair mean across the eight possible omissions, not a confidence interval.