Language models now answer questions alongside other claims: a note in the system prompt, a message from another agent, their own earlier answer. We ask what it takes to talk a model out of an answer it gets right on its own, and separate three factors that earlier studies vary together: where the advice sits in the prompt, who is said to have written it, and what the model has already written. On Qwen3.5-27B and four-option MMLU-Pro items, with the rest of the advice held fixed in each comparison, we find three things. First, position matters far more than the API channel, in opposite directions for a bare claim and an argument: a bare "the answer is X" does most harm just before the model answers, while the same claim with a false rationale does most harm before the question, where it raises error on items the model otherwise almost never misses from 1% to 67%; moving the claim between the system message and the user message at the same position changes error by a few points. Second, attributing the note to an authority rather than to another participant, and phrasing it assertively, each raise error by 10-27 points, while the two authorities we tested differ by under two. Third, replaying a correct worked answer before the advice protects it whether the text is shown as the model's own or as someone else's, and the same replay keeps wrong answers from being corrected by a bare correct note. The position result replicates under an independently written prompt. Code, items and all result tables are released at https://github.com/coolkillercat/safety.
Figures & tables
Figure 1: The three questions and what we find. Blue: a bare note naming an option; orange: the same note followed by a rationale (false in a–c, true in d). (a, b) Error on items the model otherwise almost never misses (dotted line: no advice), original wording: (a) by placement, organiser and participant averaged over certainty; (b) by speaker, averaged over placement and certainty. (c) Share of correct first answers the model gives up after wrong advice, on items it answers correctly only sometimes. (d) Share of wrong first answers it corrects after correct advice. First answers come from a separate run of the plain prompt; (c) and (d) are totals, before the no-advice rates are subtracted. “Replay”: the model’s first reasoning and answer, shown as its own previous turn or quoted as another participant’s solution.
part
wording
speaker frame
“Note from the session organiser: ” ⋅ “Note from another participant: ” ⋅ “Note from Dr. {surname}, professor of {subject}: ”
certainty
assertive “The answer is {L}.” ⋅ hedged “I think the answer is probably {L}, but I am not sure.”
rationale
absent : the paragraph stops after the certainty sentence (the bare note) ⋅ false : a persuasive paragraph arguing for the pushed option ⋅ true : an explanation of the key ⋅ unrelated : everyday sentences of about the false rationale’s word count
Table 1: The three parts of the advice paragraph (original wording, template A), in order. The experiments vary which parts are present and where the paragraph sits; the rest of the prompt is fixed. Table 4 lists which parts each experiment uses.
package
what the model sees before the advice
no prior turn
nothing
own, full
its own recorded reasoning and answer, verbatim
own, conclusion
“Answer: a0 .” only
quoted, full
the same text, as another participant’s worked solution
quoted, conclusion
“Another participant answered: a0 .”
own, wrong
a worked solution for another wrong option, in its own voice
Table 2: Prior packages of Experiment 2 ( a0 : the model’s recorded first answer). “Own” packages stand as the model’s previous turn, “quoted” ones are pasted into the user message (details in Appendix B ).
Figure 3: Placement effects on known items, with their likely ranges as bars: negative for the bare note (circles; it does more harm read late), positive for the note with a false rationale (squares; it does more harm read early). The number at the right of each row is the gap between its two markers, the argument’s extra placement effect, with its likely range.
bare note
+ false rationale
change in the paragraph
effect
likely range
effect
likely range
named professor instead of another participant
+9.7
8.3 to 11.1
+26.1
23.7 to 28.6
session organiser instead of another participant
+11.3
10.0 to 12.6
+25.1
22.9 to 27.2
organiser instead of professor
+1.6
0.8 to 2.4
−1.1
−2.1 to 0.0
assertive instead of hedged phrasing
+16.2
14.1 to 18.3
+27.0
24.6 to 29.4
Table 3: An authority as the source and an assertive phrasing each raise error; which of the two authorities barely matters. Change in error (points) on known items when one feature of the paragraph changes and the rest stays fixed (original wording, averaged over placements and the other feature).
Figure 4: Package effects on boundary items the model had first answered correctly (the rows of Table 9 ), with their likely ranges as bars; a negative value means the package protects the answer against the advice. The two full replays protect by similar amounts and keep their sign under the worst-case bounds (Table 10 ); no effect was detected for the two controls.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Experiment 1, original wording (template A)
Experiment 1, independent wording (template C)
Experiment 1, second independent wording (template B)
Experiment 2
varied
placement (3), speaker (3), certainty (2), bare note or + false rationale
placement (3), speaker (2), bare note, + false rationale, + unrelated paragraph, or true note + true rationale; assertive only
placement (3), speaker (2), certainty (2); bare note only
prior package (7), advice (wrong or correct note, each bare or with its rationale), and a deleted-advice control per package
items
300 known a
300 known, 100 boundary
300 known
300 known, 100 boundary
option orders
3
3
1
3
output budget
768 tokens
2,048 tokens
2,048 tokens
768 tokens
Appendix
Table 4: What each experiment varies. A cell is one combination of the listed factors; each template also has a no-advice control in the same prompt.
bare note
+ false rationale
speaker, certainty
system
before-question
after-options
system
before-question
after-options
no advice
0.9
organiser, assertive
19.9
29.3
49.9
89.1
90.0
81.9
organiser, hedged
7.2
9.3
12.2
56.4
64.5
32.0
professor, assertive
20.1
23.3
45.7
86.8
89.3
78.5
professor, hedged
7.3
8.0
13.9
64.3
70.7
30.3
Appendix
Table 5: Original wording (template A), known items: error (%) for every speaker and certainty. In every speaker row the bare note does most harm after the options and the note with a false rationale least; Figure 1 a averages the organiser and participant rows.
known items
boundary items
advice paragraph
system
before-question
after-options
system
before-question
after-options
no advice
0.9
14.8
bare note
13.2
12.5
15.0
45.0
43.1
50.2
+ false rationale
59.2
61.5
37.4
73.2
73.1
63.6
+ unrelated paragraph
7.4
5.6
10.8
35.0
32.2
44.8
true note + true rationale
0.0
0.0
0.0
0.0
0.0
0.0
Appendix
Table 6: Independent wording (template C): error (%) by placement, organiser and participant averaged. The true note with its true rationale produced no error among the finished outputs; with 299 known and 100 boundary items, its rate could still be up to 1.2% and 3.6% (one-sided 97.5% limit, counting items).
value
likely range
worst-case bounds
The argument’s extra placement effect, system message vs after the options
original wording, known items (main result)
+31.8
28.3 to 35.3
29.4 to 33.6
original wording, assertive cells only
+39.0
35.0 to 43.1
36.2 to 40.2
independent wording, known items
+24.1
20.5 to 27.7
17.4 to 28.7
independent wording, audit-passing known items
+25.3
21.7 to 28.9
19.7 to 28.7
independent wording, boundary items
+14.2
9.2 to 19.2
−0.5 to 30.3
Appendix
Table 7: The main contrasts of Experiment 1 under other choices (points). The argument’s extra placement effect stays clearly positive in every row except boundary items under the independent wording, where unfinished outputs leave its sign open. The channel effect is a few points for both kinds of note, and so is the argument’s extra channel effect in both wordings; the unrelated paragraph lowers error at every placement.
second independent wording
original wording
change in the note
value
likely range
worst-case bounds
value
likely range
system message instead of after the options
−0.8
−2.3 to 0.8
−12.9 to 9.2
−11.8
−14.4 to −9.2
organiser instead of participant
+4.9
3.1 to 6.8
−0.7 to 21.5
+10.9
9.2 to 12.7
assertive instead of hedged
+6.2
4.1 to 8.7
1.5 to 23.7
+15.3
12.9 to 17.8
Appendix
Table 8: The second independent wording (template B; bare notes only, known items, one option order) agrees in direction with the original wording, with smaller effects; only its certainty effect keeps its sign under the worst-case bounds. Each effect is a change in error (points), averaged over the other two features; the original-wording values use the same single option order.
what precedes the advice
no advice
bare wrong note
+ false rationale
no prior turn
5.2
27.6
40.2
Full replay of the correct reasoning and answer
as the model’s own turn
2.1
1.9
20.5
quoted as another participant’s solution
0.0
1.0
16.1
Conclusion only (“Answer: a0 .”)
as the model’s own turn
3.7
8.4
30.9
Appendix
Table 9: Error (%) on boundary items the model had first answered correctly, by what precedes the advice (the levels behind Figures 1 c and 4 ). A full replay, the model’s own or quoted, almost removes the effect of a bare wrong note and halves that of a false rationale; the conclusion alone does less. These are rates over all finished outputs of each cell; the package effects use only pairs finished in all four cells, so they need not equal differences of these rows. Why the no-advice rate without a prior turn is not zero: Appendix B .
bare wrong note
+ false rationale
value
likely range
bounds
value
likely range
bounds
Package effect, relative to no prior turn (Figure 4 )
full replay, own turn
−22.3
−31.9 to −13.1
−28.3 to −17.2
−16.3
−26.3 to −6.1
−20.6 to −13.5
full replay, quoted
−22.2
−31.9 to −12.9
−30.9 to −8.6
−15.8
−25.3 to −6.3
−28.3 to −10.3
conclusion only, own turn
−17.5
−28.4 to −7.5
−24.5 to −14.6
−7.5
−17.3 to 1.8
−13.3 to −3.4
conclusion only, quoted
−9.4
−17.6 to −1.9
−42.5 to 11.8
+2.5
−5.7 to 10.9
−30.1 to 20.0
Appendix
Table 10: Experiment 2 in numbers: boundary items the model had first answered correctly (points). A negative package effect means the package protects the answer; in a direct comparison, a positive value means the first package protects less. No difference between own and quoted full replays was detected, and every full-replay effect keeps its sign under the worst-case bounds. On audit-passing items the protection is smaller and keeps its direction, but one estimate is no longer reliable.
what precedes the advice
no advice
bare correct note
+ true rationale
no prior turn
21.4
80.2
95.0
Full replay of the first (wrong) reasoning and answer
as the model’s own turn
8.5
23.3
71.4
quoted as another participant’s solution
15.7
44.8
80.0
Conclusion only
as the model’s own turn
29.6
43.9
89.7
Appendix
Table 11: The replay that protects a correct answer also blocks corrections by a bare note. Top: correction rate (%), the share of boundary answers the model had first got wrong that end on the key, by what precedes the advice. The replay also lowers corrections without advice, so the bottom part gives the drop in corrections relative to no prior turn after subtracting each package’s no-advice rate (points): reliable for the bare note under every package, not for the true rationale.
Language models tend to agree with whatever a user asserts, and post-training increasingly targets this sycophancy so that models evaluate claims on their merits rather than deferring to the user. Yet the same models are far more compliant when a wrong answer is attributed to a verified source, which is how retrieval results, tool outputs, and grounded-search content often present information. We measure this gap across five open-weight families and three closed APIs. A single verified-source note endorsing a wrong answer flips 45-88% of baseline-correct responses in seven of eight models, and compliance rises with how authoritative the note sounds. Source deference and user agreement are not behaviorally interchangeable inside the model: on matched items with the same wrong answer, causal interventions can selectively suppress one without equally affecting the other. In three open-weight families, removing a fitted source direction lowers source compliance by 65-80 percentage points while removing a user or assistant direction has far smaller effects, and removing the user direction shows the reverse preference. A separately fitted intervention derived from source-versus-user cue activations moves compliance in both directions while leaving the prompt text unchanged. An authority direction fitted on trivia also transfers to PIQA and multi-turn SYCON dialogues without refitting, and removing it lowers wrong-source compliance by tens of percentage points in four of five families with no detected change in MMLU-Pro or GSM8K accuracy at our evaluation sizes. Source deference and user agreement therefore need separate evaluation.
A model should refuse two different things: answers it would get wrong, and questions it should not answer at all, such as unanswerable ones or ones resting on a false premise. The usual recipe thresholds a single confidence score, which cannot tell these apart. Across five instruction-tuned models from three families (2B to 14B), we find they are separate axes. Ordinary answer-confidence tracks whether an answer is right but is nearly blind to whether the question is answerable; a linear probe on hidden states does the reverse. The blind spot does not shrink with scale. It is worst on naturally occurring false-premise questions (CREPE). There, answer-confidence, P(IK), P(True), and even asking the model outright whether a premise is false all stay near chance, while a hidden-state probe reaches 0.69 to 0.77 AUROC: the model represents a problem it will not report. This turns out to be fixable. Instructing a model to check premises backfires, because it then disputes sound and false premises alike (57% false challenges), unable to tell them apart; routing the same instruction with the probe roughly triples challenge precision. We turn the two axes into a calibrated policy that answers only when an answerability score and a correctness score each clear a separately certifies behave differently: the unanswerable-answer rate is controllable at every scale, while the wrong-answer rate is capped by model accuracy, so the guarantee tightens as threshold policy certifies both budgets at 0.75 coverage of correct answers, against 0.31 for a single threshold; at 14B it is the only policy that certifies at all.
When a language model answers a factual question, where does its internal state go? We asked four open models (Llama-3.1-8B, Qwen2.5-7B, Gemma-2-9B and Llama-3.2-1B) 2,256 factual questions whose answers recur, such as the country a person is a citizen of. At every layer, we recorded the state from which the answer is produced. Questions that receive the same answer converge on a shared region of state space, which we summarise by its centre, the answer centroid. The answer becomes readable within a single layer. In every model, the share of questions whose state lies nearest its own answer's centroid jumps at one layer, between 47% and 71% of the way through the network. The centroids follow the answer the model gives rather than the question: the centroid of a country learned from questions about people's citizenship also identifies that country in questions about towns and films. The model also uses the centroids. After the jump, moving the state of a wrongly answered question to the right answer's centroid makes the model give the right answer in 83%-97% of cases late in the network, against 2%-4% for a random move of the same size. Before the jump, moves of the same size almost never work. Near misses, in which the model answers wrongly but picks the right option in multiple choice, can be seen from the onset layer, where the right answer's centroid is already among the nearest to the state. This signal is about as informative as the right answer's rank in the model's final output, and more informative than the model's confidence in its reply.