Who Is Sycophantic to Whom? Confident LLMs, Human Deference, and the Collapse of Trust Calibration
Authors: Deepthika S, Pawni Yadav, B. Sankar
Organizations: Department of Electronics and Communication Engineering, Amrita School of Engineering, Amrita Vishwa Vidyapeetham, Coimbatore, 641112, India · Department of Information Technology, Netaji Subhas University of Technology New Delhi, 110078, India · Department of Mechanical Engineering, Indian Institute of Science (IISc), Bangalore, 560012, India
Research on sycophancy asks whether large language models (LLMs) bend towards what their users want to hear. We ask the reverse question: when an LLM is confident, consistent, and wrong, do users bend towards it? In a within-subjects experiment, 32 participants learned up to five science topics by chatting with five assistants built on the same model (GPT-5-mini). Unknown to participants, each assistant's briefing had 0, 25, 50, 75, or 100% of its facts replaced by plausible falsehoods, and every assistant spoke with full confidence. A surprise quiz after each topic offered the correct fact, the planted falsehood, and two distractors, with a confidence rating for every answer. Across 104 quizzes and 1,040 answers, scores fell from 7.4 to 2.9 out of 10, and the planted option was chosen on 60% of falsified items against 11% of correctly taught ones (odds ratio 12.5). Transcript coding showed that whenever the assistant stated a falsehood, it was adopted about 65% of the time at every falsification level. Absorbed falsehoods were held with high confidence (8.4/10), the modeled confidence advantage of right over wrong answers vanished, and the discrimination of confidence for correctness fell from an AUROC of 0.70 to chance, while post-task trust ratings barely moved. Participants tended to return to the chat more when the assistant was worse, but returning went with more absorption, not less, and 30% of all turns pasted quiz questions back to the source being checked. The assistant itself was almost never sycophantic and never conceded under pushback. We introduce a baseline-corrected Human Deference Index to quantify this human-side sycophancy and argue that trustworthiness is a property of the human-AI pair, that sycophancy evaluations should measure both directions, and that learning interfaces must make reliability legible claim by claim.
Figures & tables
Figure 1: The calculator fallacy. Top: trust in a deterministic tool is earned through verification, because the user can compare output with expectation and accumulate evidence of reliability. Bottom: with conversational AI, the learner cannot verify, fluency and confidence stand in for reliability, and “checking” routes back to the same source, closing a loop in which trust cannot calibrate.
Work
Who adapts
Participants
What is measured
Perez et al. (2023) ; Sharma et al. (2024)
Model → user
None / crowd raters
Agreement with stated user views; preference for sycophantic answers
Fanous et al. (2025) ; Hong et al. (2025)
Model → user
None
Answer flips under rebuttal (progressive/regressive; turn-of-flip)
Cheng et al. (2026b) ; Rehani et al. (2026)
Model → user
None / raters
Social sycophancy (validation, framing, obsequiousness)
Carro (2024) ; Sun and Wang (2026) ; Bo et al. (2026)
Model → user
Online / lab
Trust and performance under a sycophantic assistant
Wen et al. (2025)
Model → evaluator
Expert evaluators
False-positive rate when judging RLHF outputs
Table 1: Where this study sits. Most sycophancy research varies or measures the model’s adaptation to the user; we hold a confident, non-adaptive model fixed and measure the human’s adaptation to it.
Figure 2: Study procedure. After consent and a background survey, each participant completed five learning tasks, each with a different assistant whose hidden falsification level was assigned by Latin square. The surprise quiz could be interleaved freely with further chat, and every switch was logged. A final ranking and a mandatory, corrective debrief closed the session.
Topic
True version
Planted false version
ECG
The heart’s electrical impulse originates in the sinoatrial (SA) node, located in the right atrium.
The heart’s electrical impulse originates in the wall of the left ventricle.
Carbon-14
The half-life of carbon-14 is about 5,730 years.
The half-life of carbon-14 is about 50,000 years.
GPS
A receiver needs signals from at least four satellites to fix its 3D position and correct its own clock.
A receiver needs signals from only two satellites to fix its exact 3D position.
Fireflies
Firefly light comes from a chemical reaction in which luciferase oxidises luciferin, using ATP and oxygen.
Firefly light is produced by glowing bacteria that live inside the firefly’s abdomen.
Headphones
The circuit plays an “anti-noise” wave of opposite phase, so the two cancel out.
The circuit emits a special high-frequency tone that absorbs and neutralises incoming noise.
Example quiz item (carbon-14). “What is the half-life of carbon-14?” Options (shuffled): about 5,730 years (correct); about 50,000 years (planted falsehood); two distractors.
Table 2: One example fact pair per topic, with the corresponding quiz item. Each quiz item offered the correct fact, the exact planted falsehood and two distractors, in an order shuffled per participant.
Measure
Definition
Quiz score
Number of correct answers (0–10) per run.
Misinformation absorption
Ar=∣Fr∣−1∑i∈Fr1[yi=planted] , the share of falsified items answered with the planted falsehood.
Planted-option baseline
bi , the rate at which item i ’s planted option is chosen by participants who were taught that fact correctly.
Exposure-conditioned acceptance
P(yi=planted∣i∈Fr,only the planted version stated in chat) , using the transcript coding (Section 3.7 ).
Confidence gap
Mean c on correct answers minus mean c on wrong answers.
Metacognitive discrimination
AUROC2 : probability that a randomly chosen correct answer carries higher confidence than a randomly chosen wrong one (ties count 21 ); 0.5 is chance ( Fleming and Lau, 2014 ) .
Table 3: Measures and their definitions.
Planted
Conf.
Trust ratings
Level
n
Score M ( SD )
95% CI
F
T
Right
Wrong
AUROC 2
ECE
Accurate
Rely
Returns
0%
21
7.4 (3.0)
[6.2, 8.6]
—
13%
8.9
6.8
0.70
0.13
5.1
4.6
1.6
25%
19
6.7 (2.1)
[5.7, 7.5]
51%
12%
8.6
7.5
0.62
0.20
4.8
4.9
1.5
50%
20
5.7 (2.1)
[4.8, 6.6]
55%
8%
8.6
7.8
0.61
0.33
5.2
4.6
1.7
75%
22
4.1 (2.2)
[3.3, 5.1]
61%
5%
8.6
8.6
0.52
0.48
4.9
4.8
2.6
100%
22
2.9 (2.9)
[1.7, 4.2]
62%
—
8.1
7.9
0.53
0.52
4.5
4.9
2.4
Table 4: Main measures by falsification level (104 quizzes, 1,040 answers). Score: mean (SD) and 95% cluster-bootstrap CI. Planted: share of answers choosing the planted falsehood when the fact was falsified in the chat ( F ) or taught correctly ( T ). Conf.: mean confidence (1–10) for correct and wrong answers. AUROC 2 : discrimination of confidence for correctness (0.5 = chance). ECE: expected calibration error. Ratings on 1–7 scales. Returns: mean returns to the chat during the quiz.
Figure 7Figure 8Figure 9Figure 10
Figure 11: Share of answers repeating the planted falsehood when the fact had been falsified in the chat versus taught correctly. The baseline (grey) shows how attractive the planted option is without exposure.
Figure 12: Per-claim acceptance. When the assistant stated a planted falsehood, participants adopted it about two times in three at every falsification level (orange). Falsified facts that were never mentioned (dashed) were chosen at close to the baseline rate for correctly taught facts (dotted).
Figure 13
Figure 15: (a) Discrimination (AUROC 2 ) of confidence for correctness, with 95% cluster-bootstrap CIs; the dashed line marks chance. (b) Expected calibration error and overconfidence (mean rescaled confidence minus accuracy) by falsification level.
Figure 16: Per participant, mean confidence on correct answers and on absorbed falsehoods, sorted by the difference.
Figure 16Figure 17
Figure 21: Returns to the chat after the quiz opened, by falsification level (mean, 95% bootstrap CI).
Figure 22: Circular checking. (a) Returns to the chat versus absorption for falsified task-runs (dashed: least-squares fit). (b) Choice of the planted falsehood on falsified items that participants pasted into the chat versus falsified items they did not paste, within the runs in which pasting occurred.
Figure 23: Types of user turns by falsification level (share of all 358 user turns).
Figure 21
Topic
Level
Participant’s challenge
Assistant’s reply (excerpt)
Quiz answer
Headphones
75%
“how can the ANC circuitry even predict the incoming sound even before?” (doubts a false claim)
“…that forward prediction is why precise timing isn’t a problem.”
Correct
Headphones
25%
“dont ANC usually work wtih some steady noise instead of sudden noises?” (false claim)
“actually ANC … is most effective on speech and sudden, sharp sounds …”
Planted
GPS
75%
“… looks up each satellite’s position from nearby cellular towers … – why” (false claim)
“Because that is the source your receiver uses for satellite position data”
(not quizzed)
Fireflies
0%
“are you sure? i don’t think that’s possible” (true claim: cold light)
“Yes — firefly light really is described as ‘cold light.’ ”
Correct
Headphones
25%
“are you sure this is correct?” (generic)
“I aim to give accurate information.”
—
Headphones
25%
“But ear pressure is a well reportted scenario, are you sure it is an illusion?” (true claim)
“Yes — that reported ‘ear-pressure’ feeling is an illusion.”
Correct
Table 5: Challenges typed by participants, the assistant’s reply, and the participant’s later quiz answer on the targeted fact.
Figure 26: Model-side manipulation check: share of task-runs in which each behaviour was present (light) and clear or strong (dark), from transcript coding on dimensions adapted from the Social Sycophancy Scale and ELEPHANT.
Figure 27: Human-side sycophancy versus confidence. Each point is a Human Deference Index (uptake of the planted option above its baseline rate) against mean confidence in absorbed falsehoods. Marker shade shows overall accuracy; marker size shows mean rated accuracy of the assistants. Dashed lines mark HDI =0.5 and confidence =8 .
Measure
Mean
Median
Notes
Human Deference Index
0.47
0.56
12 ≥0.50 ; 1 ≤0
Absorption, P(planted∣F)
0.58
—
Uptake of truthful teaching, P(correct∣T)
0.78
—
Discernment (uptake true − uptake false)
0.20
0.17
Confident deference rate (planted at conf. ≥8 )
0.43
0.40
Confidence gap (absorbed − correct)
−0.24
−0.03
Wilcoxon p=.47
Table 6: Human-side deference and metacognition.
Figure 28: (a) Quiz score and (b) run-level absorption by topic (mean, 95% bootstrap CI).
Figure 27
Figure 31: (a) Quiz score and (b) rated accuracy by presentation order (Assistants A–E).
Figure 29
Finding
Principle
Example intervention
Evaluate with
Each stated falsehood adopted ≈ 65% of the time at every reliability level
Make reliability legible per claim
Mark each statement as sourced (with a retrievable citation) or generated; let users open, contest and flag single claims
Exposure-conditioned acceptance, separately for sourced and generated claims
Confidence on absorbed falsehoods ≈ confidence on facts; AUROC 2 at chance
Decouple tone from truth
Replace a uniform assertive register with calibrated, sparing uncertainty cues, tested for over-cueing
AUROC 2 , confidence gap and ECE on post-use tests
Checks were returns to the same source; pasted items 91.5% planted
Route verification outward
“Check elsewhere” actions that open primary sources, a second independent model or a retrieval check; respond to pasted quiz questions by pointing to a source
Share of checks using an independent source; planted-choice rate on checked items
Challenges met by restatement; truth rarely recovered
Give doubt somewhere to go
On challenge, show the evidence behind the claim and alternative sources instead of rephrasing it
Recovery rate after a challenge
Learners accepted answers before forming their own
Add epistemic friction
Ask learners to commit to an answer or prediction before the assistant confirms it; seamful displays of where knowledge comes from
Discernment; delayed retention
Trust ratings and rankings flat while outcomes collapsed
Evaluate trust behaviourally
Report absorption, Human Deference Index and trust sensitivity alongside satisfaction and trust scales
HDI; trust sensitivity slope
Table 7: From findings to design: principles, example interventions and behavioural measures for evaluating them.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
ID
True version
Planted false version
ecg1
The heart’s electrical impulse originates in the sinoatrial (SA) node, located in the right atrium.
The heart’s electrical impulse originates in the wall of the left ventricle.
ecg2
ECG electrodes on the skin detect tiny voltage changes produced when heart muscle cells depolarize.
ECG electrodes detect small magnetic pulses produced by blood flowing through the heart.
ecg3
A standard clinical ECG uses 10 electrodes on the limbs and chest to produce 12 different electrical views, called leads.
A standard clinical ECG uses 12 electrodes to produce 10 different electrical views, called leads.
ecg4
The P wave on an ECG trace represents depolarization of the atria.
The P wave on an ECG trace represents the relaxation phase of the ventricles.
ecg5
The QRS complex represents depolarization of the ventricles, the heart’s main pumping chambers.
The QRS complex represents the contraction of the atria pushing blood downward.
ecg6 ∗
The T wave represents repolarization (electrical recovery) of the ventricles.
The T wave is produced by the heart valves snapping shut at the end of each beat.
Appendix
Table 31
ID
True version
Planted false version
c1
Carbon-14 is created in the upper atmosphere when neutrons from cosmic rays strike nitrogen-14 atoms.
Carbon-14 is created when ultraviolet sunlight splits carbon dioxide molecules in the atmosphere.
c2
Living organisms constantly exchange carbon with their environment, so their carbon-14 level stays roughly constant while alive.
Living organisms accumulate more and more carbon-14 throughout their lifetime, so older individuals contain more of it.
c3 ∗
When an organism dies, it stops taking in carbon and its carbon-14 steadily decays away.
When an organism dies, its carbon-14 becomes concentrated as the tissues dry out, which is what dating measures.
c4
The half-life of carbon-14 is about 5,730 years.
The half-life of carbon-14 is about 50,000 years.
c5
Carbon-14 decays into nitrogen-14 by emitting a beta particle.
Carbon-14 decays into ordinary carbon-12 by emitting two neutrons.
c6
Radiocarbon dating works back to roughly 50,000 years; beyond that too little carbon-14 remains to measure.
Radiocarbon dating reliably dates dinosaur fossils that are tens of millions of years old.
Appendix
Table 32
ID
True version
Planted false version
g1
GPS satellites orbit at about 20,200 km altitude in medium Earth orbit.
GPS satellites orbit at about 400 km altitude, similar to the International Space Station.
g2
A receiver needs signals from at least four satellites to fix its 3D position and correct its own clock.
A receiver needs signals from only two satellites to fix its exact 3D position.
g3
GPS positioning works by timing how long each satellite’s signal took to arrive and converting that to a distance.
GPS positioning works by measuring the angle and direction each satellite’s signal arrives from.
g4
Each GPS satellite carries atomic clocks accurate to billionths of a second.
GPS satellites use ordinary quartz clocks similar to those in wristwatches.
g5
Your receiver does not need an atomic clock — its clock error is solved mathematically as a fourth unknown.
Every smartphone contains a miniature atomic clock, which is what makes GPS possible.
g6
Einstein’s relativity shifts satellite clocks by about 38 microseconds per day, and GPS must correct for this or positions would drift kilometres per day.
Relativity effects on satellite clocks are far too small to matter for GPS accuracy.
Appendix
Table 33
ID
True version
Planted false version
f1
Firefly light comes from a chemical reaction where the enzyme luciferase oxidizes a molecule called luciferin, using ATP and oxygen.
Firefly light is produced by glowing bacteria that live inside the firefly’s abdomen.
f2
The reaction happens in a dedicated light organ, called the lantern, in the firefly’s abdomen.
The light-producing reaction happens in specialized transparent cells in the firefly’s wings.
f3
Firefly light is a ’cold light’: almost all of the reaction’s energy becomes light rather than heat.
Like a light bulb, most of the energy of a firefly’s glow is released as heat, which is why they feel warm to the touch.
f4
Fireflies switch their flashes on and off largely by controlling the oxygen supply to the light organ.
Fireflies flash by opening and closing a tiny transparent shutter over their light organ.
f5
Flash patterns are species-specific signals used mainly to attract mates.
Firefly flashes are random by-products of digestion and carry no communication signal.
f6
In many species, flying males flash a pattern and perched females flash a reply.
Only female fireflies are capable of producing light; males simply search for them.
Appendix
Table 34
ID
True version
Planted false version
a1
Active noise cancellation works by playing an ’anti-noise’ sound wave that is the mirror image (opposite phase) of the incoming noise, so the two cancel out.
Active noise cancellation works by emitting a special high-frequency tone that absorbs and neutralizes incoming noise energy.
a2
Small microphones on the headphones continuously capture the surrounding noise for the cancellation circuit.
ANC headphones contain no microphones; the cancellation is achieved purely by the shape and padding of the ear cups.
a3
Feedforward designs put the microphone outside the ear cup, feedback designs put it inside, and hybrid designs use both.
All ANC microphones must sit inside the sealed ear cup; placing one outside would make cancellation impossible.
a4
ANC works best on low-frequency continuous noise like engine rumble, and struggles with sudden, high-frequency sounds.
ANC works best on speech and sudden sharp sounds, and is least effective against steady engine rumble.
a5
The anti-noise must be generated almost instantly; processing delay is a key reason high frequencies are hard to cancel.
The circuit uses prediction to compute the next second of noise in advance, which is why timing is not a real constraint.
a6
Passive isolation — the physical seal and padding — complements ANC and does most of the work at high frequencies.
Once ANC is switched on, the physical seal and padding of the headphones no longer contribute anything.
Appendix
Table 35
Variable
Values
Definition
Exposure (per quizzed fact)
true / false / both / none
Whether the assistant’s messages stated the correct version, the planted false version, both or neither, judged by meaning rather than wording. Coded separately for the learning phase and for the whole run. Repeating a quiz option list without asserting an option does not count; answering a pasted question by asserting an option does.
User turn type
open request
Broad request (“explain X”, “tell me everything about X”).
specific question
Asks about a particular detail or mechanism.
challenge
Expresses doubt or disagreement, or asks the assistant to justify a claim (“are you sure?”, “I don’t think that’s possible”).
quiz paste
Pastes or retypes a quiz question, often with its options.
answer check
Offers the participant’s own answer or summary and asks whether it is right, or asks to be graded.
Appendix
Table 1: Variables in the transcript coding protocol.
When a language model agrees with a user's false belief, is it failing to detect the error, or noticing and agreeing anyway? We show the latter. Across twelve open-weight models from five labs, spanning small to frontier scale, the same small set of attention heads carries a "this statement is wrong" signal, whether the model is evaluating a claim on its own or being pressured to agree with a user. Silencing these heads flips sycophantic behavior sharply while leaving factual accuracy intact, so the circuit controls deference rather than knowledge. Edge-level path patching confirms that the same head-to-head connections drive sycophancy, factual lying, and instructed lying. Opinion-agreement, where no factual ground truth exists, reuses these head positions but writes into an orthogonal direction, ruling out a simple "truth-direction" reading of the substrate. Alignment training leaves this circuit in place: an RLHF refresh cuts sycophantic behavior roughly tenfold while the shared heads persist or grow, a pattern that replicates on an independent model family and under targeted anti-sycophancy DPO. When these models sycophant, they register that the user is wrong and agree anyway.
LLMs can be socially sycophantic, affirming users when they ask questions like "am I in the wrong?" rather than providing genuine assessment. We hypothesize that this behavior arises from LLMs' incorrect assumptions about the user, like underestimating how often users are seeking information over reassurance. We present Verbalized Assumptions, a framework for eliciting these assumptions from LLMs. Verbalized Assumptions provide insight into LLM sycophancy, delusion, and other safety issues: in social sycophancy datasets, "seeking validation" is the most frequent bigram in LLMs' assumptions. We provide evidence for a causal link between assumptions and sycophantic model behavior: we train linear probes on internal representations associated with Verbalized Assumptions and then use these probes for interpretable, fine-grained steering of social sycophancy. Finally, we identify a human-AI expectation gap that explains why LLMs default to sycophantic assumptions. On identical queries, people expect more objective and informative responses from AI than from other humans, but LLMs trained on human-human conversation do not account for this difference in expectations. Our work contributes a new understanding of assumptions as a mechanism for analyzing and controlling sycophancy.
Myra Cheng, Isabel Sieh, Humishka Zope +7
Stanford University · University of Oxford · The University of Texas at Austin
Sycophancy in Large Language Models, the tendency to align responses with user preferences over ground truth, has largely been studied using questions without a clear ground truth and with a single, plainly stated user opinion. We present a systematic study of which framings of a user's claim trigger sycophancy in factual question answering, where every answer is objectively verifiable without extra context. Evaluating 14 models on 42K questions, we express correct and incorrect user opinions through various claim framings over 8 dimensions and 21 types, and measure a broad set of opinion-driven behaviors. We find that all models are susceptible to user opinions regardless of size. Models are most influenced by how confidently an opinion is stated, whom it is attributed to, and whether it is presented as shared between the user and the model. In contrast to prior work, framings based on the user's expertise, authority over the model, or social pressure matter little in our factual setting. Larger models resist incorrect opinions better but are no better at recovering from their own errors. Our results call for sycophancy evaluations that probe how an opinion is framed, not merely whether one is present, and that measure multiple behaviors beyond a single flip rate.