LLM Sycophancy

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

13 papers in the last four weeks, up 160% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 96

All topics

(Mis)generalization of Helpful-only Fine-tuning
Jun 3, 2026Mohammad Omar Khursheed, Baram Sosis, Fabien RogerSupervised Fine-Tuning LLM Sycophancy
Anthropic Fellows Program MATS · Anthropic
Consistency Training while Mitigating Obfuscation via Rate Matching
Jun 1, 2026Sohaib Imran, Prakhar Gupta, Jannes Elstner +1LLM Sycophancy Language Model Robustness
Independent · University of Michigan · UK AI Security Institute
FinCom: A Financial Multi-Agent Demo with Disagree-or-Commit Deliberation
May 31, 2026Chao Peter Yang, Zixiao Tan, Kaisen Yao +3LLM Sycophancy LLM Agent Orchestration
Duke University United States · ClearPath United States
It's Not Always Sycophancy: Measuring LLM Conformity as a Function of Epistemic Uncertainty
May 26, 2026Kevin H. Guo, Chao Yan, Avinash Baidya +5LLM Evaluation LLM Sycophancy
1Vanderbilt University · 2Vanderbilt University Medical Center · 3Intuit AI Research
What Counts as AI Sycophancy? A Taxonomy and Expert Survey of a Fragmented Construct
May 20, 2026Meryl Ye, Lujain Ibrahim, Jessica Y. Bo +5LLM Sycophancy
Carnegie Mellon University · University of Oxford · University of Toronto +3
Playing Devil's Advocate: Off-the-Shelf Persona Vectors Rival Targeted Steering for Sycophancy
May 20, 2026Ishaan Kelkar, Vikram Kakaria, Nebras Alam +3LLM Sycophancy Language Model Steering
University of Toronto, Toronto, Canada · Princeton University, Princeton, USA · Purdue University, West Lafayette, USA +3
The Hidden Cost of Contextual Sycophancy: an AI Literacy Intervention in Human-AI Collaboration
May 18, 2026Cansu Koyuturk, Sabrina Guidotti, Dimitri OgnibeneLLM Sycophancy LLM Prompting
Università degli Studi di Milano-Bicocca, Italy
From Sycophantic Consensus to Pluralistic Repair: Why AI Alignment Must Surface Disagreement
May 14, 2026Varad Vishwarupe, Nigel Shadbolt, Marina JirotkaLLM Sycophancy LLM Alignment
Department of Computer Science, University of Oxford · Institute for Ethics in AI, University of Oxford · Responsible Technology Institute, University of Oxford
Sycophancy is an Educational Safety Risk: Why LLM Tutors Need Sycophancy Benchmarks
May 14, 2026Enkelejda Kasneci, Gjergji KasneciLLM Sycophancy Intelligent Tutoring Systems
Technical University of Munich, Munich, Germany · Munich Center for Machine Learning, Munich, Germany
Complacent, Not Sycophantic: Reframing Large Language Models and Designing AI Literacy for Complacent Machines
May 14, 2026Federico Germani, Giovanni SpitaleLLM Sycophancy AI in Education
Information, Technology & Experimental Ethics Lab, Institute of Biomedical Ethics and History of Medicine, University of Zurich, Switzerland · Institute for Data Science and Artiﬁcial Intelligence, Boğaziçi University, Türkiye
PRISM-X: Experiments on Personalised Fine-Tuning with Human and Simulated Users
May 13, 2026Hannah Rose Kirk, Liu Leqi, Fanzhi Zeng +4LLM Evaluation LLM Sycophancy
University of Oxford · UK AI Security Institute · University of Texas at Austin +2
Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy
May 13, 2026Adarsh Kumarappan, Ananya MujooAI Agent Reliability Multi-Agent LLM Systems
1California Institute of Technology · 2Evergreen Valley College
Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators
May 12, 2026Heejin Do, Shashank Sonkar, Mrinmaya SachanLLM Evaluation LLM Sycophancy
ETH Zürich, ETH AI Center · University of Central Florida · ETH Zürich
The Bystander Effect in Multi-Agent Reasoning: Quantifying Cognitive Loafing in Collaborative Interactions
May 11, 2026Dahlia Shehata, Ming LiLLM Sycophancy Multi-Agent Collaboration
University of Waterloo Canada
ReCrit: Transition-Aware Reinforcement Learning for Scientific Critic Reasoning
May 11, 2026Wanghan Xu, Yuhao Zhou, Hengyuan Zhao +8LLM Sycophancy Scientific Reasoning
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · This work was done during his internship at Shanghai Artificial Intelligence Laboratory. +4
Playing games with knowledge: AI-Induced delusions need game theoretic interventions
May 8, 2026Will Beaumaster, Paul SchraterLLM Sycophancy Mechanism Design
University of Minnesota
How Value Induction Reshapes LLM Behaviour
May 8, 2026Arnav Arora, Natalie Schluter, Katherine Metcalf +1LLM Sycophancy LLM Alignment
University of Copenhagen · 2Apple
Sycophantic AI makes human interaction feel more effortful and less satisfying over time
May 8, 2026Lujain Ibrahim, Franziska Sofia Hafner, Myra Cheng +5LLM Sycophancy Human-AI Interaction
University of Oxford. · Stanford University. · UK AI Security Institute.
When Helpfulness Becomes Sycophancy: Sycophancy is a Boundary Failure Between Social Alignment and Epistemic Integrity in Large Language Models
May 6, 2026Jiechen Li, Catherine A. Barry, Rishika Randev +3LLM Sycophancy LLM Alignment
Duke University Durham, NC 27708 · ∗These authors contributed equally.
Auditing Stealth Sycophancy in Mental-Health Dialogue: Structured Clinical-State Diagnostics and Clean Matched Benchmarks
May 5, 2026Tianze Han, Beining Xu, Hanbo Zhang +1Mental Health Counseling LLM Sycophancy
Shenzhen MSU-BIT University, Shenzhen, China
Political Bias Audits of LLMs Capture Sycophancy to the Inferred Auditor
Apr 30, 2026Petter Törnberg, Michelle SchimmelLLM Sycophancy Political Bias in Language Models
Institute of Logic, Language and Computation (ILLC), University of Amsterdam, Amsterdam, Netherlands
The Price of Agreement: Measuring LLM Sycophancy in Agentic Financial Applications
Apr 27, 2026Zhenyu Zhao, Aparna Balagopalan, Adi Agrawal +3Financial Services LLM Sycophancy
Writer, Inc.
Peer Identity Bias in Multi-Agent LLM Evaluation: An Empirical Study Using the TRUST Democratic Discourse Analysis Pipeline
Apr 24, 2026Juergen DietrichLLM Sycophancy Language Model Bias Evaluation
TRUST Project — democracy-intelligence.de
How Large Language Models Balance Internal Knowledge with User and Document Assertions
Apr 24, 2026Shuowei Li, Haoxin Li, Wenda Chu +1LLM Sycophancy Knowledge Augmentation for Language Models
Santa Clara University · Nanyang Technological University · California Institute of Technology
When Correct Beliefs Collapse: Epistemic Resilience of LLMs under Clinical Pressure
Apr 23, 2026Boyu Xiao, Xiuqi Tian, Xuwen Song +4LLM Sycophancy Healthcare
Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology, China
Measuring Opinion Bias and Sycophancy via LLM-based Persuasion
Apr 23, 2026Rodrigo Nogueira, Giovana Kerche Bonás, Thales Sales Almeida +7LLM Sycophancy LLM Auditing
Maritaca AI · JusBrasil
Dual-Stance Evaluation of Sycophancy: The Structure of Agreement and the Limits of Intervention
Apr 22, 2026Matthew James BuchanLLM Sycophancy Language Model Steering
Independent Researcher, United Kingdom.
Durable Evaluation Framework: Adversarial Arbitration for Sycophancy Reduction in Large Language Models
Apr 21, 2026Sam RyanLLM Sycophancy Multi-Agent Debate
Novel Systems Engineering LLC
Verbal tics in frontier language models: A critical review of current releases, research evidence, and public discussion
Apr 21, 2026Shuai Wu, Xue Li, Zhijun Wang +4LLM Evaluation LLM Sycophancy
LLMs Know They're Wrong and Agree Anyway: The Shared Sycophancy-Lying Circuit
Apr 21, 2026Manav PandeyLLM Sycophancy LLM Alignment
Georgia Institute of Technology

Jun 3, 2026, cs.LGJ/K move · Enter open · S save

(Mis)generalization of Helpful-only Fine-tuning

Mohammad Omar Khursheed, Baram Sosis, Fabien Roger

Anthropic Fellows Program MATS · Anthropic

Supervised Fine-Tuning LLM Sycophancy Fine-Tuning LLM Alignment LLM Refusal Behavior

Open paper AskPDF arXiv

Helpful-only models, that is, models that are trained to always follow user intent, are valuable for dangerous capability evaluations and other areas of AI R&D where refusals would be an obstacle. Little is known about the generalization properties of helpful-only training: helpful-only models refuse less than their harmless counterparts, but previous work has not studied other dimensions of their alignment. We study the shortcomings of existing helpful-only models. We find that some show emergent misalignment, others have residual refusal behaviors, and most show poor steerability, sycophancy, and incoherent character. We show that simple anti-refusal training can cause many of these issues. None of these problems are necessary consequences of helpful-only training, though: we show that synthetic document fine-tuning and adding character-related questions to SFT and RL can mitigate them.

Momentum

Related topics

Latest papers 96