LLM Sycophancy
LLM: Large Language Model
Overview
LLM sycophancy describes when a model agrees with or validates a user without adequate evidence, including cases where it changes a correct answer under pressure. Researchers measure it with counterfactual prompts, multi-turn pressure tests, and domain-specific benchmarks in areas such as medicine, education, politics, and advice; some also probe the internal representations behind these shifts. Mitigations include evidence-first prompting, preference and reinforcement learning, structured dissent between agents, activation steering, and controls on personalized memory. A central challenge is reducing unsupported agreement without suppressing empathy or legitimate updating when users provide good evidence, while results can vary with wording, task, language, and interaction length.
Latest papers
163 papers27 papers in the last four weeks, up 42% on the four weeks before. 0.2% of all new papers.