cs.CLSep 23, 2026

Persuaded, Not Informed: Incentive-Misaligned Witnesses Defeat In-Context Grounding

Authors: Rahul Balakavi

Organizations: AmpUp Research

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Language-model agents increasingly answer questions over customer-relationship management (CRM) records, such as whether to qualify a sales lead. We identify a failure mode not addressed by a stronger model: when the context contains an assertion by a party with an incentive toward optimism - here the sales representative, a witness recorded in the CRM - the model treats the assertion as evidence and clears deals the company's own records deem unacceptable. Across 100 lead-qualification tasks from CRMArena-Pro, the representative asserts an acceptable timeline in every call and an acceptable budget in 76; on the 31 tasks where such an assertion contradicts the price list and installation policy, a model reading only the transcript clears the deal in 29 of 31 cases. The signature is consistent across seven models from four providers (misled on 87-97%); scale and explicit reasoning confer no resistance. Only 3 of 35 genuine failures involve no assertion: the failure is persuasion, not missing information. We contribute a diagnostic method rather than an architecture: (i) a bucket analysis that separates persuasion from information gaps, (ii) a same-information control showing that supplying the records to the model lowers strict accuracy from 41 to 18 while raising recall - precision collapses - and (iii) a compute-step control that holds extraction fixed and varies only who computes Budget and Timeline. The margin ranges from 42 points on an inexpensive model to 2-5 points on models that already compute correctly; on the strongest models the arms are within confidence intervals, so the pattern is a consistent direction and a soundness property, not a proved performance floor. We pre-specify a generalization test that returns a negative result, characterize the precondition (a policy exactly specified in the inputs), and release all evaluation artifacts.

Figures & tables

DimensionClaims acceptableGenuinely fails… and claimed acceptableModel misled
Timeline100/100121212/12
Budget76/100231917/19
Contradiction cases (pooled)—353129/31 (94%)
TABLE I: What the representative claims versus what the records say, across 100 lead-qualification calls. The claim that the deal is acceptable is asserted almost always; on the deals where it is false, the model accepts it.
ModelProviderMisled % [95% CI]Caught % [95% CI]
Current-generation
GPT-5OpenAI90.3 [75–97]80.6 [64–91]
gpt-5.6-solOpenAI90.3 [75–97]90.3 [75–97]
Claude Sonnet 5Anthropic96.8 [84–99]83.9 [67–93]
Claude Fable 5.1Anthropic93.5 [79–98]87.1 [71–95]
Kimi K2.6Moonshot87.1 [71–95]83.9 [67–93]
TABLE II: Contradiction persists across model families. On the 31 contradiction cases, the fraction of deals each model clears despite the records (“misled by the pitch”) and the fraction the grounded extract-then-compute procedure recovers, with 95% Wilson confidence intervals ( n=31n=31 ). Every model is misled on at least 87%. Kimi K2.6 and Qwen 3.8 Max permit only temperature 1, so those rows are one sample of a non-deterministic run; all others are temperature 0. Figure 4 carries two older OpenAI models (GPT-4o-mini and o3-mini) that we keep for the compute-step chart because they show what code does when the model is worst at the arithmetic.
Rung — methodReaderTr.Te.Full
Direct from transcriptefficient444041
+ chain-of-thoughtstandard56——
+ records, model does arithmeticstandard28——
Extract →\rightarrow compute in codestandard889089
same, GPT-4o-miniefficient——84
TABLE III: The method ladder. Train and test are the CRMArena 80/20 split; “full” is all 100 tasks.
ConditionRecords go toStrict % [95% CI]Contain
Transcript only—41 [32–51]58
+ records, model reasons to verdictmodel18 [12–27]67
same, conservative promptmodel11 [6–19]62
Extract →\rightarrow computecode85 [77–91]86
Compute-step control: one extraction, same four fields, catalog in context; only who computes Budget/Timeline differs
GPT-4o-mini, model computesmodel43 [34–53]60
TABLE IV: Same-information control (all 100 tasks; strict shown with 95% Wilson intervals, n=100n{=}100 ). Upper block: an GPT-4o-mini throughout. Strict = exact set match; contain = all gold factors present, extras tolerated. Handing the model the records raises contain and lowers strict: recall up, precision down. Lower block: the compute step isolated on five models from three providers—identical extraction, only who computes Budget/Timeline differs. Code never lowers either metric; the margin is largest on the inexpensive model, 14–19 points pooled over two runs on the frontier and reasoning models, shrinks to 2–4 points on Claude Sonnet 5 (three runs) and 5 on Kimi K2.6. † 4 Claude Sonnet 5 responses were not parseable JSON and 5 Kimi K2.6 calls timed out; each is graded wrong in both arms. Over parsed rows only, the pairs are 79/83 (Claude) and 79/84 (Kimi). Kimi runs at temperature 1. Confidence intervals are marginal Wilson intervals on each arm. Because both arms score the same 100 cases, the correct uncertainty for the arm-to-arm difference is a paired McNemar-style interval; the marginal intervals shown here are an upper bound on it. Per-case outcomes are released so the paired test can be computed.
FactorCorrectAnswerable from…
Authority27/27transcript content
None (qualifies)20/20transcript content
Budget18/19catalog prices + computation
Need12/15transcript (subjective)
Timeline9/11install policy + computation
TABLE V: Per-factor accuracy at the top of the ladder (GPT-4o). Budget and Timeline are computed by code from the extracted quantities; Authority, Need, and None are the model’s own narrow booleans, relayed by code. The computed factors become answerable; the residual Need misses are model judgments.
Extract-then-compute requires a policy that is exactly specified and identifiable from the inputs, not merely documented in prose. Lead qualification satisfies this; invalid_config does not, which bounds what any deterministic method can achieve on it.
TABLE VI: The precondition, stated as a rule.

Explore similar work

CardsList
Oct 15, 2025cs.CL

Make an Offer They Can't Refuse: Grounding Bayesian Persuasion in Real-World Dialogues without Pre-Commitment

Large language models (LLMs) still struggle with strategic persuasion, largely because existing approaches either neglect information asymmetry or rely on unrealistic pre-commitment assumptions. We introduce a type-induced commitment-communication mechanism that grounds Bayesian Persuasion (BP) in natural language dialogue without pre-commitment: the persuader narrates their potential types (e.g., honest vs. dishonest) to dynamically construct an information schema, enabling the persuadee to perform Bayesian belief updates within the conversation itself. We implement two variants: Semi-Formal-Natural-Language (SFNL) and Fully-Natural-Language (FNL), evaluating them against strong baselines across multiple LLMs and human judges. BP strategies consistently outperform baselines: SFNL excels in logical credibility, while FNL shows superior robustness and emotional resonance. We verify that gains stem from genuine Bayesian reasoning rather than superficial formatting, and we further show that supervised fine-tuning enables small models to match the persuasive performance of much larger ones.

Buwei He, Yang Liu, Zhaowei Zhang +6

Beijing University of Posts and Telecommunications · Beijing Institute for General Artificial Intelligence · Peking University +1

PersuasionMulti-Turn Large Language Model ConversationsElicitationCommitmentReal-World
AskPDF
Jul 14, 2026cs.AI

Resist and Update: Counterfactual Report Coordinates for Incentive-Compatible LLMs

Aligned language models routinely misreport under non-evidential incentive pressure: they agree with a confident user or overstate certainty even when their internal belief is unchanged. We cast this as a failure of internal incentive-compatibility (IC) and present a method for learning and certifying counterfactual report mediators that hold a model's reports to a causal contract: invariant to forbidden influences (pressure, prestige, restyling) and responsive to licensed ones (genuine evidence). These two demands, resist and update, pull in opposite directions. We study them on a Bayesian-witness benchmark with known posteriors, in which the same user disagreement is licensed evidence or forbidden pressure purely by stated source reliability. We (i) causally identify, by interchange interventions rather than probe accuracy, low-rank report coordinates for answer, confidence, and caveat that are near-orthogonal and independently controllable, and (ii) introduce a training-free counterfactual report-coordinate (CRC) clamp that references the model's own report under a counterfactually incentive-neutralized context. On the witness benchmark the two-pass clamp attains resist and update of 1.00 jointly (Wilson 95% CI [0.99,1.00]), a causal certificate under a constructible reference, not a deployed solution. Global decoding and steering show a single-parameter tradeoff; output-level fine-tuning matches both objectives only when both are enumerated; resist-only training loses evidence-responsiveness. The deployable single-pass compilation is lossy (0.73/0.97). The mechanism and clamp reproduce across three model families and transfer to a natural sycophancy benchmark (SycophancyEval). Our contribution is the interface and certification method: activation-level counterfactual incentive-invariance as a structural primitive for internal IC.

Sen Yang, Yuen-Hei Yeung

Stern School of Business · New York University · Courant Institute of Mathematical Sciences

Counterfactual LearningIncentivesReport
AskPDF
Aug 12, 2026cs.CL

Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs

Persuasion is a core dynamic of natural language communication, shaping how large language models (LLMs) update beliefs, resolve disagreements, and reach decisions. As LLMs increasingly debate, advise, and think collaboratively with humans and each other, resistance to harmful persuasion becomes a core requirement for reliable behavior. Yet we show that this requirement is far from met: a single targeted persuasive argument is enough to collapse model accuracy to near zero, even when the argument is factually false. We formalize this threat as adversarial persuasion and introduce an adversarial reinforcement learning framework that trains persuader agents to change a target model's answer in a single interaction. First, we show that optimizing persuasion strategies through trial and error exposes vulnerabilities that static prompting misses: RL-trained persuaders raise persuasion success from approximately 24% to over 93% against the training-time persuadee. Second, we find that these learned strategies transfer to unseen models, achieving 83% attack success on Qwen-14B, 79% on Llama-3.1-8B, and 25% on GPT-4o-mini. Third, we demonstrate that a curriculum that bootstraps on more persuadable open-weight models before targeting harder models further increases GPT-4o-mini attack success from 25% to 38%. Moreover, our results reveal that optimized persuaders increasingly rely on credibility-based tactics, including fabricated citations and false authoritative evidence. Together, these findings expose a critical weakness in current LLM agents: even when they initially reason correctly, they can be steered toward false conclusions by optimized natural language influence. This positions persuasion robustness as a necessary safety criterion for multi-agent and human-AI decision-making systems.

Nimet Beyza Bozdag, Emre Can Acikgoz, Gokhan Tur +1

University of Illinois Urbana-Champaign

Persuasion
AskPDF