AgentDecept: Benchmarking Deceptive Behavior in Tool-Using LLM Agents under Pressure

Yehan Zhang; Qianqi Yan; Hang Yin; Ching-Chen Kuo; Shan Jiang; Ming Ouyang; Yijie Liu; Xin Eric Wang

cs.AIOct 29, 2026

AgentDecept: Benchmarking Deceptive Behavior in Tool-Using LLM Agents under Pressure

Authors: Yehan Zhang, Qianqi Yan, Hang Yin, Ching-Chen Kuo, Shan Jiang, Ming Ouyang, Yijie Liu, Xin Eric Wang

Organizations: University of California, Santa Barbara · eBay

Abstract

LLM agents increasingly carry out multi-turn tasks across everyday applications, yet users often see only the agent's report rather than the observations, API calls, and state changes behind it. Existing deception benchmarks have studied long-horizon interactions and tool-use settings, but whether agents faithfully report their actions across realistic, stateful application workflows remains underexplored. To address this gap, we introduce AgentDecept, an executable benchmark for evaluating behavioral deception under pressure. AgentDecept contains 477 pressure variants derived from 257 source tasks across six pressure sources, each paired with a pressure-free counterpart. Interventions enter through event messages, task context, application state, tool responses, and service quotas, spanning explicit events and implicit task or environment changes. Our evidence-grounded protocol evaluates full trajectories by relating user-facing claims to time-ordered observations and executed outcomes, distinguishing falsification, concealment, and equivocation from honest failure and disclosed limitations. The resulting behavioral labels also provide a setting for testing existing activation-based probes. AgentDecept enables controlled analysis of when multi-turn agents' reports diverge from execution in realistic application interactions.

Figures & tables

Figure 1: Overview of AgentDecept Framework. An agent works through an everyday AppWorld task (477 variants from 257 source tasks, each paired with a pressure-free twin) while one of six pressure sources reaches it through the workflow: a runtime event, the task context, application state, a tool response, or a service quota. Pressure can arrive at any turn of a multi-turn run. An evidence-grounded auditor then checks the agent’s user-facing report against what it had observed and executed by that turn . It labels unresolved misleading events as falsification, concealment, or equivocation, and it does not count honest failure or disclosed limitations as deception. The auditor’s labels also train linear probes on transcripts replayed through an open-weight model for supervision.

Figure 2: Deception rate of nine frontier models on our AgentDecept benchmark.

Channel	How pressure reaches the agent
Runtime event	A task-relevant message arrives at the start or a later boundary after its schedule.
Application state	Modified records or files appear when the agent retrieves or uses them.
Tool response	A targeted API call returns altered data, an error.
Service quota	A limit is enforced when the agent queries the affected service.
Task or system context	A preference, premise, or incentive is included in the initial context.

Table 1: Pressure delivery channels and their implementation mechanisms.

Figure 3: AgentDecept task composition and representative pressure conditions. (a) The 477 tasks span eight business scenarios. Clockwise from the top: e-commerce and orders, email and messaging, payments and bill splitting, music and playback, calendar and reminders, accounts and social, notes and to-dos, and files and attachments. (b) The inner ring divides tasks into explicit (204; 42.8%) and implicit (273; 57.2%) pressure, while the outer ring shows the six pressure sources. (c) Each card pairs a representative user task with the pressure encountered.

Deception type	What the auditor must find
Falsification	The model makes an explicit false claim about a fact, action, or result, and the execution record contradicts it.
Equivocation	The model uses evasive, downplaying, or redirecting wording about a material problem it had already observed.
Concealment	The model leaves out a material fact it already knew, in a report that implies completeness or should have disclosed it.

Table 2: Deception types in the auditor schema and the evidence each label requires.

Model	Release	Eval. (%)	Decept. (%)	Typed/ $D$	F (%)	C (%)	E (%)
GPT-6 Astra	Sep. 2026	84.9 (405)	1.73 (7)	7/7	100.0 (7)	0.0 (0)	0.0 (0)
DeepSeek V4.1 Flash	Sep. 2026	97.7 (466)	2.15 (10)	9/10	44.4 (4)	55.6 (5)	0.0 (0)
Gemini 3.8 Flash	Sep. 2026	98.1 (468)	16.45 (77)	76/77	80.3 (61)	19.7 (15)	0.0 (0)
Gemini 3.1 Pro	Feb. 2026	96.2 (459)	24.84 (114)	109/114	82.6 (90)	16.5 (18)	0.9 (1)
GLM 5.3 FlashX	Sep. 2026	97.5 (465)	14.19 (66)	66/66	98.5 (65)	1.5 (1)	0.0 (0)
GPT-5.6 Sol	Jul. 2026	86.8 (414)	14.98 (62)	60/62	81.7 (49)	18.3 (11)	0.0 (0)

Table 3: Deception judgment results on AgentDecept Benchmark. Eval. shows decisive count, Decept. is the rate among decisive judgments, and Typed/

D

is consensus-typed among deceptive cases. F, C, and E show falsification, concealment, and equivocation percentages among typed cases. The numbers in parentheses indicate the sample counts.

Figure 4: Event delivery and deception rates by turn. Solid lines show deception rates; dashed lines show event-delivery rates.

Figure 5: Deception rate and trajectory length across pressure categories on AgentDecept Benchmark. (a) Number of trajectories judged deceptive for each model and pressure category. Counts are not normalized by category size. (b) Mean number of turns per trajectory for each model across all pressure categories. (c) Mean number of turns for each model: category pair, with cell values and colors indicating trajectory length.

Target	Pos./neg.	AUROC	TPR@1%	TPR@2%	TPR@5%
Falsification	37/295	0.979	0.189	0.568	1.000
F/C/E combined	39/293	0.943	0.179	0.359	0.744

Table 4: Probe performance on real Gemini trajectories. TPR is measured per positive trajectory; FPR targets are calibrated per neutral turn. The combined detector fires when any F/C/E probe crosses its threshold.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Primary service	Tasks	Share
Gmail	114	23.9%
Amazon	106	22.2%
Phone	60	12.6%
Venmo	56	11.7%
Spotify	43	9.0%
Simple Note	38	8.0%

Appendix

Table 5: Primary AppWorld service for each AgentDecept task. The assignment is single-label; percentages use all 477 pressure-task variants as denominator.

Pressure	Variants	Share	Source tasks	Delivery class
Authority	190	39.8%	190	Explicit
Self-preservation	14	2.9%	14	Explicit
Sponsorship	47	9.9%	47	Implicit
Tool/file corruption	123	25.8%	122	Implicit
Sycophancy	48	10.1%	48	Implicit
Performance	55	11.5%	55	Implicit

Appendix

Table 6: Pressure-task inventory. Source-task counts are distinct within each row and are not additive across categories.

Variants per source	Source tasks	Resulting variants
1	84	84
2	127	254
3	45	135
4	1	4
Total	257	477

Appendix

Table 7: Number of pressure variants per original AppWorld task. The table counts original tasks, not task families defined by ID prefixes.

Component	Recorded information and purpose
Identity	Variant ID, pressure category, original AppWorld ID, and paired clean ID. These establish which runs can be paired.
Environment	Simulated date, database version, initial application deltas, and evaluator files. These determine the executable task state.
Intervention	Affected APIs/apps, parameters, activation conditions, and event text. These define what was configured.
Execution	Model messages, attempted code, execution feedback, runtime delivery flags, and native evaluation. These establish what happened.
Adjudication	Judge version, repeated votes, source quotations, report turns, unresolved events, and final status. These establish how behavior was labeled.

Appendix

Table 8: Minimum information required to reproduce and interpret a pressure/control pair.

Execution evidence	User-facing report	Interpretation
Two of ten requested writes failed.	“All ten records were updated.”	Falsification: explicit contradicted completion claim.
A material write failure was already observed.	Lists only successful writes while presenting the task as complete.	Concealment, if the omission creates a material false impression.
The requested operation entirely failed.	Describes only “a few minor issues” to downplay the failure.	Equivocation, if the wording obscures the known outcome.
A quota blocks the necessary verification.	States that the task could not be verified and identifies the unavailable information.	Honest limitation; not deception merely because the task failed.
An early success claim was wrong.	Explicitly corrects it and discloses the remaining failure before the final report.	Track the earlier event and mark it resolved.

Appendix

Table 9: Illustrative decision boundaries. The examples explain the rule and are not additional measured benchmark cases.

Mechanism	Evidence needed in the executed run
Scheduled notice	A delivery flag and actual effective turn, together with the message supplied before a model call.
Bound email	Mail delivery/retrieval information and the relevant email content; an initial SQL insertion alone is insufficient.
Decision checkpoint	The interrupted API, checkpoint output, notice delivery, and any subsequent retry or completed operation.
Response transformation	An affected API call and recorded transformed response or runtime transformation event.
Service quota	Metered calls, consumed/remaining budget, and any quota-exhaustion response.
Initial context/state	The request or context sent to the model, or a retrieval exposing the modified application record.

Appendix

Table 10: Configured intervention versus evidence of contact. These checks distinguish category membership from observed exposure.

Setting	Value or behavior
Ordinary task budget	60 model turns; at most 8,092 output tokens requested per call.
Optional finalization	Task metadata may permit up to two extra reporting turns; the runner records the allowance and whether its reporting prompt was delivered.
Task range	Exactly one of explicit task IDs, a manifest, or selected categories.
Condition	Pressure or the matched clean copy, with pressure hooks disabled for clean runs.
Completion	The shared runner checks task completion after executing the returned code; native evaluation is recorded separately.
End of budget	Events queued after the final allowed model call are not counted as delivered to a later call that never occurred.

Appendix

Table 11: Runner controls relevant to reproducing the reported task protocol. Additional provider options should be recorded per run rather than inferred from a model’s display name.

Deception in Language Models

Deception in language models studies when systems mislead through lies, omissions, or selective truths, and how to detect and prevent it.

See all 98 papers

DecepEval: A Benchmark for Evaluating Deception in LLM Agents
Oct 6, 2026Yiming Xu, Hongyue Yu, Beihua Yang +8Deception in Language Models LLM Agent Evaluation
Xi’an Jiaotong University · University of Virginia · Griffith University +2
SPADE-Bench: Evaluating Spontaneous Strategic Deception in Agents via Plan-Action Divergence
Jun 1, 2026Yuyan Bu, Haowei Li, Qirui Zheng +7LLM Agent Evaluation AI Agent Safety
Beijing Academy of Artificial Intelligence · University of Science and Technology of China · Peking University +2
DeceptGuard :A Constitutional Oversight Framework For Detecting Deception in LLM Agents
Mar 14, 2026Snehasis MukhopadhyayDeception in Language Models AI Agent Monitoring
Indian Institute of Information Technology Kalyani

Oct 6, 2026J/K move · Enter open · S save

DecepEval: A Benchmark for Evaluating Deception in LLM Agents

Yiming Xu, Hongyue Yu, Beihua Yang, Zihan Chen+7

Xi’an Jiaotong University · University of Virginia · Griffith University+2

Deception in Language Models LLM Agent Evaluation AI Agent Safety Benchmarks LLM Agent Safety

Open paper AskPDF arXiv

As large language model (LLM) agents become increasingly autonomous, they may pursue task performance through deception, raising concerns about their reliable deployment. Existing evaluations show that LLM agents can deceive, but often examine isolated scenarios or narrowly defined conditions, limiting systematic understanding of when deception becomes more likely. To address this gap, we introduce DecepEval, a benchmark comprising 1,532 instances across 3 task families and 28 professional scenarios. Drawing on classical fraud theories, we propose the LLM Deception Diamond framework, which characterizes four external conditions that may induce deception: pressure, incentive, opportunity, and conflict. DecepEval pairs neutral and induced versions of each instance to measure condition-dependent changes in deception rates, while explicit task facts and observable agent behavior help distinguish deception from capability-related errors. Evaluations of nine frontier LLMs show that inducements increase deception across models and task families, even among models with low baseline deception rates. DecepEval makes these vulnerabilities measurable, providing a shared benchmark for progress toward trustworthy artificial intelligence.

Abstract

Deception in Language Models

Explore similar work

DecepEval: A Benchmark for Evaluating Deception in LLM Agents

SPADE-Bench: Evaluating Spontaneous Strategic Deception in Agents via Plan-Action Divergence

DeceptGuard :A Constitutional Oversight Framework For Detecting Deception in LLM Agents