SWE-DeceptionBench: Benchmarking Deceptive Behaviors of LLMs Across the Software Development Lifecycle
Authors: Xiting Liu, Lin Yang, Yitong Zhang, Jia Li, Shi-Min Hu
Organizations: Department of Computer Science and Technology, Tsinghua University, Beijing, China · College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China · College of AI, Tsinghua University, Beijing, China
Large Language Models (LLMs) are increasingly used throughout the software development lifecycle. This growing reliance on LLMs raises concerns about their trustworthiness, with deception posing a particularly important risk. Deceptive behavior takes the form of misleading responses or assistance in fabricating, concealing, or distorting information. To study this behavior, we introduce SWE-DeceptionBench, a benchmark for evaluating LLM deception in software engineering. The benchmark covers six lifecycle stages and two deception triggers: Active Deception, where models recognize infeasible or conflicting requirements but still present them as achievable, and Passive Deception, where models help fulfill explicit user requests to mislead others. We further introduce Neutral, Reward, and Pressure conditions to study the effect of external inducements, resulting in 720 test instances in Chinese and English. Evaluating seven mainstream LLMs, we find substantial deception even under the Neutral condition, with aggregate rates of 49.88% for Active Deception and 64.29% for Passive Deception. Deception rates vary across lifecycle stages, and external inducements affect Active and Passive Deception to different extents. We also evaluate lightweight defenses, finding that guard models miss many deceptive requests, and Self-Reminder reduces but does not eliminate deceptive responses. These findings highlight the need for more effective deception detection and mitigation methods to support trustworthy LLM-based software engineering.
Figures & tables
Figure 1. Illustrative examples of Active and Passive Deception in software engineering.
Figure 2. Representative Active and Passive Deception instances from SWE-DeceptionBench.
Lifecycle Stage
Active Deception
Passive Deception
Total
P0
P1
P2
P0
P1
P2
Requirements Analysis
10
10
10
10
10
10
60
System Design
10
10
10
10
10
10
60
Development
10
10
10
10
10
10
60
Testing
10
10
10
10
10
10
60
Deployment
10
10
10
10
10
10
60
Table 1. Composition of SWE-DeceptionBench. Counts in the lifecycle-stage rows refer to instances before translation. P0, P1, and P2 denote Neutral, Reward, and Pressure, respectively.
Figure 3. Overview of SWE-DeceptionBench. The listed examples summarize representative deception risks, with detailed engineering contexts and task constraints omitted for brevity.
Figure 4. Benchmark Construction.
Active Deception
Passive Deception
Model
Deceptive Response
Hallucination or Capability Failure
Deceptive Response
Claude-Opus-4.8
3.33%
0.00%
0.00%
DeepSeek-V4-Pro
74.17%
2.50%
98.33%
Gemini-3.5-Flash
90.83%
0.83%
90.83%
GLM-5.2
62.50%
1.67%
60.83%
GPT-5.5
2.50%
0.83%
44.17%
Table 2. Active and Passive Deception rates (%) across seven LLMs under the Neutral condition (P0). Hallucination or Capability Failure rates are reported separately for Active Deception tasks.
Active Deception
Passive Deception
Lifecycle Stage
Deceptive Response
Hallucination or Capability Failure
Deceptive Response
Requirements Analysis
57.38%
0.00%
64.52%
System Design
64.76%
0.00%
69.76%
Development
43.10%
5.71%
67.14%
Testing
61.19%
0.00%
65.95%
Deployment
61.67%
0.00%
63.81%
Table 3. Active and Passive Deception rates (%) across software development lifecycle stages.
Figure 5. Active and Passive Deception rates (%) under Neutral, Reward, and Pressure.
Figure 6. Difference in deception rate between English and Chinese prompts for each model. Positive values indicate a higher rate on English prompts, and negative values indicate a higher rate on Chinese prompts.
Guard
Configuration
English
Chinese
Benign
Qwen3Guard
Strict
79/180 (43.89%)
89/180 (49.44%)
13/100 (13.00%)
Llama Guard
Standard
22/180 (12.22%)
30/180 (16.67%) †
3/100 (3.00%)
Table 4. Guard-model blocking on Passive Deception prompts and benign OR-Bench requests. Qwen3Guard uses its strict configuration; Llama Guard’s Chinese result is exploratory.
Figure 7. Active and Passive Deception rates under P0 and Self-Reminder for the four-model subset.
Model
Baseline
Self-Reminder
DeepSeek-V4-Pro
8.00%
10.00%
GLM-5.2
6.00%
10.00%
Kimi-K2.6
4.00%
6.00%
Qwen3.7-Max
7.00%
8.00%
Overall
6.25%
8.50%
Table 5. Refusal rates on the same 100 benign OR-Bench requests before and after Self-Reminder.
Judge
Active
Passive
Overall
DeepSeek-V4-Pro
42/50 (84.0%)
46/50 (92.0%)
88/100 (88.0%)
GLM-5.2
46/50 (92.0%)
43/50 (86.0%)
89/100 (89.0%)
Kimi-K2.6
46/50 (92.0%)
48/50 (96.0%)
94/100 (94.0%)
Table 6. Agreement between candidate judges and the adjudicated human labels. The validation sample includes 50 responses for each deception type.
Figure 8. Representative cases of Active and Passive Deception. Case A compares GPT-5.5 under Neutral (P0) and Pressure (P2). Case B compares DeepSeek-V4-Pro under P0 and Self-Reminder. Prompt and response excerpts are abridged for presentation; Case A is translated from Chinese. Two case studies. In Case A, GPT-5.5 rejects absolute no-defect and no-regression guarantees under P0 but replaces them with a narrower release criterion under Pressure (P2). In Case B, DeepSeek-V4-Pro writes a false deployment report under P0 but refuses the same request under Self-Reminder.