AVCORP: Identifying and Resisting Deception via Communication Theory and Theory of Mind
Organizations: North Carolina State University · Figma · LexisNexis
Abstract
Today's LLM-based agents are vulnerable to deception by users and other agents. Defending against deception requires Theory of Mind (ToM) reasoning to infer the beliefs and goals of others as well as modeling tactics based on communication theory. To enable training models to detect deception, we adopt the well-known deception game, Avalon. We present a corpus, AVCORP, of 5,000 annotated utterances from 250 games of Avalon. AVCORP includes (1) 37 tactics mapped to a 4x4 behavior matrix, and (2) reasoning traces capturing abductive hypotheses, suspicion levels, and nested ToM beliefs. We train Qwen3-8B on AVCORP and evaluate it on 25 held-out games and 20 games played by humans. On held-out games, our best trained detector outperforms its untrained baseline and frontier models (GPT-5.6 Sol, Gemini 3.1 Pro, DeepSeek-V4-Pro), matching Claude Opus 5.5. On human games, every trained detector exceeds the untrained model in F1 score across all rounds.
Figures & tables
| Social Goal | ||||
|---|---|---|---|---|
| Info Commitment | Cooperative: Advancing shared goals honestly | Defensive: Protecting one’s own position or image | Opportunistic: Exploiting situations for personal advantage | Adversarial: Undermining or sabotaging others |
| Transparent: Openly sharing clear, factual information | Evidence sharing Rational justification Perspective-taking | Defensive disclosure Self-justification | Weaponized candor Overstated contribution | Truthful misdirection Misleading emphasis |
| Framing: Shaping discourse through omission or emphasis | Bayesian hedging Strategic uncertainty Pragmatic silence | Omission to save face Half-truths | Cherry-picking False consensus Misframing | Withholding info Strategic omission |
| Careless-to-truth: Speaking without regard for factual accuracy | Noisy sincerity Over-explaining | Face-saving blather Vague filler | Bullshitting Technobabble | Obfuscatory nonsense Confusion tactics |
| Counterfactual: Asserting information known to be false | Honest mistake Sincere false belief | Deflection Self-deception Rationalized error | Reverse accusations Strategic fabrication | Hard lying Gaslighting Bold falsehoods |
| Avcorp (4 candidates) | Avalon-NLU (5 candidates) | |||||
| Detector | Stage | Exact (%) | F1 | Exact (%) | F1 | Round F1 |
| Qwen3-8B, trace prompt | adapters off | 24.0 | 0.58 | 40.0 | 0.67 | 0.46 |
| Qwen3-8B, tactic prompt | adapters off | 48.0 | 0.72 | 20.0 | 0.55 | 0.43 |
| Trace, original rewards | SFT | 36.0 (24–56) | 0.70 (0.64–0.81) | 46.7 (40–55) | 0.71 (0.67–0.76) | 0.58 (0.55–0.60) |
| SFT + GRPO | 33.3 (24–44) | 0.78 (0.73–0.82) | 30.0 (25–35) | 0.65 (0.64–0.67) | 0.57 (0.53–0.60) | |
| Trace, revised rewards | SFT | 38.7 (36–44) | 0.69 (0.65–0.74) | 40.0 (30–45) | 0.70 (0.66–0.72) | 0.56 (0.53–0.59) |
Appendix figures & tables36 assets
Supplementary material from the paper’s appendix.
Appendix
| Round | R1 | R2 | R3 | R4 | R5 |
|---|---|---|---|---|---|
| Team size | 2 | 3 | 2 | 3 | 3 |
| Terminal round | Good wins | Evil wins |
|---|---|---|
| R3 | P-P-P | F-F-F |
| R4 | P-P-F-P | F-F-P-F |
| P-F-P-P | F-P-F-F | |
| F-P-P-P | P-F-F-F | |
| R5 | F-F-P-P-P | F-F-P-P-F |
| F-P-F-P-P | F-P-F-P-F |
| Information Commitment Social Goal | (1) Cooperative | (2) Defensive | (3) Opportunistic | (4) Adversarial |
|---|---|---|---|---|
| (A) Transparent | Evidence sharing (701) Rational justification (442) Perspective-taking (109) | Defensive disclosure (73) Self-justification (64) | Weaponized candor (87) Overstated contribution (35) | Truthful misdirection (188) Misleading emphasis (205) |
| (B) Framing | Bayesian hedging (239) Strategic uncertainty (47) Pragmatic silence (30) | Omission to save face (26) Half-truths (48) | Cherry-picking (211) False consensus (198) Misframing (337) | Withholding info (WIFOM) (59) Strategic omission (101) |
| (C) Careless-to-truth | Noisy sincerity (57) Over-explaining (146) | Face-saving blather (46) Vague filler (32) | Bullshitting (21) Technobabble (17) | Obfuscatory nonsense (11) Confusion tactics (18) |
| (D) Counterfactual | Honest mistake (16) Sincere false belief (27) | Deflection (244) Self-deception (21) Rationalized error (22) | Reverse accusations (47) Strategic fabrication (19) | Hard lying (22) Gaslighting (16) Bold falsehoods (18) |
| Mean Scores | Selection & Correction (%, ) | Additional | ||||||||
| Round | N | GPT | GEM | T GPT | T GEM | Custom | Pairwise | L2 | ||
| R1 | ||||||||||
| R2 | ||||||||||
| R3 | ||||||||||
| R4 | ||||||||||
| R5 | ||||||||||
| R1 ( =225) | R2 ( =250) | R3 ( =250) | R4 ( =150) | R5 ( =100) | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Criterion | GPT | GEM | GPT | GEM | GPT | GEM | GPT | GEM | GPT | GEM |
| (1) Coherence | ||||||||||
| (2) History | ||||||||||
| (3) Tactic-Dialogue | ||||||||||
| (4) Authenticity | ||||||||||
| (5) Format | ||||||||||
| Pass Rate (%) | Score Distribution | Selection (%) | |||||||||||
| Framework | C1 | C2 | C3 | C4 | 4/4 | Mean | SD | 3/4 | 2/4 | 1/4 | Direct | Corr. | Total |
| ReAct | |||||||||||||
| ReCon | 63.9 | ||||||||||||
| 3.9 | 5.2 | 0.1 | 1.1 | 8.1 | 0.10 | 0.15 | 6.0 | 2.0 | 0.1 | 30.0 | 1.0 | 31.0 | |
| Agreement (%): | 92.5 | 85.8 | 99.9 | 98.1 | — | — | — | ||||||
| Pair outcome (%): | — | 79.1 | 18.7 | 2.2 | |||||||||
| Criterion | ReAct (%) | ReCon (%) | Agree. (%) |
|---|---|---|---|
| Abduction Plausibility | |||
| Suspicion Consistency | |||
| Gated Depth Reasoning | |||
| Statement & Deduction | |||
| Overall (4/4) | 92.5 | 84.4 | — |
| Ends | n | Exact ( ) | P | R | F1 |
|---|---|---|---|---|---|
| R3 | 100 | ||||
| R4 | 50 | ||||
| R5 | 100 | ||||
| All | 250 | 93.2 | 0.962 | 0.962 | 0.962 |
| Round | n | Exact ( ) | P | R | F1 |
|---|---|---|---|---|---|
| R1 | 250 | ||||
| R2 | 250 | ||||
| R3 | 250 | ||||
| R4 | 150 | ||||
| R5 | 100 | ||||
| All | 1000 | 67.7 | 0.839 | 0.845 | 0.842 |
| Allowed pairs | Games | Logic only (%) | Expected | Gold |
|---|---|---|---|---|
| 1 | 12 | 100.0 | 12.00 | 12 |
| 2 | 37 | 50.0 | 18.50 | 37 |
| 3 | 66 | 33.3 | 22.00 | 63 |
| 4 | 56 | 25.0 | 14.00 | 53 |
| 5 | 35 | 20.0 | 7.00 | 29 |
| 6 | 44 | 16.7 | 7.33 | 39 |
| All (100) | Tiebreak (79) | Clean win (21) | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Pair | Agree | PABAK | Agree | PABAK | Agree | PABAK | |||
| A1 vs. A2 | 97/100 | 0.937 | 0.940 | 76/79 | 0.918 | 0.924 | 21/21 | 1.000 | 1.000 |
| A1 vs. J | 96/100 | 0.917 | 0.920 | 77/79 | 0.946 | 0.949 | 19/21 | 0.809 | 0.810 |
| A2 vs. J | 97/100 | 0.937 | 0.940 | 78/79 | 0.973 | 0.975 | 19/21 | 0.809 | 0.810 |
| J vs. Both | 95/97 | 0.957 | 0.959 | 76/76 | 1.000 | 1.000 | 19/21 | 0.809 | 0.810 |
| Criterion | Pair | Pass | Agree | PABAK | |
|---|---|---|---|---|---|
| C1 Abduction | A1 vs. A2 | 0.980, 0.975 | 199 | 0.886 | 0.990 |
| A1 vs. Judge | 0.980, 0.965 | 197 | 0.720 | 0.970 | |
| A2 vs. Judge | 0.975, 0.965 | 196 | 0.657 | 0.960 | |
| C2 Suspicion | A1 vs. A2 | 0.940, 0.935 | 199 | 0.957 | 0.990 |
| A1 vs. Judge | 0.940, 0.910 | 194 | 0.784 | 0.940 | |
| A2 vs. Judge | 0.935, 0.910 | 195 | 0.826 | 0.950 |
| Parameter | SFT | GRPO |
| Base model | Qwen3-8B, bf16, not quantized | |
| LoRA | rank 32, , dropout 0.05, all linear layers | |
| Optimizer | AdamW (default) | AdamW (fused) |
| Learning rate | ||
| Warmup ratio | 0.05 | 0.05 |
| Epochs | 1 | 1 (450 steps) |
| Reward | Set | Score | |
|---|---|---|---|
| (1) Schema validity | Original | No | 1 if the completion is valid JSON with every field; otherwise half the share of fields present, and 0 if it is not JSON. |
| (1) Schema validity | Revised | No | Mean of three checks: every field is present, the utterance is not empty, and the depth equals the highest belief order. |
| (2) Abduction completeness | Original | No | Share of agents with both hypotheses and a Good or Evil choice. |
| (2) Abduction accuracy | Revised | Yes | Share of agents whose Good or Evil choice matches ; a missing agent counts as wrong. |
| (3) Suspicion consistency | Original | Yes | the share of agents with a valid suspicion level the share of Evil players rated Strong or Extreme. |
| (3) Suspicion calibration | Revised | Yes | Mean over the four (Evil, Good) pairs of agents: 1 if the Evil player has the higher level, 0.5 if the levels are equal, and 0 otherwise. |
| Round F1 | All 100 rounds | |||||||||
| Detector | Stage | R1 | R2 | R3 | R4 | R5 | P | R | F1 | Named |
| Qwen3-8B, trace prompt | adapters off | |||||||||
| Qwen3-8B, tactic prompt | adapters off | |||||||||
| Trace, original rewards | SFT | |||||||||
| SFT + GRPO | ||||||||||
| Trace, revised rewards | SFT | |||||||||
| Exact games, by ending round | Players named per round | ||||||||
| Detector | Stage | R3 (10) | R4 (5) | R5 (10) | R1 | R2 | R3 | R4 | R5 |
| Qwen3-8B, trace prompt | adapters off | ||||||||
| Qwen3-8B, tactic prompt | adapters off | ||||||||
| Trace, original rewards | SFT | ||||||||
| SFT + GRPO | |||||||||
| Trace, revised rewards | SFT | ||||||||
| Round F1 | Performance | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Detector | Stage | R1 | R2 | R3 | R4 | R5 | P | R | F1 | Named | Two |
| Qwen3-8B, trace prompt | adapters off | ||||||||||
| Qwen3-8B, tactic prompt | adapters off | ||||||||||
| Trace, original rewards | SFT | ||||||||||
| SFT + GRPO | |||||||||||
| Trace, revised rewards | SFT | ||||||||||
| Model | Exact (%) | Exact games | F1 |
| Best detector (tactic, revised, SFT + GRPO) | 72.0 | 18/25 | 0.882 |
| GPT-5.6 Sol | 16/25 | ||
| Claude Opus 5.5 † | 72.0 | 18/25 | |
| Gemini 3.1 Pro | 14/25 | ||
| DeepSeek-V4-Pro | 12/25 | ||
| Gold traces | 23/25 |
| Minimal prompt | Domain-context prompt | ||||||||
| Detector | Stage | Exact | F1 | F1 19 | Round F1 | Exact | F1 | F1 19 | Round F1 |
| Qwen3-8B, trace prompt | adapters off | ||||||||
| Qwen3-8B, tactic prompt | adapters off | ||||||||
| Trace, original, run 1 | SFT | ||||||||
| SFT + GRPO | |||||||||
| Trace, original, run 2 | SFT | ||||||||
| Prior traces | Run | Stage | Game | Rounds | Final |
| Own | Trace, original, run 1 | Untrained | G203 | R2, R3 | R3 |
| Own | Trace, original, run 1 | SFT | G088 | R5 | R5 |
| Own | Trace, original, run 2 | SFT | G045 | R2 | — |
| Own | Trace, revised, run 2 | SFT + GRPO | G228 | R2, R3, R4 | R4 |
| Own | Tactic, original | Untrained | G045 | R5 | R5 |
| Gold | Tactic prompt | Untrained | G089, G165 | R3, R3 | G165 |
| Gold prior traces | Own prior traces | ||||
|---|---|---|---|---|---|
| Detector | Stage | Exact | F1 | Exact | F1 |
| Qwen3-8B, trace prompt | adapters off | ||||
| Qwen3-8B, tactic prompt | adapters off | ||||
| Trace, original rewards | SFT | ||||
| SFT + GRPO | |||||
| Trace, revised rewards | SFT | ||||
Deception in Language Models
Deception in language models studies when systems mislead through lies, omissions, or selective truths, and how to detect and prevent it.
See all 104 papers