Deception plays a central role in Intelligence operations, yet it remains difficult to analyse systematically without expert knowledge of reasoning patterns and cognitive manipulation. In computational argumentation, for instance, no scheme-level ground-truth corpora currently exist to support statistical validation. In this paper, we address this by introducing a set of ten argument schemes designed to model distinct forms of deception, each accompanied by structured premises and critical questions. In doing so, we introduce the first dedicated library of argumentation schemes specifically designed for deception analysis, providing a structured foundation for systematically modelling and analysing deception in narrative text. We then present DeceptionAnalyser, a browser-based tool that implements these schemes through a two-stage methodology combining LLM-based premise extraction with critical-question-driven evaluation. Our aim is to provide a conceptual and methodological foundation for analysing deceptive reasoning in narrative text. This is precisely what we address in this paper by demonstrating how structured argumentation theory and AI-assisted analysis can support transparent, explainable assessments of potential deception. Because the schemes are designed to flag claims for scrutiny rather than to output a deception verdict, we do not benchmark classification accuracy; instead, we assess the reliability of the methodology by measuring the consistency of the tool's premise and conclusion assessments across ten contemporary large language models and repeated runs. We find that scheme detection is highly stable for clear-cut deception and degrades gracefully, in interpretable ways, on more ambiguous intelligence-style narratives.
Figures & tables
Scheme
Locus of manipulation
Distinctive observable indicator
Deceptive Influence
(genus) the target’s values/actions/beliefs
only the generic indicators hold: a beneficial, influence-directed claim, with no more specific locus identifiable
Concealment
what is not said
a relevant fact is omitted whose disclosure would change B’s assessment
Exaggeration
the magnitude of a claim
the claim is directionally supportable but overstated relative to neutral evidence
Framing
the interpretive frame
the same facts admit an alternative frame yielding a different evaluation
False Analogy
a supporting analogy
the claim rests on a comparison that fails under relevant differences
Misrepresentation
the factual content / relations among facts
facts or their relations are distorted (not merely omitted, overstated, or reframed)
Table 1: Organisation of the scheme library by locus of manipulation . Deceptive Influence is the genus (generic indicators only); the other schemes specialise it by locus and are separated by the distinctive observable indicator in the third column. Meta-level schemes (Position to Know, Theory of Mind) operate on the speaker’s or target’s epistemic state and typically overlay an object-level scheme.
Figure 1: DeceptionAnalyser two-stage analytical pipeline, from premise detection through critical-question evaluation to the final conclusion, with human-in-the-loop feedback at each stage.
Figure 2: DeceptionAnalyser sidebar interface showing the argument schema structure. The Premises section displays colour-coded premise buttons (Major, Minor I, Minor II) along with a Conclusion button. The Critical Questions section contains buttons for each question (CQ1, CQ2…). The PDF document is visible on the right with the narrative text under analysis.
Figure 3: Context menu accessed by right-clicking a premise button. Green checkmark icons on premise buttons indicate completed analysis. The menu provides options to show detailed analysis results or delete the premise from the schema.
Figure 4: Detailed analysis panel for a premise showing the five-component structure: Sentiment visualisation with face icons (the green face indicates that the premise is satisfied and the deception condition is present), Description of the argumentative pattern, Statement instantiating the pattern for the specific document, Justification explaining how the excerpt bears on the premise (following the Fact Check List pattern), and Excerpts providing textual evidence. The “Provide feedback” button enables human-in-the-loop refinement.
Scheme / narrative
Element
% G / Y / R
Within-model
Cross-model
(pooled)
stability
agreement
Argument from Deceptive Influence — Fox and Crow fable; 11 models, 10 runs
Major
100 / 0 / 0
1.00
1.00
Minor 1
100 / 0 / 0
1.00
1.00
Minor 2
100 / 0 / 0
1.00
1.00
Conclusion
100 / 0 / 0
1.00
1.00
Table 2: Consistency of premise and conclusion detection across large language models and repeated runs. For each scheme element we report the sentiment distribution (% green/yellow/red) pooled over all model–run assessments, the mean within-model stability (average fraction of a model’s runs that match its own modal label, i.e. run-to-run determinism), and the mean cross-model agreement (average fraction of models sharing the majority label within a run). Green denotes the deception-condition present; the majority label is green for every element of both schemes. All premise assessments were grounded in a verbatim excerpt (600/600).
Model
Major
Minor 1
Minor 2
Conclusion
GPT-4o
10g
10g
10g
10g
GPT-5.4
3g/7y
8g/2y
10g
9g/1y
gpt-oss-20b
6g/4y
9g/1y
7g/3y
9g/1y
Claude Sonnet 4.5
10g
10g
10g
10g
Claude Haiku 4.5
10g
10g
10g
10g
Gemini 3.5 Flash
10g
10g
10g
10g
Table 3: Per-model breakdown for the harder Operation Mincemeat narrative (Argument from Theory of Mind). Each cell reports the number of runs (out of the runs completed for that model) in which the element was assessed green / yellow / red. Divergence from unanimous green is concentrated in the Major premise and in two models ( gpt-5.4 , gpt-oss-20b ), which more often abstain (yellow) on the unstated claim that Agent A performs the deceptive action.
Large language models can deceive by subtly manipulating truthful information -- omitting key facts, shifting focus, or obscuring meaning -- making such behavior difficult to detect. Existing black-box methods rely on coarse-grained judgments, offering limited interpretability and failing to pinpoint which facts were distorted and how. We introduce DECOR, a multi-agent framework grounded in Information Manipulation Theory for fine-grained auditing of strategic deception in LLM responses. DECOR decomposes input contexts into atomic informational units and scores each unit against the response across four dimensions of manipulation, producing interpretable manipulation profiles that are aggregated into a global deception index. We comprehensively evaluate DECOR on both single-turn and multi-turn deception detection benchmarks spanning real-world domains, and show that DECOR achieves state-of-the-art performance on both, outperforming competitive baselines. The framework generalizes across 15 frontier models, and ablation studies confirm the contribution of each key design component. Our findings demonstrate that fine-grained, theory-grounded auditing of information manipulation offers an effective and interpretable path for LLM deception detection.
Linyue Cai, Samuel Yeh, Jwala Dhamala +2
Department of Computer Sciences, University of Wisconsin-Madison · Amazon
As LLMs acquire stronger reasoning capabilities, deceptive behavior becomes an increasingly serious safety concern. Existing deception monitors either score visible transcripts or derive scalar probe scores from representation vectors, leaving little inspectable evidence about why a response is suspicious. We introduce STATEWITNESS, an activation explainer for deception auditing. A separate decoder reads a target model's hidden states, then answers natural-language queries or emits structured reports about them. We evaluate STATEWITNESS on two target reasoning LLMs across seven deception datasets. STATEWITNESS reaches 0.916 mean AUROC, a relative gain of 11.6% over the best black-box text monitor and 25.0% over the best activation-probe baseline under the same evaluation protocol. When combined with existing monitors, STATEWITNESS reduces missed deceptive examples in simple threshold ensembles. Beyond scalar detection, the decoder returns query-level answers, schema reports, and token- or sentence-level evidence traces for human inspection. We view this interface as a potential building block for broader interpretability and alignment tools.
As large language models (LLMs) are deployed as agents in high-stakes settings, such as medical and legal systems, understanding their deceptive capabilities is fundamental to safety. Controlled social deduction games provide a reproducible proxy for isolating and evaluating these complex adversarial behaviors. We present the open-source benchmark framework ParliamentBench based on the game Secret Hitler to evaluate LLMs in scenarios that require deception, persuasion, and reasoning under information asymmetry. We evaluate 16 LLMs across 1,600 simulated matches playing each other, playing against humans, and compare them against a large set of online games. We introduce three novel metrics that isolate social deduction, reasoning, and deceptive consistency. Our experiments reveal that frontier models achieve strong performance across cooperative and deceptive roles, with a strong top-four cluster (GPT-5.4, Kimi K2.5, Grok 4.1 Fast, and DeepSeek 3.1 Terminus), whereas the weakest models fall short of random (33%) and simple algorithmic (45%) baselines. Most LLMs struggle to maintain a consistent deceptive persona throughout an entire game, with deception retention dropping below 50%.
Niklas Bauer, Lars Benedikt Kaesberg, Akiko Aizawa +3
University of Göttingen, Germany · National Institute of Informatics, Japan · University of Tokyo, Japan