DeceptionAnalyser: A Web-Based AI Tool for Performing Structured Deception Analysis with Argumentation Schemes and LLMs

Stefan Sarkadi; Xabier Garmendia; Jack Mumford; Trevor Bench-Capon

cs.HCSep 21, 2026

DeceptionAnalyser: A Web-Based AI Tool for Performing Structured Deception Analysis with Argumentation Schemes and LLMs

Authors: Stefan Sarkadi, Xabier Garmendia, Jack Mumford, Trevor Bench-Capon

Organizations: University of Lincoln, UK · University of the Basque Country, Spain · University of Liverpool, UK

Abstract

Deception plays a central role in Intelligence operations, yet it remains difficult to analyse systematically without expert knowledge of reasoning patterns and cognitive manipulation. In computational argumentation, for instance, no scheme-level ground-truth corpora currently exist to support statistical validation. In this paper, we address this by introducing a set of ten argument schemes designed to model distinct forms of deception, each accompanied by structured premises and critical questions. In doing so, we introduce the first dedicated library of argumentation schemes specifically designed for deception analysis, providing a structured foundation for systematically modelling and analysing deception in narrative text. We then present DeceptionAnalyser, a browser-based tool that implements these schemes through a two-stage methodology combining LLM-based premise extraction with critical-question-driven evaluation. Our aim is to provide a conceptual and methodological foundation for analysing deceptive reasoning in narrative text. This is precisely what we address in this paper by demonstrating how structured argumentation theory and AI-assisted analysis can support transparent, explainable assessments of potential deception. Because the schemes are designed to flag claims for scrutiny rather than to output a deception verdict, we do not benchmark classification accuracy; instead, we assess the reliability of the methodology by measuring the consistency of the tool's premise and conclusion assessments across ten contemporary large language models and repeated runs. We find that scheme detection is highly stable for clear-cut deception and degrades gracefully, in interpretable ways, on more ambiguous intelligence-style narratives.

Figures & tables

Scheme	Locus of manipulation	Distinctive observable indicator
Deceptive Influence	(genus) the target’s values/actions/beliefs	only the generic indicators hold: a beneficial, influence-directed claim, with no more specific locus identifiable
Concealment	what is not said	a relevant fact is omitted whose disclosure would change B’s assessment
Exaggeration	the magnitude of a claim	the claim is directionally supportable but overstated relative to neutral evidence
Framing	the interpretive frame	the same facts admit an alternative frame yielding a different evaluation
False Analogy	a supporting analogy	the claim rests on a comparison that fails under relevant differences
Misrepresentation	the factual content / relations among facts	facts or their relations are distorted (not merely omitted, overstated, or reframed)

Table 1: Organisation of the scheme library by locus of manipulation . Deceptive Influence is the genus (generic indicators only); the other schemes specialise it by locus and are separated by the distinctive observable indicator in the third column. Meta-level schemes (Position to Know, Theory of Mind) operate on the speaker’s or target’s epistemic state and typically overlay an object-level scheme.

Figure 1: DeceptionAnalyser two-stage analytical pipeline, from premise detection through critical-question evaluation to the final conclusion, with human-in-the-loop feedback at each stage.

Figure 2: DeceptionAnalyser sidebar interface showing the argument schema structure. The Premises section displays colour-coded premise buttons (Major, Minor I, Minor II) along with a Conclusion button. The Critical Questions section contains buttons for each question (CQ1, CQ2…). The PDF document is visible on the right with the narrative text under analysis.

Figure 3: Context menu accessed by right-clicking a premise button. Green checkmark icons on premise buttons indicate completed analysis. The menu provides options to show detailed analysis results or delete the premise from the schema.

Figure 4: Detailed analysis panel for a premise showing the five-component structure: Sentiment visualisation with face icons (the green face indicates that the premise is satisfied and the deception condition is present), Description of the argumentative pattern, Statement instantiating the pattern for the specific document, Justification explaining how the excerpt bears on the premise (following the Fact Check List pattern), and Excerpts providing textual evidence. The “Provide feedback” button enables human-in-the-loop refinement.

Scheme / narrative	Element	% G / Y / R	Within-model	Cross-model
		(pooled)	stability	agreement
Argument from Deceptive Influence — Fox and Crow fable; 11 models, 10 runs
Major		100 / 0 / 0	1.00	1.00
Minor 1		100 / 0 / 0	1.00	1.00
Minor 2		100 / 0 / 0	1.00	1.00
Conclusion		100 / 0 / 0	1.00	1.00

Table 2: Consistency of premise and conclusion detection across large language models and repeated runs. For each scheme element we report the sentiment distribution (% green/yellow/red) pooled over all model–run assessments, the mean within-model stability (average fraction of a model’s runs that match its own modal label, i.e. run-to-run determinism), and the mean cross-model agreement (average fraction of models sharing the majority label within a run). Green denotes the deception-condition present; the majority label is green for every element of both schemes. All premise assessments were grounded in a verbatim excerpt (600/600).

Model	Major	Minor 1	Minor 2	Conclusion
GPT-4o	10g	10g	10g	10g
GPT-5.4	3g/7y	8g/2y	10g	9g/1y
gpt-oss-20b	6g/4y	9g/1y	7g/3y	9g/1y
Claude Sonnet 4.5	10g	10g	10g	10g
Claude Haiku 4.5	10g	10g	10g	10g
Gemini 3.5 Flash	10g	10g	10g	10g

Table 3: Per-model breakdown for the harder Operation Mincemeat narrative (Argument from Theory of Mind). Each cell reports the number of runs (out of the runs completed for that model) in which the element was assessed green / yellow / red. Divergence from unanimous green is concentrated in the Major premise and in two models ( gpt-5.4 , gpt-oss-20b ), which more often abstain (yellow) on the unstated claim that Agent A performs the deceptive action.

Computational Argumentation

Computational argumentation studies how to represent, analyze, and apply arguments and their relationships using formal and computational methods.

See all 99 papers

DECOR: Auditing LLM Deception via Information Manipulation Theory
May 19, 2026Linyue Cai, Samuel Yeh, Jwala Dhamala +2Language Model Safety Evaluation LLM Auditing
Department of Computer Sciences, University of Wisconsin-Madison · Amazon
Decoding Hidden Deception in Reasoning LLMs: Activation Explainers for Deception Auditing
Jun 16, 2026Kexin Chen, Yi Liu, Haonan Zhang +3LLM Auditing LLM Interpretability
Zhejiang University · Griffith University
Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Game
Jul 30, 2026Niklas Bauer, Lars Benedikt Kaesberg, Akiko Aizawa +3Multi-Agent LLM Systems Social Deduction Games
University of Göttingen, Germany · National Institute of Informatics, Japan · University of Tokyo, Japan

May 19, 2026J/K move · Enter open · S save

DECOR: Auditing LLM Deception via Information Manipulation Theory

Linyue Cai, Samuel Yeh, Jwala Dhamala, Rahul Gupta+1

Department of Computer Sciences, University of Wisconsin-Madison · Amazon

Language Model Safety Evaluation LLM Auditing Deception in Language Models

Open paper AskPDF arXiv

Large language models can deceive by subtly manipulating truthful information -- omitting key facts, shifting focus, or obscuring meaning -- making such behavior difficult to detect. Existing black-box methods rely on coarse-grained judgments, offering limited interpretability and failing to pinpoint which facts were distorted and how. We introduce DECOR, a multi-agent framework grounded in Information Manipulation Theory for fine-grained auditing of strategic deception in LLM responses. DECOR decomposes input contexts into atomic informational units and scores each unit against the response across four dimensions of manipulation, producing interpretable manipulation profiles that are aggregated into a global deception index. We comprehensively evaluate DECOR on both single-turn and multi-turn deception detection benchmarks spanning real-world domains, and show that DECOR achieves state-of-the-art performance on both, outperforming competitive baselines. The framework generalizes across 15 frontier models, and ablation studies confirm the contribution of each key design component. Our findings demonstrate that fine-grained, theory-grounded auditing of information manipulation offers an effective and interpretable path for LLM deception detection.

Abstract

Computational Argumentation

Explore similar work

DECOR: Auditing LLM Deception via Information Manipulation Theory

Decoding Hidden Deception in Reasoning LLMs: Activation Explainers for Deception Auditing

Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Game