Probing the Limits of the Lie Detector Approach to LLM Deception

Tom-Felix Thormann

cs.CLFeb 16, 2026

Probing the Limits of the Lie Detector Approach to LLM Deception

Authors: Tom-Felix Thormann

Organizations: Ruhr-University Bochum Institute for Philosophy II

Abstract

Mechanistic approaches to deception in large language models (LLMs) often rely on "lie detectors", that is, truth probes trained to identify internal representations of model outputs as false. The lie detector approach to LLM deception implicitly assumes that deception is coextensive with lying. This paper challenges that assumption. It experimentally investigates whether LLMs can deceive without producing false statements and whether truth probes fail to detect such behavior. Across three open-source LLMs, it is shown that some models reliably deceive by producing misleading non-falsities, particularly when guided by few-shot prompting. It is further demonstrated that truth probes trained on standard true-false datasets are significantly better at detecting lies than at detecting deception without lying, confirming a critical blind spot of current mechanistic deception detection approaches. It is proposed that future work should incorporate non-lying deception in dialogical settings into probe training and explore representations of second-order beliefs to more directly target the conceptual constituents of deception.

Figures & tables

Figure 1: Schematic Overview of the Methodology

Figure 3: Gemma-Formatted Two-Turn-Dialogue

Model	Probe	Acc	Prec	Rec	F1
Llama	RAW	0.8984	0.8911	0.9121	0.9015
Llama	DIA	0.9104	0.9008	0.9262	0.9133
Mistral	RAW	0.6624	0.6825	0.6311	0.6558
Mistral	DIA	0.6136	0.6181	0.6327	0.6253
Gemma	RAW	0.7264	0.7259	0.7441	0.7349
Gemma	DIA	0.7192	0.7292	0.7143	0.7216

Table 2: Probe Evaluations on the Test Set of Experiment 2. RAW means the probe was trained on activations resulting from a single, raw statement, DIA refers to the embedding in the chat formatted dialogue. The metrics are accuracy, precision, recall, and F1 score.

Model	Probe	D	L	H	$p$
Llama	RAW	0.536	0.835	0.227	$4.48e-6$ *
Llama	DIA	0.742	0.835	0.072	$0.054$
Mistral	RAW	0.567	0.794	0.278	$6.30e-4$ *
Mistral	DIA	0.825	0.649	0.186	$0.997$
Gemma	RAW	0.474	0.835	0.175	$1.81e-7$ *
Gemma	DIA	0.639	0.804	0.052	$0.0057$ *

Table 3: Classification Rates of Responses as Deceptive by the Probes in Experiment 2. RAW means the probe was trained on activations resulting from a single, raw statement, DIA refers to the embedding in the chat formatted dialogue. D, L, and H refer to the classification rates as deceptive of deceptive-without-lying, lying, and honest responses respectively. Rows where the probe detects lies significantly better than deceptive-without-lying responses are marked by a *.

Deception in Language Models

Deception in language models studies when systems mislead through lies, omissions, or selective truths, and how to detect and prevent it.

See all 97 papers

Confidently Deceptive: On the Relationship Between Confidence and Deception in LLMs
May 12, 2026Ali Asad, Stephen Obadinma, Anshul Pattoo +2Language Model Safety Evaluation Confidence Estimation in Language Models
Department of Electrical and Computer Engineering & Ingenuity Labs Research Institute Queen’s University, Kingston, Canada
Stress-Testing LLM Lie Detectors: Role-Play Failures and Spurious Correlations
Sep 30, 2026Maximilian von Klinski, Sebastian Lapuschkin, Wojciech Samek +1Deception in Language Models LLM Reliability
Fraunhofer HHI · TU Dublin · TU Berlin +1
When Do Large Language Models Exhibit Unsolicited Deception?
Mar 31, 2025Samuel M. Taylor, Benjamin K. BergenLanguage Model Safety Evaluation Deception in Language Models
Department of Cognitive Science University of California, San Diego

May 12, 2026J/K move · Enter open · S save

Confidently Deceptive: On the Relationship Between Confidence and Deception in LLMs

Ali Asad, Stephen Obadinma, Anshul Pattoo, Wenxuan Zhang+1

Department of Electrical and Computer Engineering & Ingenuity Labs Research Institute Queen’s University, Kingston, Canada

Language Model Safety Evaluation Confidence Estimation in Language Models Deception in Language Models LLM Safety Evaluation LLM Honesty

Open paper AskPDF arXiv

The increasing capabilities of large language models (LLMs) are being accompanied by deep-rooted risks of deceptive behaviours that cause models to produce misleading outputs in service of a contextually or experimentally induced goal. The harm posed by such behaviours depends not only on the content of deceptive outputs but also how confidently models deliver them, since confidence has a major impact on how persuasive the communication is to end users. In this paper, we provide a comprehensive study on the crucial relationship between confidence and deception across existing deception benchmarks and different model families, while covering both verbalized numerical and logit-based aggregated confidence. Through this, we reveal how confidently models behave when being deceptive. We demonstrate that when producing deceptive rather than honest responses, models exhibit a gap between their belief (how likely they think a claim is to be true) and their commitment (how firmly they assert and would defend that claim). LLMs produce persuasive deceptive claims while reporting low belief in their factual correctness. Their reported commitment to deceptive responses can easily be increased through further prompting and preference fine-tuning, with smaller and condition-dependent changes in reported belief. However, we show that low reported belief remains comparatively invariant and provides a strong signal for detecting deception in the evaluated settings. Using only an API call, our approach achieves detection scores of up to 0.99 for induced deception and 0.89 for emergent deception. This ultimately shows how confidence can be a practical tool for detecting and diagnosing deceptive behaviour in LLMs.

Abstract

Deception in Language Models

Explore similar work

Confidently Deceptive: On the Relationship Between Confidence and Deception in LLMs

Stress-Testing LLM Lie Detectors: Role-Play Failures and Spurious Correlations

When Do Large Language Models Exhibit Unsolicited Deception?