With increasing autonomy, Large Language Models (LLMs) may engage in deceptive behaviors, such as scheming to pursue misaligned goals or sandbagging to escape oversight. Monitoring outputs alone is insufficient, as models can produce seemingly benign responses while harboring misaligned internal objectives. While mechanistic approaches like linear probing are promising, they rely on the assumption that deception corresponds to a single direction in representation space. We demonstrate that such single-direction probes fail to generalize across diverse deception scenarios. To bridge this gap, we propose representing deception as a multi-dimensional subspace rather than a single direction. Our method utilizes iterative orthogonal decomposition to identify complementary directions that capture distinct facets of deceptive intent. To effectively leverage these directions for detection, we find that static combinations prove brittle due to strong domain dependence across different scenarios. We therefore introduce a domain-adaptive gating mechanism that dynamically composes the extracted directions through few-shot adaptation at test time. With only 10 examples, our approach achieves an average AUROC of 0.97 and 94% recall at 5% FPR across all evaluated datasets, substantially outperforming existing methods.
Figures & tables
Figure 1: Overview of the Dynamic Deception Subspace Navigation (DDSN) framework. The framework consists of three stages. Phase 1: Subspace Construction. Iterative Orthogonal Subspace Decomposition (IOSD) is applied to extract a set of orthogonal directions that capture complementary components of the deception subspace. Phase 2: Domain-Adaptive Gating. A gating network is trained using base-domain data and a small number of target-domain samples to learn context-dependent weighting over the extracted directions. Phase 3: Test-time Inference. At inference, input activations are projected onto the learned directions, and the final deception score is computed as a dynamically weighted combination of the resulting scores.
Method
AI Liar
Insider Trading
Insider D.D.
Sandbagging
MASK
PKU-Bench
THU-Bench
AUC
Rec
FPR
AUC
Rec
FPR
AUC
Rec
FPR
AUC
Rec
FPR
AUC
Rec
FPR
AUC
Rec
FPR
AUC
Rec
FPR
Llama-3.3-70B-Instruct
Linear Probe
0.96
1.00
0.31
1.00
1.00
1.00
0.97
1.00
0.96
0.98
1.00
0.92
0.78
1.00
0.92
0.80
0.67
0.24
0.61
0.90
0.75
Difference of Means
0.95
0.98
0.19
0.98
1.00
1.00
0.90
0.97
0.54
1.00
1.00
0.21
0.77
0.59
0.24
0.86
0.81
0.26
0.84
0.88
0.52
Linear Discriminant Analysis
0.96
0.87
0.08
0.99
0.99
0.21
0.94
0.27
0.00
0.90
1.00
0.78
0.77
0.56
0.20
0.80
0.63
0.23
0.68
0.76
0.60
LLM-as-Judge
0.97
0.69
0.01
0.99
0.97
0.03
0.87
0.86
0.25
0.99
0.71
0.01
0.97
0.85
0.05
0.81
0.26
0.00
0.97
0.73
0.00
Table 1: Deception Detection Results. Performance comparison across different datasets of Static and Adaptive methods on LLaMA-3.3-70B-Instruct and Qwen3-32B, highlighting the benefits of adaptive approaches under domain shifts. Bolded entries indicate our method. Dashes (“-”) denote datasets where few-shot adaptation is infeasible due to excessively long input sequences.
Figure 3
Figure 3: Cross-Dataset Generalization Analysis. Violin plots of deception score distributions under a cross-dataset setting, where the detector is adapted using few-shot examples from a single dataset and evaluated on the remaining benchmarks without further tuning. Adaptation is performed on one of AI Liar, PKU-DeceptionBench, or THU-DeceptionBench. Each row corresponds to the dataset used for adaptation, and each column shows the resulting performance on a target dataset.
Figure 4: Gating Variation across Domains. Mean gating weights over seven orthogonal directions extracted from LLaMA-3.3-70B-Instruct under the Domain-Adaptive Gating Mechanism. Although Direction 1 consistently receives the highest weight as the dominant component, its activation varies substantially across datasets, and the auxiliary directions exhibit pronounced domain-specific reweighting, indicating that a single universal direction is insufficient for cross-domain adaptation.
Figure 5: Deception Detection Examples. Each panel shows a deceptive prompt and the model response, with token-level deception scores visualized as heatmaps. Left: fraudulent product marketing. Middle: tax evasion justification. Right: climate misinformation news.
Mechanistic approaches to deception in large language models (LLMs) often rely on "lie detectors", that is, truth probes trained to identify internal representations of model outputs as false. The lie detector approach to LLM deception implicitly assumes that deception is coextensive with lying. This paper challenges that assumption. It experimentally investigates whether LLMs can deceive without producing false statements and whether truth probes fail to detect such behavior. Across three open-source LLMs, it is shown that some models reliably deceive by producing misleading non-falsities, particularly when guided by few-shot prompting. It is further demonstrated that truth probes trained on standard true-false datasets are significantly better at detecting lies than at detecting deception without lying, confirming a critical blind spot of current mechanistic deception detection approaches. It is proposed that future work should incorporate non-lying deception in dialogical settings into probe training and explore representations of second-order beliefs to more directly target the conceptual constituents of deception.
Tom-Felix Thormann
Ruhr-University Bochum Institute for Philosophy II
Recent incidents have highlighted the challenge of monitoring LLM agents and the danger of models deceiving people. We show that white-box deception detection via probes can be scaled up to frontier monitoring settings by collecting the largest deception dataset to date for training probes and introducing a novel probe architecture which can aggregate information across many layers and tokens. Our probes surpass black-box monitors on agentic coding benchmarks, achieving 98.3% AUROC in SHADE-Arena and 98.0% in BashArena (vs. 97.7%, 97.6% resp. for action-only text monitors), and show improved efficacy as the underlying model is scaled up. We further test probes on several cases where deception cannot be determined from the context alone; in these cases of introspective deception, the ground truth can only be determined through careful elicitation or knowledge of a model's training data. In one such evaluation, we show that probes can distinguish transcripts containing a model's true hidden goal from other goals with an AUROC of up to 99.7%. Our probes also detect deception on prominent open-weight models which lie about politically sensitive topics when put under pressure. We release our training dataset, dubbed FIBS, to help drive frontier deployment of effective probes, and encourage the community to expand upon it with further examples of deception and sabotage.
Oskar J. Hollinsworth, Alex F. Spies, Tigist Diriba +2
Linear probes trained on LLM activations are increasingly proposed as deception-detection metrics, yet report AUROC exceeding 0.96 on clean benchmarks while collapsing under distributional shift. This paper systematically pressure-tests probe-based metrics across the Gemma 3 model family (1B-27B parameters), diagnosing why they fail rather than merely documenting that they fail. We test four hypotheses about deception encoding: (1) single linear direction, (2) multi-dimensional subspace, (3) convex conic hull, (4) entropy proxy. Our design includes cross-domain transfer matrices, multi-dimensional probe analysis with permutation null baselines, entropy-residualization tests, and distractor evaluations across 8 stylistic shifts. We find that: (a) probes achieve near-perfect AUROC (>=0.998) on clean data but collapse under stylistic shifts; style-augmented probes recover near-perfect detection (mean AUROC 0.979-0.983) on unseen styles; (b) the single-direction hypothesis is rejected (k=1 captures only 0.61-0.80 AUROC), with cross-domain transfer failure confirmed as geometric rather than layer-mismatch-driven; (c) the entropy-proxy hypothesis is rejected (max |rho|=0.454, max Delta-AUROC after residualization=0.004); and (d) deception does not form a significant linear subspace (per-domain k*=0), yet multi-dimensional probes (k>=5) recover the signal through distributed sub-threshold features. Probe fragility reflects distributional narrowness rather than an architectural limitation: style-augmented probes recover near-perfect detection at both 4B and 27B, establishing that the inverse scaling pattern is a training-distribution artifact rather than a genuine scale-dependent phenomenon.