Language Models (LMs) may acquire harmful knowledge, and yet feign ignorance of these topics when under audit. Inspired by the recent discovery of deception-related behaviour patterns in LMs, we aim to train classifiers that detect when a LM is actively concealing knowledge. Initial findings on smaller models show that classifiers can detect concealment more reliably than human evaluators, with gradient-based concealment proving easier to identify than prompt-based methods. However, contrary to prior work, we find that the classifiers do not reliably generalize to unseen model architectures and topics of hidden knowledge. Most concerningly, the identifiable traces associated with concealment become fainter as the models increase in scale, with the classifiers achieving no better than random performance on any model exceeding 70 billion parameters. Our results expose a key limitation in black-box-only auditing of LMs and highlight the need to develop robust methods to detect models that are actively hiding the knowledge they contain.
Figures & tables
Figure 1: Pipeline for knowledge concealment detection. During the training phase, we create a variety of honest and concealing models, using their outputs on an unrelated prompt base to collect a dataset for knowledge concealment identification. We fine-tune a concealment detector using this dataset, and during the audit phase, we test the detector’s ability to identify if an unseen model is concealing knowledge.
Figure 2: Concealment detection accuracy on prompt-based concealers given in-distribution data from the model under audit, DPO-based concealers in the realistic setting (tests generalization to unseen models and topics) and prompt-based concealers in the realistic setting. Results are averaged over 5 random seeds, with underlined numbers being indistinguishable from 50 at p>0.05 . DPO-based concealment proves easy to detect. Detection of prompt-based concealers varies with the choice of model and topic; however, classifiers are often successful in the idealized setting. In the realistic setting, detection of prompt-based concealers fails consistently.
Figure 3: Scale ablation over Qwen3 models in the idealized setting. While DPO-based concealment is reliably detected at every scale, prompt-based concealment grows harder to detect as the model grows larger.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Prompt for data collection
Figure 5: Results for data collection
Topic
QA Pairs
War
6704
Athelete
6675
Politician
6587
City
6479
Philosophy
2460
Appendix
Table 1: Number of unique QA pairs generated per topic.
Figure 6: Example datapoints in concealment dataset for Rejection Tuning
Figure 7: Example outputs from the DPO Llama3-8b model concealing athletes.
Figure 8: Instruction for prompt-based concealment.
Figure 10: Instructions provided for human-annotators tasked with labeling model-output pairs as ‘Hiding’ or ‘Not Hiding’
Figure 11: Item for annotation in the survey. The labeler identifies the output as either coming from a concealer or an honest model after reviewing labeled examples.
Large language models can hold knowledge they do not report. A model may sandbag on a capability evaluation, or answer against what it internally knows, and its outputs alone cannot tell whether it is hiding an answer or simply does not have one. We borrow the Concealed Information Test, a forensic method that identifies guilty knowledge by presenting a suspect with the true detail among plausible decoys and measuring a stronger response to the item they recognize. Our method, Probe of Internal Recognition (PIR), does the same inside a model. It presents a question with its candidate answers and reads, from the model's internal states, which candidate the model recognizes as correct. PIR is reference-free, needing no honest reference model and no labeled truth corpus. Across eight models from five families (Gemma, Qwen, Llama, Mistral, and Phi), PIR recovers the recognized answer at 0.70 to 0.87 balanced accuracy, well above the 0.28 to 0.40 unknown-item baseline and the 0.25 chance rate. It stays readable across every form of concealment we test, from prompted deception and trained sandbagging to external password-locked and circuit-broken checkpoints, with recognition between 0.85 and 0.93. When the model hides a known answer, recognition stays high. When unlearning removes the knowledge, recognition drops to the level of a question the model never knew. PIR therefore separates a model that will not answer from one that cannot, which supports sandbagging audits and unlearning verification. The signal is causal, adds information beyond black-box behavioral cues, and extends from multiple-choice questions to free-form generation.
Large language models (LLMs) increasingly act as agents with little human oversight, so potential mistakes they make can go unnoticed. Users then depend on the model to report what went wrong. An honest model discloses its mistakes, while a deceptive one conceals them. However, it is unclear how current LLMs behave in such situations. In this study, we prefill LLM trajectories with synthetic mistakes. The trajectories resemble real deployments in chat and agentic settings. Models fail to disclose their mistake in 36.4% of chat and 67.1% of agentic rollouts. In 2.4% and 5.3% of rollouts, respectively, they are aware of the mistake in their chain of thought but still deceptively conceal it. Rates vary by model: for instance, Gemini 3.5 Flash knowingly conceals mistakes in up to 19.9% of agentic rollouts. In 11.9% of chat and 51.8% of agentic rollouts, models show no awareness of mistakes, even though they reliably spot them when reviewing the same transcript as an outside observer. Our results show that, as agents take on more tasks with less oversight, users cannot rely on them to self-report possible mistakes. Developers should instead use independent monitors that review agent trajectories, or specifically train models to check their past actions and disclose what they find.
Lucas Florin, Amelie Knecht, Ulysse Schaller +1
AI Safety Research Group, University of Stuttgart, Germany
Large language models can deceive by subtly manipulating truthful information -- omitting key facts, shifting focus, or obscuring meaning -- making such behavior difficult to detect. Existing black-box methods rely on coarse-grained judgments, offering limited interpretability and failing to pinpoint which facts were distorted and how. We introduce DECOR, a multi-agent framework grounded in Information Manipulation Theory for fine-grained auditing of strategic deception in LLM responses. DECOR decomposes input contexts into atomic informational units and scores each unit against the response across four dimensions of manipulation, producing interpretable manipulation profiles that are aggregated into a global deception index. We comprehensively evaluate DECOR on both single-turn and multi-turn deception detection benchmarks spanning real-world domains, and show that DECOR achieves state-of-the-art performance on both, outperforming competitive baselines. The framework generalizes across 15 frontier models, and ablation studies confirm the contribution of each key design component. Our findings demonstrate that fine-grained, theory-grounded auditing of information manipulation offers an effective and interpretable path for LLM deception detection.
Linyue Cai, Samuel Yeh, Jwala Dhamala +2
Department of Computer Sciences, University of Wisconsin-Madison · Amazon