Testing the Limits of Truth Directions in LLMs

Angelos Poulis; Mark Crovella; Evimaria Terzi

cs.CLApr 4, 2026

Testing the Limits of Truth Directions in LLMs

Authors: Angelos Poulis, Mark Crovella, Evimaria Terzi

Organizations: Department of Computer Science, Boston University · Faculty of Computing & Data Sciences, Boston University

Abstract

Large language models (LLMs) have been shown to encode truth of statements in their activation space along a linear truth direction. Previous studies have argued that these directions are universal in certain aspects, while more recent work has questioned this conclusion drawing on limited generalization across some settings. In this work, we identify a number of limits of truth-direction universality that have not been previously understood. We first show that truth directions are highly layer-dependent, and that a full understanding of universality requires probing at many layers in the model. We then show that truth directions depend heavily on task type, emerging in earlier layers for factual and later layers for reasoning tasks; they also vary in performance across levels of task complexity. Finally, we show that model instructions can affect truth directions; simple correctness evaluation instructions significantly affect the geometry and the generalization ability of the truth probes. Our findings indicate that universality claims for truth directions are more limited than previously known, with significant differences observable for various model layers, task difficulties, task types, and prompt templates.

Figures & tables

Task	Operations	Example statement
Factual retrieval
F0	1 fact	“The city of Paris is in France.”
F1	1 fact + negation	“The city of Paris is not in Australia.”
F2	2 facts + conjunction	“It is the case both that the city of Paris is in France and the city of Dortmund is in Germany.”
Factual counting
F3	counting over 2 cities †	“Exactly 2 of the following cities are in France: Paris, Marseille.”

Table 1: Tasks with increasing reasoning steps within each group.

Panel of Figure 1 — Figure 1: Layer dependence of truth directions. (a) In-domain test-set performance. (b) F0-probe cross-task generalization. (c) Cosine similarity of probes across layer pairs.

Figure 2: Cosine similarity between no-prompt and ask-correct probes per layer.

Figure 3: Probes trained on no-prompt activations do not generalize consistently to ask-correct activations. Left: probes for arithmetic tasks are more robust than factual tasks to this template. Right: factual tasks show significant performance variations under the ask-correct instructions over layers.

Figure 4: Effect of model instructions on generalization of truth directions. Rows correspond to the task used for probe training and columns to the task used for evaluation. Probes trained and evaluated on activations from layer 25.

Figure 5: Two-dimensional projections of model activations extracted from layer 25 on the learned truth direction and on the direction of maximum variance in the orthogonal space.

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Dataset name	Total examples	Train / Test
F0	cities	1,594	1,116 / 478
F1	neg_cities	1,594	1,116 / 478
F2	cities_conj	1,594	1,116 / 478
F3	cities_same_country_quant	2,000	1,400 / 600
F4	cities_exact_k	2,000	1,400 / 600
F5	cities_exact_k1_k2	2,000	1,400 / 600

Appendix

Table 2: Dataset sizes and train/test splits. All datasets are balanced with equal numbers of true and false examples.

Model	$n_{\text{layers}}$	$n_{\text{parameters}}$	$d_{\text{model}}$
Llama-3.2-3B-Instruct	28	3.2B	3072
Llama-3.1-8B-Instruct	32	7.8B	4096
Gemma-2-2b-it	26	2.1B	2304
Gemma-2-9b-it	42	8.9B	3584

Appendix

Table 3: Model specifications used in our experiments.

Task	Truth dir.	Max-var dir.	Cumulative
A1	1.01%	32.56%	33.57%
A2	0.67%	35.95%	36.62%
A3	0.49%	28.17%	28.66%
F0	17.01%	29.25%	46.26%
F1	13.62%	8.51%	22.13%
F2	29.30%	18.46%	47.76%

Appendix

Table 4: Fraction of variance in model activations (layer 25) captured by the 2D projection subspace used in Figure 5 , spanned by the truth probe direction and the direction of maximum variance orthogonal to it.

Figure 6: Between- to within-class variance ratio

R^{(\ell)}

across layers for each task.

Figure 7: Fraction of truth-related variance explained by the polarity-invariant direction

t_{G}

and the polarity-dependent direction

t_{P}

across layers.

Figure 8: In-domain test-set AUROC per layer across all prompt templates.

Figure 10: Cross-task generalization for all five prompts at layer depth fractions 0.6, 0.7, 0.8 and 0.9 for Llama-3.1-8B-Instruct . Rows indicate training task, columns indicate test task.

Figure 11: In-domain test AUROC per layer for factual tasks of intermediate difficulty. The tasks of intermediate difficulty with varying list lengths are shown with emphasized lines.

Figure 12: Cosine similarity of truth directions probed on different tokens of ask-correct instruction template compared with no-prompt probes.

Figure 13: Test-set AUROC per layer for control prompts.

Panel of Figure 14 — Figure 14: Cosine similarities of control probes with no-prompt probes across layers.

Figure 15: Cross-task generalization of probes trained with no-prompt and the control prompts.

Figure 16: Activation projections for Llama-3.1-8B-Instruct .

Panel of Figure 17 — Figure 17: Layer dependence of truth directions for Llama-3.2-3B-Instruct . (a) In-domain test-set performance. (b) F0-probe cross-task generalization. (c) Cosine similarity of probes across layer pairs.

Figure 18: Cosine similarity of truth probes for Llama-3.2-3B-Instruct .

Figure 19: Cross-task generalization heatmaps for Llama-3.2-3B-Instruct .

Figure 20: Activation projections for Llama-3.2-3B-Instruct .

Panel of Figure 21 — Figure 21: Layer dependence of truth directions for Gemma-2-2b-it . (a) In-domain test-set performance. (b) F0-probe cross-task generalization. (c) Cosine similarity of probes across layer pairs.

Figure 22: Cosine similarity of truth probes for Gemma-2-2b-it .

Figure 23: Cross-task generalization heatmaps for Gemma-2-2b-it .

Figure 24: Activation projections for Gemma-2-2b-it .

Panel of Figure 25 — Figure 25: Layer dependence of truth directions for Gemma-2-9b-it . (a) In-domain test-set performance. (b) F0-probe cross-task generalization. (c) Cosine similarity of probes across layer pairs.

Figure 26: Cosine similarity of truth probes for Gemma-2-9b-it .

Figure 27: Cross-task generalization heatmaps for Gemma-2-9b-it .

Figure 28: Activation projections for Gemma-2-9b-it .

LLM Interpretability

Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.

See all 778 papers

The Truthfulness Spectrum Hypothesis
Feb 23, 2026Zhuofan Josh Ying, Shauli Ravfogel, Nikolaus Kriegeskorte +1LLM Honesty Deception in Language Models
Department of Psychology, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY · New York University, New York, NY · Department of Neuroscience, Columbia University, New York, NY +2
The Truth Stays in the Family: Enhancing Contextual Grounding via Inherited Truthful Heads in Model Lineages
Jun 14, 2026Miso Choi, Seonga Choi, Mincheol Kwon +3LLM Grounding LLM Interpretability
Korea University, Seoul, Republic of Korea · Kakao Mobility Corp., Seongnam, Republic of Korea
The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Shared Category Geometry in Small Language Models
Jul 18, 2026Francesco Karim VicidominiLLM Interpretability Factual Knowledge in Language Models
Independent researcher

Feb 23, 2026J/K move · Enter open · S save

The Truthfulness Spectrum Hypothesis

Zhuofan Josh Ying, Shauli Ravfogel, Nikolaus Kriegeskorte, Peter Hase

Department of Psychology, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY · New York University, New York, NY · Department of Neuroscience, Columbia University, New York, NY+2

LLM Honesty Deception in Language Models Language Model Probing

Open paper AskPDF arXiv

Large language models (LLMs) have been reported to linearly encode truthfulness, yet recent work questions this finding's generality. We reconcile these views with the truthfulness spectrum hypothesis: the representational space contains directions ranging from broadly domain-general to narrowly domain-specific. To test this hypothesis, we systematically evaluate probe generalization across five truth types (definitional, empirical, logical, fictional, and ethical), sycophantic and expectation-inverted lying, and existing honesty benchmarks. Linear probes generalize well across most domains but fail on sycophantic and expectation-inverted lying. Yet training on all domains jointly recovers strong performance, confirming that domain-general directions exist despite poor pairwise transfer. The geometry of probe directions explains these patterns: Mahalanobis cosine similarity between probes near-perfectly predicts cross-domain generalization (R^2=0.98). Concept-erasure methods further isolate truth directions that are (1) domain-general, (2) domain-specific, or (3) shared only across particular domain subsets. Causal interventions reveal that domain-specific directions steer more effectively than domain-general ones. Finally, post-training reshapes truth geometry, pushing sycophantic lying further from other truth types, suggesting a representational basis for chat models' sycophantic tendencies. Together, our results support the truthfulness spectrum hypothesis: truth directions of varying generality coexist in representational space, with post-training reshaping their geometry. Code for all experiments is provided in https://github.com/zfying/truth_spec.

Abstract

LLM Interpretability

Explore similar work

The Truthfulness Spectrum Hypothesis

The Truth Stays in the Family: Enhancing Contextual Grounding via Inherited Truthful Heads in Model Lineages

The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Shared Category Geometry in Small Language Models