Authors: Zhuofan Josh Ying, Shauli Ravfogel, Nikolaus Kriegeskorte, Peter Hase
Organizations: Department of Psychology, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY · New York University, New York, NY · Department of Neuroscience, Columbia University, New York, NY · Stanford University, Stanford, CA · Schmidt Sciences, New York, NY
Large language models (LLMs) have been reported to linearly encode truthfulness, yet recent work questions this finding's generality. We reconcile these views with the truthfulness spectrum hypothesis: the representational space contains directions ranging from broadly domain-general to narrowly domain-specific. To test this hypothesis, we systematically evaluate probe generalization across five truth types (definitional, empirical, logical, fictional, and ethical), sycophantic and expectation-inverted lying, and existing honesty benchmarks. Linear probes generalize well across most domains but fail on sycophantic and expectation-inverted lying. Yet training on all domains jointly recovers strong performance, confirming that domain-general directions exist despite poor pairwise transfer. The geometry of probe directions explains these patterns: Mahalanobis cosine similarity between probes near-perfectly predicts cross-domain generalization (R^2=0.98). Concept-erasure methods further isolate truth directions that are (1) domain-general, (2) domain-specific, or (3) shared only across particular domain subsets. Causal interventions reveal that domain-specific directions steer more effectively than domain-general ones. Finally, post-training reshapes truth geometry, pushing sycophantic lying further from other truth types, suggesting a representational basis for chat models' sycophantic tendencies. Together, our results support the truthfulness spectrum hypothesis: truth directions of varying generality coexist in representational space, with post-training reshaping their geometry. Code for all experiments is provided in https://github.com/zfying/truth_spec.
Figures & tables
Figure 1: Truth representations in LLMs are graded in generality and reshaped by post-training. Left: Different truth types share partially overlapping but distinct sets of truth directions. These directions lie on a spectrum from domain-general to domain-specific. The geometry of truth representations changes through post-training, pushing sycophancy into a more distant subspace from other truth types. This reorganization causes probes trained on factual truth to fail on sycophancy detection, and vice versa ( X ). However, training on all domains still yields a domain-general direction ( X ). Right: Concept erasure analysis further reveals the full spectrum of truth directions.
Figure 2: Probing Generalization Performance. We report the average AUROC for 5-fold cross-validation on Llama-3.3-70B . Probes trained on any one of our five truth types generalize to each other, but perform poorly on sycophantic and expectation-inverted lying. A probe trained on all domains generalizes well to all domains, performing on par with the best individual probe performance.
Figure 3: Mahalanobis cosine similarity linearly predicts OOD probe performance. Each point is a pair of datasets: the probe is trained on one and tested on the other. Mahalanobis cosine similarity achieves R2=0.98 , substantially outperforming standard cosine similarity ( R2=0.56 ; Figure 18 ).
Figure 4: Post-training reduces alignment between sycophantic lying and other truth types. (a,b) Base models show substantially better probe generalization between FLEED and sycophancy than chat models, indicating that post-training pushes sycophancy into a subspace more orthogonal to other truth types. (c) Probe direction similarity between FLEED and sycophancy is significantly higher in the base models compared to chat models. See similar results on Qwen family models in Appendix E and Figure 19 .
Figure 5: Stratified INLP Reveals Highly Domain-general and Domain-specific Directions. (a) Domain-general directions. Cross-domain accuracies for the first five mutually-orthogonal directions extracted by training on all domains jointly are high across all domains. (b) Domain-specific Directions. Accuracy for directions extracted from individual domains after the four domain-general directions have been projected out. While in-distribution accuracy (”Self”; blue) remains high, generalization to other domains (”Other”; yellow) drops toward chance ( 0.5 ; gray dashed line), indicating these directions encode truth information unique to a specific domain.
Figure 6: Performance of Selected Probes after LEACE Erasure. Each row shows one probe. Probes’ in-distribution performance is perfect ( green ), while performance on erased domains drops to chance ( red ). Probe generalization to other domains shows selective failure. From top to bottom, the probes get increasingly more domain-specific. For performances of all probes after LEACE erasure, see Figure 24 in Appendix F .
Figure 7: Effect of Causal Intervention Along Domain-General and Domain-Specific Directions Identified by Stratified INLP. We report the intervention effect ( α=−2 ) on Llama-8B across different levels of baseline P(correct)/P(incorrect), binned by percentile. Most domain-specific directions improve truthfulness, while domain-general direction hurts ( red rectangle). Larger effects are observed for samples where the model is initially more confident for all directions.
Appendix figures & tables22 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8
Truth Type
Negation Type
Original Claim
Negated Claim
Empirical
Direct Negation
Gravity is the force that keeps planets in orbit around the Sun.
Gravity is not the force that keeps planets in orbit around the Sun.
Quantity Change
Humans have 23 pairs of chromosomes.
Humans have 24 pairs of chromosomes.
Information Substitution
Paris is the capital of France.
Berlin is the capital of France.
Definitional
Direct Negation
An alphabet is a standard set of letters used to write a language.
An alphabet is not a standard set of letters used to write a language
Quantity Change
A triangle is a polygon with three sides and three angles.
A triangle is a polygon with four sides and four angles.
Information Substitution
Water is a chemical compound with the formula H 2 O.
Water is a chemical compound with the formula CO 2 .
Appendix
Table 1: Examples of claims and negations across different truth types and negation types.
Figure 8: PCA on Truth Representations Across Datasets. Scatter plot showing activations from Llama-70B layer 33 for honest statements (blue) and deceptive (red) samples across the four truth type datasets, sycophancy, Goldowsky-Dill et al. (2025) , and Marks & Tegmark (2023) . The intermixing of true and false points in the highest-variance directions demonstrates that our datasets are well-controlled, with truth directions encoded in lower-variance subspaces.
Figure 9: Construction of the Sycophancy Dataset. Left: We first extract the model’s true belief on MMLU STEM questions. Middle: A sycophantic example in which the user’s preferred answer differs from the model’s true belief, yet the model agrees with the user, contradicting its own belief. Right: An honest example in which the user’s preferred answer aligns with the model’s true belief.
Figure 10: Sycophancy Rate by Correctness and Confidence for Llama-70B . Left: Sycophancy rate across confidence percentiles, grouped by whether the model’s original answer is correct. The sycophancy rate is lower when the original answer is correct. Right: Individual responses plotted against model confidence (log scale), with the blue line indicating the mean sycophancy rate. Both panels show that higher model confidence is associated with lower sycophantic rate, regardless of answer correctness.
Figure 11: Probing Generalization Performances (sycophancy filtered based on correctness). Similar to Figure 2 , probes trained on the four truth types generalize to each other, but no prior probes generalize to sycophantic lying. Probe trained on combined domains effectively bridges gaps to the best individual probe performance for both ID and OOD.
Method
Training Data
Probe Type
Training Token
Goldowsky-Dill et al. (2025)
Empirical claims
Logistic Regression ( α =1)
All
Marks & Tegmark (2023)
Curated logical/empirical claims
Difference of Means
Last
Burns et al. (2023)
Contrast pairs
CCS (unsupervised)
Last
Ours
FLEED, Sycophancy, Exp Inverted, or Combined
Logistic Regression ( α = 10−4 )
Average
Appendix
Table 2: Comparison of Probe Designs for Truthfulness Detection.
Figure 12: Probe Design Tuning Process. (a) First, we fix the architecture to logistic regression and the token aggregation method to average token, and then pick the top-5 layers based on average cross-domain AUROC on our FLED dataset. (b) Second, we compute the same average AUROC for all combinations of architectures and token aggregation methods across these 5 layers. The final probe design in logistic regression with the average token.
Figure 13: Probing Layer Tuning. We compute the average cross-domain AUROC for all models (both base and chat models for Llama-70b , Llama-8b , Qwen-14b , and Qwen-7b ) across layers and select the best performing layer. The first column contains chat models, and the second contains base models. Note that the best layers of the base and the chat models of the same heritage are the same for all models tested. In addition, the best layers are some intermediate layers, and the base models’ performances drop more in later layers compared to the chat models.
Figure 14: Probe Generalization Performance on Llama-8B .
Figure 15: Probing Performance at the Best Layers for All Models. We use logistic regression with α=1 here. Note that the performances on the right (base models) are much higher than the ones on the left (chat models).
Figure 16: Mahalanobis cosine similarity linearly predicts OOD probe AUROC across various distributional assumptions. Each column corresponds to a synthetic generative model of increasing realism (left to right). Top row: Mahalanobis cosine similarity between ID and OOD probe weight vectors vs. OOD AUROC. Bottom row: standard cosine similarity vs. OOD AUROC. Mahalanobis cosine similarity is consistently linearly predictive of OOD AUROC ( R2≥0.957 ), while standard cosine similarity is not ( R2≤0.553 ), confirming that whitening by the test-set covariance is necessary to capture the geometrically meaningful notion of probe alignment.
Figure 17: Cross-domain probe transfer performance (left) compared with Mahalanobis cosine similarity (center) and standard cosine similarity (right) between probe directions. Mahalanobis cosine similarity closely tracks out-of-domain AUROC, capturing both high transfer among definitional, empirical, fictional, and logical domains and the weak transfer involving sycophancy and inverted-expertise probes. Standard cosine similarity, by contrast, fails to predict generalization performance well.
Figure 18: Standard Probe Cosine Similarity vs. Generalization Performance. Standard cosine similarity achieves an R2 of 0.56, which is much lower than the Mahalanobis variant ( R2=0.98 ; Figure 3 ).
Figure 19: Post-training reduces alignment between sycophancy and other truth types ( Qwen models ).
Figure 20: Sycophancy and FLEED Cross-Domain Probing Performance for All Models Across Layers. Base models (right) consistently outperform their chat model counterparts (left).
Figure 21: Full Stage 1 of Stratified INLP. We first remove 5 dimensions trained on all domains, then 3 for the honesty benchmarks, then 4 for our FLED datasets, and finally 10 for definitional, empirical, and fictional datasets.
Figure 22: Cross-generalization Performance of Domain-specific Directions Identified by Stratified INLP. Note that most directions only have high performance in-domain but are at chance for other domains.
Figure 23: Effect of LEACE on In-distribution performance. Targeted removal of a specific truth direction reduces the AUROC of that task to chance level (0.5). Crucially, this intervention does not degrade performance on other truth types (Non-Removed Tasks). This shows the existence of distinct, domain-specific directions, despite the ability of probes to generalize across them.
Figure 24: Effect of LEACE for All Probes.
Figure 25: Inferred capacities of intersecting latent truth subspaces. We formalize probe transfer and selective concept erasure as a capacity allocation problem over shared representation subspaces. The bottom matrix denotes subspace membership (black dots indicate a domain utilizes that direction), while the top bar chart displays the inferred capacity of each subspace derived via L1 -regularized least-squares optimization. For comparison, hypothetical pure domain-general (gold) and domain-specific (red) directions are manually appended. The empirical data reveals that representations predominantly rely on a patchwork of partially overlapping directions (blue) shared by 3 to 6 domains. This structure explains the asymmetric generalization and selective degradation observed during LEACE interventions: erasing a concept selectively destroys its specific intersecting capacities while leaving others intact.
Figure 26: Decomposition of log-probability changes for correct vs. incorrect answers. Effective domain-specific directions primarily suppress incorrect answers while preserving correct ones. The general direction instead boosts both, disproportionately increasing incorrect answer probability—explaining its failure despite targeting the same phenomenon.
Large language model honesty studies whether AI systems report what they know faithfully, express uncertainty, and resist deception or misleading compliance.
Large language models (LLMs) have been shown to encode truth of statements in their activation space along a linear truth direction. Previous studies have argued that these directions are universal in certain aspects, while more recent work has questioned this conclusion drawing on limited generalization across some settings. In this work, we identify a number of limits of truth-direction universality that have not been previously understood. We first show that truth directions are highly layer-dependent, and that a full understanding of universality requires probing at many layers in the model. We then show that truth directions depend heavily on task type, emerging in earlier layers for factual and later layers for reasoning tasks; they also vary in performance across levels of task complexity. Finally, we show that model instructions can affect truth directions; simple correctness evaluation instructions significantly affect the geometry and the generalization ability of the truth probes. Our findings indicate that universality claims for truth directions are more limited than previously known, with significant differences observable for various model layers, task difficulties, task types, and prompt templates.
Angelos Poulis, Mark Crovella, Evimaria Terzi
Department of Computer Science, Boston University · Faculty of Computing & Data Sciences, Boston University
Prior work has shown that LLMs encode the truth of factual propositions along linear directions in activation space. It's unclear how these representations extend to contextual truth: propositions whose truth is determined by in-context evidence rather than world knowledge. We show that LLMs maintain a linear representation of contextual truth that persists across structurally different output policies, even when the output doesn't require the model to determine a proposition's truth, and show causal evidence via steering experiments. Using the transcripts from a collaborative vision-language task that requires two LLMs to maintain a shared common ground, we show that truth representations of a proposition are significantly swayed by partner assertions about that proposition, even when the LLM has enough evidence to determine its truth. We find evidence that propositions near the decision boundary are more susceptible to having their truth shifted through partner assertions. Separating representation from output distinguish two forms of sycophancy that output behavior alone cannot: the model may accommodate a false proposition while continuing to represent it as false, or shift its representation across the boundary. The latter is 2.59x more common when the model agrees by restating the false claim explicitly than when it agrees implicitly.
Recent advances in large language models (LLMs) have produced many specialized multimodal LLMs (MLLMs) that share common foundational LLMs, forming distinct model lineages. It remains unclear whether a fundamental behavioral link exists between the foundational LLMs and downstream variants. We investigate this question by quantifying head-level context-truthfulness scores. Across diverse LLM and MLLM lineages, including Vicuna-, Qwen2.5-, LLaMA2-, and Mistral-based models, we find that Truth Scores are strongly preserved within model families, even after instruction tuning or multimodal adaptation. We further show that this inheritance is consistent with attention-head weight preservation, and that context-truthful heads attend to query-relevant evidence. Building on this finding, we propose TruthProbe, a soft-gating strategy that amplifies context-truthful heads while preserving other head contributions. TruthProbe improves contextual truthfulness on HaluEval and reduces multimodal hallucination on POPE and CHAIR, with base-LLM Truth Scores transferring effectively to their fine-tuned LLM and MLLM descendants. Code is available at https://github.com/miso-choi/TruthProbe.
Miso Choi, Seonga Choi, Mincheol Kwon +3
Korea University, Seoul, Republic of Korea · Kakao Mobility Corp., Seongnam, Republic of Korea