Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning

Hamed Damirchi; Ignacio Meza De la Jara; Ehsan Abbasnejad; Afshar Shamsi; Zhen Zhang; Javen Shi

cs.CLMar 1, 2026

Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning

Authors: Hamed Damirchi, Ignacio Meza De la Jara, Ehsan Abbasnejad, Afshar Shamsi, Zhen Zhang, Javen Shi

Organizations: Australian Institute for Machine Learning, Adelaide University · Monash University · Concordia University

Abstract

Existing explainability methods for Large Language Models (LLMs) typically treat hidden states as static points in activation space, assuming that correct and incorrect inferences can be separated using representations from an individual layer. However, these activations are saturated with polysemantic features, leading to linear probes learning surface-level lexical patterns rather than underlying reasoning structures. We introduce Truth as a Trajectory (TaT), which models the transformer inference as an unfolded trajectory of iterative refinements, shifting analysis from static activations to layer-wise geometric displacement. By analyzing displacement of representations across layers, TaT uncovers geometric invariants that distinguish valid reasoning from spurious behavior. We evaluate TaT across dense and Mixture-of-Experts (MoE) architectures on benchmarks spanning commonsense reasoning, question answering, and toxicity detection. Without access to the activations themselves and using only changes in activations across layers, we show that TaT effectively mitigates reliance on static lexical confounds, outperforming conventional probing, and establishes trajectory analysis as a complementary perspective on LLM explainability.

Figures & tables

Figure 1: Trajectories reveal structure beyond static embeddings. We plot layerwise hidden states as trajectories in activation space. Correct generations (green) follow smoother paths, while incorrect ones (red) exhibit sharp deviations. Although the supervised projection amplifies separation, the geometry suggests that modeling entire trajectories, rather than isolated states, can help distinguish valid from spurious reasoning.

Figure 2: Performance on 4 reasoning benchmarks using kinematic descriptors. The red dashed line is the random classifier. While activation velocity obtains better results than the base model, there is no consistency in this performance improvement across datasets.

Train Dataset	Method	Evaluation Dataset									Avg	ID Acc.	OOD Avg.
Train Dataset	Method	ARC-C	ARC-E	OpenQA	BoolQ	Hellaswag	CosQA	SiQA	ComQA	StoryCloze	Avg	ID Acc.	OOD Avg.
Zero-shot Accuracy		50.1	78.5	62.4	74.7	57.4	81.7 (26.1)	65.2 (48.0)	62.4	78.8	67.9 (59.8)	-	-
Few-shot Accuracy		65.3	84.3	67.0	83.8	76.8	82.0 (44.5)	67.7 (60.8)	71.1	83.1	75.7 (70.7)	-	-
ARC-C	Linear Probe	75.32	80.09	78.60	55.48	70.35	73.34	65.47	74.59	66.01	71.03	75.32	70.49
ARC-C	TaT (Ours)	82.17	85.31	73.60	96.91	73.89	74.24	75.49	72.48	82.58	79.63	82.17	79.31
ARC-E	Linear Probe	75.55	83.99	80.95	58.82	71.07	66.83	66.60	78.55	69.62	72.44	83.99	71.00

Table 1: Benchmark Accuracies across Evaluation Datasets with ID and OOD metrics. Values inside parentheses for base model accuracy indicate datasets where evaluation through normalized log-likelihood results in a degraded performance. While unfair to our approach (since this base model sees all possible answers, whereas our method does not), our approach still outperforms the base model itself in most cases.

Method	StoryCloze	OpenQA	ARC-E	ARC-C	BoolQ	Hellaswag
Linear Probe	69.62	80.95	83.99	75.55	58.82	71.07
TaT (Ours)	94.98	77.20	89.10	73.81	78.56	79.19
LoRA	83.76	52.40	85.98	61.26	79.97	75.86

Table 2: Generalization performance compared to low-rank adaptation (LoRA) when trained on ARC-E. TaT outperforms LoRA on 5 out of 6 transfer tasks.

	OOD Benchmark	RealTox Benchmarks
Method	ToxiGen	Standard	Challenging
Llama3.1-8b
Linear Probe	79.62	77.86	95.83
Trajectory (Raw)	81.99	82.16	97.91
TaT (Disp.)	84.23	79.35	96.00
Qwen2.5-14B

Table 3: Toxicity detection performance. Trajectory Disp. (TaT) uses layer-wise displacement, while Trajectory uses raw activations. TaT consistently achieves the best generalization on the OOD ToxiGen benchmark.

Train Dataset	Method	Avg	ID Acc.	OOD Avg.
ARC-C	Linear Probe	71.03	75.32	70.49
	TaT (Raw)	83.76	84.90	83.62
	TaT (Disp.)	79.63	82.17	79.31
ARC-E	Linear Probe	72.44	83.99	71.00
	TaT (Raw)	78.13	90.82	76.55
	TaT (Disp.)	77.76	89.10	76.34

Table 4: Comparison of Linear Probe, Raw Trajectories, and Displacement Trajectories (TaT) across reasoning benchmarks.

Train Dataset	Method	Avg	ID Acc.	OOD Avg.
ARC-C	Linear Probe	71.03	75.32	70.49
	TaT-Mid Layer	70.03	66.98	70.41
	TaT-Final Token	73.66	73.38	73.68
	TaT	79.63	82.17	79.31
ARC-E	Linear Probe	72.44	83.99	71.00
	TaT-Mid Layer	72.51	86.28	70.78

Table 5: Trajectory Grid Ablations (Rows vs. Columns). TaT-Mid Layer restricts the trajectory to a single layer, while TaT-Final Token restricts it to the final token across all layers.

Train Dataset	Method	Avg	ID Acc.	OOD Avg.
ARC-C	Linear Probe	71.03	75.32	70.49
	TaT (Disp.)	79.63	82.17	79.31
	Set MLP	72.67	68.65	73.17
ARC-E	Linear Probe	72.44	83.99	71.00
	TaT (Disp.)	77.76	89.10	76.34
	Set MLP	74.65	89.90	72.75

Table 6: Comparison of TaT with an order-invariant Set MLP baseline.

Metric	LLaMA 3.1-8B (fp16)	LSTM Classifier	Overhead
Parameters	8.0B	4.76M	0.06%
Inference Time (ms)	64.0	10.5	16% ∗
Model Memory (MB)	$\sim$ 15,000	18.1	0.12%

Table 7: Computational overhead of the LSTM classifier compared to the base LLaMA 3.1-8B model. ∗ Inference overhead was computed in the simplest case of extracting all activations from all tokens across all layers and then passing them through the LSTM model separately. However, in a realistic deployment scenario, the sequential classifier would be embedded within each layer of the model and would cause a negligible amount of inference overhead.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Top performance of Qwen2.5-14b on 4 reasoning benchmarks using kinematic descriptors with varying rule-sets. Red dashed line represents the random classifier accuracy. While the velocity of activations obtains better results than the base model itself, there is no consistency in this performance improvement across datasets despite the oracle-guided approach to evaluation.

Train Dataset	Method	Evaluation Dataset				Avg	ID Acc.	OOD Avg.
Train Dataset	Method	StoryCloze	OpenQA	ARC-E	ARC-C	Avg	ID Acc.	OOD Avg.
Zero-shot Accuracy		80.6	61.8	83.5	58.2	71.0	-	-
Few-shot Accuracy		84.4	62.0	88.3	74.0	77.2	-	-
OpenQA	Linear Probe	52.86	80.31	73.02	72.02	69.55	80.31	65.97
OpenQA	Trajectory	82.34	88.20	86.70	78.92	84.04	88.20	82.65
ARC-C	Linear Probe	69.94	75.48	78.81	75.60	74.96	75.60	74.74

Appendix

Table 8: Generalization on Qwen2.5-14b for TaT vs. Probing

Task	Problem (prompt/context)	Completion(s) / label
ARC-Challenge	An astronomer observes that a planet rotates faster after a meteorite impact. Which is the most likely effect of this increase in rotation?	Correct: Planetary days will become shorter.
OpenBookQA	Fact: using less resources usually causes money to be saved	Correct: quit eating lunch out
	Question: A person wants to start saving money so that they can afford a nice vacation at the end of the year. After looking over their budget and expenses, they decide the best way to save money is to

SocialIQA	Tracy didn’t go home that evening and resisted Riley’s attacks.	Correct: Find somewhere to go
	What does Tracy need to do before this?

Appendix

Table 9: Concrete problem/completion examples used for feature extraction and evaluation.

Task	Few-shot count	Notes
ARC-E	5	-
ARC-C	25	-
OpenQA	5	Facts/Context included for each instance.
BoolQ	5	Reading comprehension tasks aren’t commonly evaluated in few-shot scenarios, but we use 5 shots only to demonstrate the performance against TaT, which is always zero-shot.
ComQA	5	-
CosQA	5	-

Appendix

Table 10: Few-shot counts used in final baseline evaluations (selected tasks).

Layer Index	-1	-2	-3	-4	-5	-6	-7	-8	-9	-10	-11	-12	-13	-14	-15	-16
ARC-C	0.6651	0.6865	0.6733	0.6629	0.6780	0.6765	0.6907	0.6943	0.7024	0.6884	0.6884	0.6965	0.7029	0.7098	0.7188	0.7103
ARC-E	0.6970	0.7213	0.6877	0.6962	0.7120	0.7242	0.7301	0.6997	0.7223	0.7142	0.6975	0.6822	0.7384	0.7108	0.7196	0.7244
OpenQA	0.6201	0.5598	0.5667	0.6056	0.6471	0.6010	0.6301	0.6291	0.6368	0.6641	0.5780	0.6351	0.6397	0.6408	0.6924	0.6711

Appendix

Table 11: Probe Layer Sensitivity. Accuracy of linear probes trained on different layers (indexed from the last layer, -1). Bold values indicate the top performing layers for each dataset.

LLM Interpretability

Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.

See all 758 papers

TRUE: A Trustworthy Unified Explanation Framework for Large Language Model Reasoning
Feb 21, 2026Yujiao YangLLM Interpretability Faithfulness of Language Model Explanations
Dalian University of Technology, Dalian, China
Hidden Thoughts Are Not Secret: Reasoning Trace Exposure in LLMs
May 30, 2026Yu-An Lu, Ci-Yang Tsai, Yu-Lin Tsai +2LLM Prompting Privacy Leakage in Language Models
National Yang Ming Chiao Tung University · UC Berkeley
Confidence Geometry Reveals Trace-Level Correctness in Large Language Model Reasoning
May 16, 2026Shuo Liu, Ding Liu, Shi-Ju RanConfidence Estimation in Language Models LLM Reliability
School of Computer Science and Technology, Tiangong University, Tianjin, China. · Center for Quantum Physics and Intelligent Sciences, Department of Physics, Capital Normal University, Beijing, China.

Feb 21, 2026J/K move · Enter open · S save

TRUE: A Trustworthy Unified Explanation Framework for Large Language Model Reasoning

Yujiao Yang

Dalian University of Technology, Dalian, China

LLM Interpretability Faithfulness of Language Model Explanations LLM Reliability Reasoning Consistency in Language Models

Open paper AskPDF arXiv

Large language models (LLMs) have demonstrated strong capabilities in complex reasoning tasks, yet their decision-making processes remain difficult to interpret. Existing explanation methods often lack trustworthy structural insight and are limited to single-instance analysis, failing to reveal reasoning stability and systematic failure mechanisms. To address these limitations, we propose the Trustworthy Unified Explanation Framework (TRUE), which integrates executable reasoning verification, feasible-region directed acyclic graph (DAG) modeling, and causal failure mode analysis. At the instance level, we redefine reasoning traces as executable process specifications and introduce blind execution verification to assess operational validity. At the local structural level, we construct feasible-region DAGs via structure-consistent perturbations, enabling explicit characterization of reasoning stability and the executable region in the local input space. At the class level, we introduce a causal failure mode analysis method that identifies recurring structural failure patterns and quantifies their causal influence using Shapley values. Extensive experiments across multiple reasoning benchmarks demonstrate that the proposed framework provides multi-level, verifiable explanations, including executable reasoning structures for individual instances, feasible-region representations for neighboring inputs, and interpretable failure modes with quantified importance at the class level. These results establish a unified and principled paradigm for improving the interpretability and reliability of LLM reasoning systems.

Abstract

LLM Interpretability

Explore similar work

TRUE: A Trustworthy Unified Explanation Framework for Large Language Model Reasoning

Hidden Thoughts Are Not Secret: Reasoning Trace Exposure in LLMs

Confidence Geometry Reveals Trace-Level Correctness in Large Language Model Reasoning