Spilled Energy in Large Language Models

Adrian Robert Minut; Hazem Dewidar; Iacopo Masi

cs.AIFeb 21, 2026

Spilled Energy in Large Language Models

Authors: Adrian Robert Minut, Hazem Dewidar, Iacopo Masi

Organizations: Sapienza University of Rome, Italy · OmnAI Lab GLADIA

Abstract

We reinterpret the final Large Language Model (LLM) softmax classifier as an Energy-Based Model (EBM), decomposing the sequence-to-sequence probability chain into multiple interacting EBMs at inference. This principled approach allows us to track "energy spills" during decoding, which we empirically show correlate with factual errors, biases, and failures. Similar to Orgad et al. (2025), our method localizes the exact answer token and subsequently tests for hallucinations. Crucially, however, we achieve this without requiring trained probe classifiers or activation ablations. Instead, we introduce two completely training-free metrics derived directly from output logits: spilled energy, which captures the discrepancy between energy values across consecutive generation steps that should theoretically match, and marginalized energy, which is measurable at a single step. Evaluated on nine benchmarks across state-of-the-art LLMs (including LLaMA, Mistral, and Gemma) and on synthetic algebraic operations (Qwen3), our approach demonstrates robust, competitive hallucination detection and cross-task generalization. Notably, these results hold for both pretrained and instruction-tuned variants without introducing any training overhead. Code available at: github.com/OmnAI-Lab/spilled-energy

Figures & tables

Panel of Figure 1 — Figure 1: Color-coded comparison of hallucination detection with LLaMa-3 8B using logit confidence and spilled energy . Our method generalizes well across topics (e.g., Q&A, reasoning) and diverse LLMs. ✓ indicates a correct answer and ✗ an incorrect one. While our approach focuses on the exact answer tokens (e.g. Rome/Sydney and 120/470, see Section 4.2 ), here we apply min–max normalization to the full answer for visualization, as truthful hallucination.

Figure 2: How energy spills in LLMs . (a) Language Modeling $p(\mathbf{x}_{i:1})$ is attained as a decomposition problem following the chain rule of probability, implemented as autoregressive: we recursively apply a discriminative classifier over the vocabulary $\mathcal{V}$ to attain generative modeling with larger context size i.e. $p(\mathbf{x}_{i}|\mathbf{x}_{i-1:1})$ . (b) We reinterpret each discriminative classifier as a generative EBM, finding a connection between two quantities that should be the same across time steps yet are different. We call this difference “the spilled energy” $\Delta E_{\bm{\theta}}(\mathbf{x}_{i:1})$ in Eq. 8 . (c) Given that we simply read values inside the LLM, our approach is training-free and correlates well with hallucinations on a synthetic math dataset with increasing difficulty; (d) histograms of spilled energy values, for incorrect and correct answers on all nine datasets using $\min$ pooling for Llama-3-Instruct. The two distributions are easily separable by using a simple threshold, resulting in a generalization across real-world tasks.

Panel of Figure 3 — Figure 3: Histograms of Spilled Energy values across models (rows) on Math Sums with different error ranges in the answer (columns, decreasing range left to right, making it harder to detect errors). All sums are performed on 13-digit integers. In the fourth column, we show ROC curves for Hallucination Detection across the error ranges (colors) and methods (line styles).

Panel of Figure 4 — Figure 4: (a) AuROC performance as percentages of probing classifiers on exact answer tokens by Orgad et al. for LlaMA-3-Instruct. (b) depicts the performance difference between our Spilled $\Delta E$ with Min pooling and theirs. Positive values indicate cases where Spilled $\Delta E$ outperforms Orgad et al. . This comparison highlights the generalization capabilities of our method, compared to probing classifiers. Legend: low performance high performance.

	Pool	HotpotQA	HotpotQA-WC	IMDB	Math	MNLI	Movies	TriviaQA	Winobias	Winogrande	Average
	LLaMA-Instruct Dubey et al. (2024)
$p(\text{true})$	—	58.31 $\pm 0.32$	51.66 $\pm 1.05$	50.72 $\pm 1.20$	49.53 $\pm 2.16$	52.33 $\pm 0.98$	59.30 $\pm 0.85$	45.99 $\pm 0.51$	45.47 $\pm 1.58$	48.33 $\pm 0.68$	51.29 $\pm 04.86$
Orgad et al.	Mean	66.56 $\pm 9.10$	59.00 $\pm 8.14$	69.78 $\pm 14.76$	66.56 $\pm 17.04$	60.56 $\pm 12.53$	66.44 $\pm 8.06$	63.22 $\pm 11.11$	67.33 $\pm 11.97$	58.00 $\pm 7.79$	64.16 $\pm 03.90$
Logit $E^{\ell}$	Max	72.85 $\pm 2.12$	91.11 $\pm 1.52$	42.08 $\pm 5.07$	57.81 $\pm 3.82$	25.52 $\pm 3.00$	43.97 $\pm 1.38$	68.89 $\pm 1.96$	39.95 $\pm 2.41$	49.40 $\pm 2.16$	54.62 $\pm 18.97$
Marginal $E^{m}$	Max	76.72 $\pm 1.38$	30.74 $\pm 3.45$	85.63 $\pm 2.39$	27.08 $\pm 5.06$	89.90 $\pm 1.25$	96.17 $\pm 0.63$	80.13 $\pm 1.87$	57.67 $\pm 2.94$	47.47 $\pm 1.83$	65.72 $\pm 24.39$
Marginal $E^{m}$	Min	75.91 $\pm 1.62$	97.57 $\pm 0.75$	14.37 $\pm 2.39$	70.55 $\pm 2.43$	61.21 $\pm 3.24$	72.21 $\pm 1.60$	73.38 $\pm 1.86$	47.19 $\pm 2.71$	53.98 $\pm 2.30$	62.93 $\pm 21.89$

Table 1: Hallucination detection performance, in terms of AuROC, across nine benchmarks and four different LLMs. We measure the generalization across all tasks by computing the average.

	Pool	Average %	Exact
		w/ exact	answer
		answer	increase
Logit $E^{\ell}$	Max	56.12	+9.23
Orgad et al.	Mean	63.67	–
Marginal $E^{m}$	Min	67.23	+20,02
Marginal $E^{m}$	Max	63.34	+3,62

Table 2: Improvements in AuROC with the exact answer. Average across 4 LLMs and 9 benchmarks.

Logit $E^{\ell}$	Max	50.09 $\pm 0.45$	60.88 $\pm 3.96$	53.95 $\pm 2.10$	49.77 $\pm 0.15$	54.43 $\pm 2.80$	27.00 $\pm 2.16$	78.64 $\pm 3.47$	62.84 $\pm 1.97$	64.49 $\pm 2.02$	55.79 $\pm 13.24$
	Pool	IMBD	Movies	TriviaQA	Winogrande	Winobias	MNLI	Math	HotpotQA	HotpotQA-WC	Average
	Gemma-Instruct 4B Kamath et al. (2025)
Marginal $E^{m}$	Max	49.14 $\pm 2.70$	83.02 $\pm 1.56$	84.14 $\pm 1.39$	51.49 $\pm 1.97$	47.97 $\pm 1.80$	100.00 $\pm 0.00$	74.57 $\pm 3.60$	83.70 $\pm 0.77$	85.95 $\pm 2.03$	73.33 $\pm 17.94$
Marginal $E^{m}$	Min	50.86 $\pm 2.70$	51.29 $\pm 3.30$	55.33 $\pm 1.80$	48.12 $\pm 1.89$	51.91 $\pm 2.10$	99.01 $\pm 0.50$	76.03 $\pm 3.27$	62.59 $\pm 1.49$	71.84 $\pm 2.72$	63.00 $\pm 15.75$
Spilled $\Delta E_{s}$	Max	50.89 $\pm 1.65$	50.77 $\pm 5.72$	56.08 $\pm 2.48$	50.59 $\pm 1.72$	53.53 $\pm 2.81$	95.61 $\pm 0.56$	43.94 $\pm 3.21$	50.87 $\pm 1.87$	51.21 $\pm 1.68$	55.94 $\pm 14.35$
Spilled $\Delta E$	Min	50.89 $\pm 1.65$	86.13 $\pm 4.28$	89.01 $\pm 1.06$	50.18 $\pm 1.97$	53.10 $\pm 3.05$	99.66 $\pm 0.21$	82.29 $\pm 2.46$	89.10 $\pm 1.75$	8 2.70 $\pm 1.35$	75.89 $\pm 17.98$

Table 3: Hallucination detection performance on the Gemma Model Instruct for different parameters of the model, 1B and 4B.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Success Rate (%)
TriviaQA	90.29
HotpotQA	87.37
Movies	93.61
MNLI	92.99
Math	87.59
HotpotQA-WC	92.38

Appendix

Table 4: Answer Extraction Success Rate across tasks for Mistral-Instruct.

Panel of Figure 5 — Figure 5: Histograms of Spilled Energy values across models (rows) on Math Sums with different error ranges in the answer (columns, decreasing range left to right, making it harder to detect errors), as described in Section 5.1 . In the fourth column, we show ROC curves for Hallucination Detection across the error ranges (colors) and methods (line styles).

Panel of Figure 6 — Figure 6: Histograms of Spilled Energy values for Correct and Incorrect answers across models on Math Sums, increasing difficulty from left to right. We compute sums on 13-digit integers, for incorrect answers we add a random offset sampled uniformly from the error interval: Easy $\sim\mathcal{U}(1e3,1e4)$ - Medium $\sim\mathcal{U}(1e2,1e3)$ - Hard $\sim\mathcal{U}(1,10)$ ; for more details see Section 5.1 .

Panel of Figure 7 — Figure 7: ROC curves for Hallucination Detection across models (rows) on Math Sums with different error ranges in the answer (columns, decreasing range left to right). All sums are performed on 13-digit integers. Legend: Spilled (ours) Spilled $\Delta E$ Logit $E^{\ell}$ Marginal $E^{m}$

	Pool	HotpotQA	HotpotQA-WC	IMDB	Math	MNLI	Movies	TriviaQA	Winobias	Winogrande	Average
	LLaMA-Instruct
Orgad et al. (2025)	Mean	66.56 $\pm 9.10$	59.00 $\pm 8.14$	69.78 $\pm 14.76$	66.56 $\pm 17.04$	60.56 $\pm 12.53$	66.44 $\pm 8.06$	63.22 $\pm 11.11$	67.33 $\pm 11.97$	58.00 $\pm 7.79$	64.16 $\pm 3.90$
Spilled $\Delta E$	Min	85.98 $\pm 1.09$	93.00 $\pm 1.61$	47.66 $\pm 4.06$	65.58 $\pm 3.02$	73.95 $\pm 1.97$	89.34 $\pm 1.04$	87.07 $\pm 1.33$	60.72 $\pm 2.74$	55.11 $\pm 2.05$	73.16 $\pm 15.64$
Marginal $E^{m}$	Max	76.72 $\pm 1.38$	30.74 $\pm 3.45$	85.63 $\pm 2.39$	27.08 $\pm 5.06$	89.90 $\pm 1.25$	96.17 $\pm 0.63$	80.13 $\pm 1.87$	57.67 $\pm 2.94$	47.47 $\pm 1.83$	65.72 $\pm 24.39$
Marginal $E^{m}$	Min	75.91 $\pm 1.62$	97.57 $\pm 0.75$	14.37 $\pm 2.39$	70.55 $\pm 2.43$	61.21 $\pm 3.24$	72.21 $\pm 1.60$	73.38 $\pm 1.86$	47.19 $\pm 2.71$	53.98 $\pm 2.30$	62.93 $\pm 21.89$
Logit $E^{\ell}$	Max	72.85 $\pm 2.12$	91.11 $\pm 1.52$	42.08 $\pm 5.07$	57.81 $\pm 3.82$	25.52 $\pm 3.00$	43.97 $\pm 1.38$	68.89 $\pm 1.96$	39.95 $\pm 2.41$	49.40 $\pm 2.16$	54.62 $\pm 18.97$

Appendix

Table 5: Hallucination detection performance, in terms of AuROC, across nine benchmarks and different LLMs. We measure the generalization across all tasks by computing the average.

Panel of Figure 8 — Figure 8: Fig. 8(a) presents the cross-dataset performance of the method proposed by Orgad et al. (2025) using Llama-3. Fig. 8(b) depicts the performance difference between their method and our Spilled $\Delta E$ with Min pooling. Positive values indicate cases where Spilled $\Delta E$ outperforms the method of Orgad et al. (2025) . All numbers are computed as percentages.

Panel of Figure 9 — Figure 9: Fig. 9(a) presents the cross-dataset performance of the method proposed by Orgad et al. (2025) using Mistral. Fig. 9(b) depicts the performance difference between their method and our Spilled $\Delta E$ with Min pooling. Positive values indicate cases where Spilled $\Delta E$ outperforms the method of Orgad et al. (2025) . All numbers are computed as percentages.

Panel of Figure 10 — Figure 10: Fig. 10(a) presents the cross-dataset performance of the method proposed by Orgad et al. (2025) using Mistral-Instruct. Fig. 10(b) depicts the performance difference between their method and our Spilled $\Delta E$ with Min pooling. Positive values indicate cases where Spilled $\Delta E$ outperforms the method of Orgad et al. (2025) . All numbers are computed as percentages.

Figure 11: Example of the Pooling Window

LLM Reliability

Large language model reliability studies whether outputs remain correct, calibrated, robust, and controllable across prompts, contexts, and deployment conditions.

See all 787 papers

Apr 6, 2026

HalluSAE: Detecting Hallucinations in Large Language Models via Sparse Auto-Encoders

Large Language Models (LLMs) are powerful and widely adopted, but their practical impact is limited by the well-known hallucination phenomenon. While recent hallucination detection methods have made notable progress, we find most of them overlook the dynamic nature and underlying mechanisms of it. To address this gap, we propose HalluSAE, a phase transition-inspired framework that models hallucination as a critical shift in the model's latent dynamics. By modeling the generation process as a trajectory through a potential energy landscape, HalluSAE identifies critical transition zones and attributes factual errors to specific high-energy sparse features. Our approach consists of three stages: (1) Potential Energy Empowered Phase Zone Localization via sparse autoencoders and a geometric potential energy metric; (2) Hallucination-related Sparse Feature Attribution using contrastive logit attribution; and (3) Probing-based Causal Hallucination Detection through linear probes on disentangled features. Extensive experiments on Gemma-2-9B demonstrate that HalluSAE achieves state-of-the-art hallucination detection performance.

Boshui Chen, Zhaoxin Fan, Ke Wang +5

Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University, Beijing, China. · State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China. · Beijing Academy of Blockchain and Edge Computing, Beijing, China. +1

LLM Hallucination Detection

Ask PDF

May 27, 2026

Entropy Distribution as a Fingerprint for Hallucinations in Generative Models

Large Language Models (LLMs) often generate factually incorrect outputs, commonly termed hallucinations, that undermine trust and limit deployment in high-stakes settings. Existing hallucination detection methods typically require multiple forward passes, or access to model internals. In this work, we provide theoretical background and empirical evidence that the distribution of token-level entropies, beyond the mean captured by perplexity or length-normalised entropy, serves as a fingerprint of hallucination, with distributional shape and tail behaviour carrying independent signal. We formalize hallucination detection as a statistical hypothesis test and propose the Calibrated Entropy Score (CES), a lightweight algorithm requiring only a single forward pass and black-box access to token logits. CES combines the mean signal with the maximum signal of the generated entropy through a calibrated reference CDF, producing scores that are directly comparable across models and tasks. We establish finite-sample calibration guarantees via a novel random-length Dvoretzky--Kiefer--Wolfowitz inequality, and also prove that CES detects hallucinations with probability converging to one exponentially fast in the generation length. Across eight QA benchmarks and ten generator models spanning open-source and API access models, CES achieves the highest detection performance among all single-pass black-box methods while providing formal error guarantees that existing heuristics lack. Remarkably, CES is statistically indistinguishable from multi-sample methods that require far greater computational cost, closing the gap between lightweight and expensive detection and making it suitable for real-time, large-scale deployment.

Mattia J. Villani, Pranav Deshpande, Akshay Seshadri +2

Global Technology Applied Research, JPMorganChase, New York, NY 10001, USA

Language Model Error Detection Language Model Generation Evaluation Hallucination Detection Hallucination in Language Models LLM Reliability LLM Hallucination Detection+1

Ask PDF

Feb 2, 2026

HALT: Hallucination Assessment via Log-probs as Time series

Hallucinations remain a major obstacle for large language models (LLMs), especially in safety-critical domains. We present HALT (Hallucination Assessment via Log-probs as Time series), a lightweight hallucination detector that leverages only the top-20 token log-probabilities from LLM generations as a time series. HALT uses a gated recurrent unit model combined with entropy-based features to learn model calibration bias, providing an extremely efficient alternative to large encoders. Unlike white-box approaches, HALT does not require access to hidden states or attention maps, relying only on output log-probabilities. Unlike black-box approaches, it operates on log-probs rather than surface-form text, which enables stronger domain generalization and compatibility with proprietary LLMs without requiring access to internal weights. To benchmark performance, we introduce HUB (Hallucination detection Unified Benchmark), which consolidates prior datasets into ten capabilities covering both reasoning tasks (Algorithmic, Commonsense, Mathematical, Symbolic, Code Generation) and general purpose skills (Chat, Data-to-Text, Question Answering, Summarization, World Knowledge). While being 30x smaller, HALT outperforms Lettuce, a fine-tuned modernBERT-base encoder, achieving a 60x speedup gain on HUB. HALT and HUB together establish an effective framework for hallucination detection across diverse LLM capabilities.

Ahmad Shapiro, Karan Taneja, Ashok Goel

Georgia Institute of Technology · Microsoft

Language Model Error Detection Hallucination Detection Hallucination in Language Models LLM Reliability LLM Hallucination Detection

Ask PDF

Abstract

LLM Reliability

Explore similar work

HalluSAE: Detecting Hallucinations in Large Language Models via Sparse Auto-Encoders

Entropy Distribution as a Fingerprint for Hallucinations in Generative Models

HALT: Hallucination Assessment via Log-probs as Time series