cs.AIApr 4, 2026

Entropy and Attention Dynamics in Small Language Models: A Trace-Level Structural Analysis on the TruthfulQA Benchmark

Authors: Adeyemi Adeseye, Aisvarya Adeseye, Hannu Tenhunen, Jouni Isoaho

Organizations: Brilloconnetz Partners avoin yhtiö, Turku, Finland · University of Turku, Turku, Finland · Royal Institute of Technology, Stockholm, Sweden

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Small language models (SLMs) have been increasingly deployed in edge devices and other resource-constrained settings. However, these models make confident mispredictions and produce unstable output, making them risky for factual and decision-critical tasks. Current evaluation methodology relies on final accuracy or hallucination rates without explaining how internal model behavior affects outputs. Specifically, how entropy evolves during decoding, how attention is distributed across layers, and how hidden representations contribute to uncertainty, logical inconsistencies, and misinformation propagation are often overlooked. Consequently, this study introduces a trace-level analysis of entropy and attention dynamics in SLMs evaluated with the TruthfulQA dataset. Four models with parameter ranges of 1B-1.7B parameters were examined via token-level output entropy, attention entropy, head dispersion, and hidden-state representation. The results reflect three model classifications by entropy patterns. Deterministic models (DeepSeek-1.5B and LLaMA-1B): output entropy decreases over time. Exploratory models (Gemma-1B): with increasing entropy, and balanced models (Qwen-1.7B): have moderate and stable entropy. Also, each group has distinctively different hidden-state movement and attention dispersion patterns. The analysis demonstrates that truthfulness in SLMs emerges from structured entropy and attention dynamics. Monitoring and optimizing these internal uncertainty patterns can guide the design of a more reliable, hallucination-aware, and application-specific edge SLMs.

Figures & tables

Lowest Entropy LayersHighest Entropy Layers
LayerEntropyHDILayerEntropyHDI
331.0151.0150.8210.821112.7872.7871.0991.099
221.1621.1620.9220.922882.7412.7410.9320.932
14141.1711.1710.7660.766992.6382.6380.8820.882
13131.3521.3520.7080.708772.3802.3800.8810.881
16161.3781.3780.7280.72810102.3212.3210.9890.989
Table 6: Comparison of Lowest and Highest Entropy Layers with Corresponding HDI for LLaMA-1B
ModelHidden L2 (Mean ±\pm SD)Delta L2 from Prev (Mean ±\pm SD)
LLaMA-1B13.13213.132 0.3629.7359.735 0.183
Gemma-1B6419.7096419.709 738.5241600.6491600.649 148.048
DeepSeek-1.5B137.372137.372 6.27767.79867.798 2.762
Qwen-1.7B567.026567.026 41.272303.196303.196 31.500
Table 7: Hidden Representation Magnitude and Drift
Figure 1: Component Trends Across Outcome Types (Per Model)
Figure 2: Entrophy Drift
Figure 3: Relationships Between Output Entropy, Top-1 Confidence, KV Memory, Representational Drift, and Attention Dispersion Across Small Language Models
Figure 4: Correlation heatmap of entropy, confidence, attention dispersion, representation drift, and memory metrics.
More in this topic

Small Language Models

Small language models are compact neural language systems designed to deliver useful text and reasoning capabilities with limited compute, memory, and energy.

See all 277 papers

Explore similar work

CardsList
Aug 12, 2026

Benchmarking Trustworthiness of SLMs: Pre-trained vs. Compressed

Small Language Models (SLMs) have emerged as a more efficient alternative to traditional Large Language Models (LLMs), offering promising potential in resource-constrained scenarios. Existing approaches to building SLMs typically follow two paths: training compact models from scratch, or compressing larger pre-trained models using methods such as pruning, quantization, or distillation. As language models become increasingly integrated into real-world applications, ensuring their trustworthiness has become a critical concern. However, how to build trustworthy SLMs remains an underexplored question. In this work, we present a comprehensive evaluation of SLM trustworthiness across multiple dimensions, including fairness, robustness, privacy, and ethics. We first examine the effects of pruning and quantization, and find that quantization is significantly more effective in preserving trustworthiness compared to pruning. More importantly, we demonstrate that compressing a reliable large model via quantization can produce SLMs with superior trustworthiness and adaptability compared to using small models trained from scratch. Furthermore, knowledge distillation from trustworthy teacher models can further enhance the reliability of SLMs. We hope our findings provide practical guidance and a foundation for future research into the development and deployment of trustworthy small language models.

Haokun Lin, Kaijie Zhu, Haobo Xu +4

Institute of Automation, CAS Beijing, China · Tsinghua University Beijing, China · Harvard Medical School Boston, U.S. +1

LLM QuantizationLLM PruningLLM CompressionSmall Language ModelsLanguage Model Distillation
AskPDF
Mar 28, 2026

GUARD-SLM: Token Activation-Based Defense Against Jailbreak Attacks for Small Language Models

Small Language Models (SLMs) are emerging as efficient and economically viable alternatives to Large Language Models (LLMs), offering competitive performance with significantly lower computational costs and latency. These advantages make SLMs suitable for resource-constrained and efficient deployment on edge devices. However, existing jailbreak defenses show limited robustness against heterogeneous attacks, largely due to an incomplete understanding of the internal representations across different layers of language models that facilitate jailbreak behaviors. In this paper, we conduct a comprehensive empirical study on 9 jailbreak attacks across 7 SLMs and 3 LLMs. Our analysis shows that SLMs remain highly vulnerable to malicious prompts that bypass safety alignment. We analyze hidden-layer activations across different layers and model architectures, revealing that different input types form distinguishable patterns in the internal representation space. Based on this observation, we propose GUARD-SLM, a lightweight token activation-based method that operates in the representation space to filter malicious prompts during inference while preserving benign ones. Our findings highlight robustness limitations across layers of language models and provide a practical direction for secure small language model deployment.

Md Jueal Mia, Joaquin Molto, Yanzhao Wu +1

Knight Foundation School of Computing and Information Sciences Florida International University Miami, FL, USA · Security, Optimization, and Learning for InterDependent networks laboratory (solid lab) Florida International University Miami, FL, USA

Jailbreak DefenseJailbreak DetectionLLM Jailbreak AttacksSmall Language ModelsLLM Safety
AskPDF
May 30, 2026

Hallucination Is Linearly Decodable from Mid-Layer Hidden States in Quantized LLMs

We investigate whether open-source LLMs encode a linearly separable truthfulness signal in their hidden states, and at which network depth this signal is strongest. Across three 77B--88B instruction-tuned models (Llama-3.1-8B, Mistral-7B, Qwen2.5-7B) loaded in 44-bit NF4 quantization, we extract per-layer hidden states on four hallucination benchmarks (TruthfulQA, HaluEval-QA, FEVER, and a controlled synthetic set) and compare four detection approaches: linear and MLP probes, INSIDE EigenScore, self-consistency, and attention entropy. A linear probe on a single mid-network layer achieves 0.9040.904--1.0001.000 AUROC on held-out splits, while sampling-based detectors do not exceed 0.5410.541 AUROC under the same protocol. The truthfulness signal is approximately linear: MLP probes rarely surpass linear probes by more than 0.010.01 AUROC. Peak probing layers fall in a consistent band across model families on natural-language benchmarks -- blocks~1313--1818 of~3232 for Llama and Mistral, and blocks~1919--2525 of~2828 for Qwen. First-block attention entropy provides a complementary signal in knowledge-grounded settings (0.8660.866--0.9410.941 AUROC on HaluEval-QA) at no additional inference cost. The low discriminability of sampling methods under this protocol reflects a structural mismatch between paired-label evaluation and the information these methods access, rather than an inherent limitation of those methods. Code and data are released for full reproducibility on a single 88,GB GPU.

Aizierjiang Aiersilan

University of Macau

Hallucination in Language ModelsLLM Hallucination DetectionLanguage Model ProbingHidden-State Probing
AskPDF