Language Model Probing

Follow this topic Ask about this topic

Momentum

19 papers in the last four weeks, up 12% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 192

All topics

Refusal Before Decoding: Detecting and Exploiting Refusal Signals in Intermediate LLM Activations
May 27, 2026Matteo Gioele Collu, Riccardo Conte, Alberto Giaretta +4LLM Interpretability Adversarial Attacks on LLMs
University of Padua, Italy · 2Örebro University, Sweden · 3Fondazione Bruno Kessler, Italy
What Makes Chain-of-Thought Work at Probe Time? Local Co-occurrence Rather Than Global Derivation
May 26, 2026Xiang Wang, Wei WeiLLM Interpretability CoT Reasoning
Huazhong University of Science and Technology
Probing Minimalist Phase Structure in LLMs: What Universal Dependencies Cannot Represent
May 26, 2026Yuanhao Chen, Peter ChinLanguage Model Probing
Dartmouth College
Can LLMs Introspect? A Reality Check
May 25, 2026Shashwat Singh, Tal Linzen, Shauli RavfogelLLM Evaluation LLM Interpretability
Center for Data Science New York University
Tool-Call Dependency Structure is Linearly Decodable in LLM Agent Residual Streams
May 25, 2026Tianda Sun, Dimitar KazakovLLM Tool Use Language Model Probing
University of York Department of Computer Science Heslington, York YO10 5DD
MultiHaluDet: Multilingual Hallucination Detection via LLM Hidden State Probing
May 24, 2026Riasad Alvi, Nurul Labib Sayeedi, Md. Faiyaz Abdullah SayeediHallucination Detection Hallucination in Language Models
United International University · BRAC University
Relational Linear Properties in Language Models: An Empirical Investigation
May 21, 2026Giovanni Valer, Luigi Gresele, Marco Bronzini +1LLM Interpretability Factual Knowledge in Language Models
Fondazione Bruno Kessler, Trento, Italy · University of Bologna, Bologna, Italy · University of Pisa, Pisa, Italy +2
Represented Is Not Computed: A Causal Test of Candidate Algorithmic Intermediates in a Transformer
May 21, 2026Ishita Darade, Sushrut ThoratNumerical Reasoning in Language Models Transformer Interpretability
MKSSS’s Cummins College of Engineering for Women, Pune 411052, India · Institute of Cognitive Science, Osnabrück University, Osnabrück 49069, Germany
Check Your LLM's Secret Dictionary! Five Lines of Code Reveal What Your LLM Learned (Including What It Shouldn't Have)
May 21, 2026Hisashi MiyashitaLLM Auditing LLM Interpretability
Mgnite Inc.
Do LLMs Know What Luxembourgish Borrows? Probing Lexical Neology in Low-Resource Multilingual Models
May 20, 2026Nina Hosseini-KivananiMultilingual Language Model Evaluation Knowledge Augmentation for Language Models
University of Luxembourg, Luxembourg · Radio Télévision Luxembourg (RTL), Luxembourg
Reading Calibrated Uncertainty from Language Model Trajectories
May 19, 2026Aliai Eusebi, Alexander Herzog, Xiaoyu Liang +3Confidence Estimation in Language Models LLM Reliability
University College London, London, United Kingdom · Core46, London, United Kingdom
Monitoring the Internal Monologue: Probe Trajectories Reveal Reasoning Dynamics
May 18, 2026Maciej Chrabąszcz, Aleksander Szymczyk, Marcin Sendera +2Language Model Safety Evaluation LLM Interpretability
NASK - National Research Institute, Poland · Faculty of Electronics and Information Technology, Warsaw University of Technology, Poland · Faculty of Mathematics and Computer Science, Jagiellonian University, Poland +3
Probing for Representation Manifolds in Superposition
May 18, 2026Alexander ModellFeature Superposition Neural Representation Geometry
Department of Mathematics Imperial College London, U.K.
Geometry-Lite: Interpretable Safety Probing via Layer-Wise Margin Geometry
May 18, 2026Woo Seob Sim, Yu Rang ParkLanguage Model Safety Evaluation LLM Safety
Department Biomedical Systems Informatics Yonsei University College of Medicine, Seoul, Republic of Korea · Yonsei University
PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifacts
May 16, 2026Khizar Hussain, Murat KantarciogluLLM Evaluation Language Model Error Detection
Virginia Tech
When Answers Stray from Questions: Hallucination Detection via Question-Answer Orthogonal Decomposition
May 14, 2026Siyang Yao, Erhu Feng, Yubin XiaHallucination in Language Models LLM Hallucination Detection
Shanghai Jiao Tong University Shanghai,China
Polar probe linearly decodes semantic structures from LLMs
May 13, 2026Pablo J. Diego-Simón, Pierre Orhan, Emmanuel Chemla +2Representation Learning Linear Probing
LSCP, ENS, PSL, EHESS, CNRS, Paris, France · Paris Brain Institute, Paris, France · Earth Species Project, Berkeley, California, USA +1
Probing Persona-Dependent Preferences in Language Models
May 13, 2026Oscar Gilg, Pierre Beckmann, Daniel Paleka +1LLM Alignment LLM Interpretability
MATS · MATS, EPFL · ETH Zürich +1
Controlling Logical Collapse in LLMs via Algebraic Ontology Projection over F2
May 13, 2026Hisashi Miyashita, Mgnite IncLLM Interpretability Reasoning Consistency in Language Models
Mgnite Inc.
A Controlled Counterexample to Strong Proxy-Based Explanations of OOD Performance: in a Fixed Pretraining-and-Probing Setup
May 12, 2026Hongmin LiOOD Generalization Language Model Probing
School of Life Science and Technology, Institute of Science Tokyo · Department of Computational Biology and Medical Sciences, Graduate School of Frontier Sciences, The University of Tokyo
Instructions Shape Production of Language, not Processing
May 11, 2026Andreas Waldis, Leshem Choshen, Yufang Hou +1LLM Interpretability Instruction Following
Department of Linguistics, University of Tübingen · IBM Research, MIT, and MIT-IBM Watson AI Lab · MIT, and MIT-IBM Watson AI Lab +2
Likelihood scoring for continuations of mathematical text: a self-supervised benchmark with tests for shortcut vulnerabilities
May 11, 2026Daniel RanardLLM Evaluation Next-Token Prediction
Department of Physics, California Institute of Technology, Pasadena, CA 91125, USA
Measuring Embedding Sensitivity to Authorial Style in French: Comparing Literary Texts with Language Model Rewritings
May 11, 2026Benjamin Icard, Lila Sainero, Alice Breton +2LLM Evaluation Text Embeddings
LIP6, Sorbonne University, CNRS, France · Infopro Digital, France
Position: Let's Develop Data Probes to Fundamentally Understand How Data Affects LLM Performance
May 11, 2026Shiqiang Wang, Herbert Woisetschläger, Hans Arno Jacobsen +1Synthetic Data Generation LLM Training
Department of Computer Science, University of Exeter, UK · Technical University of Munich, Germany · Department of Electrical and Computer Engineering, University of Toronto, Canada +1
Tensor Product Representation Probes Reveal Shared Structure Across Linear Directions
May 11, 2026Andrew Lee, Fernanda Viégas, Martin WattenbergTransformer Interpretability Neural Representation Geometry
Harvard University · Google DeepMind* · *Work done at Harvard.
The Metacognitive Probe: Five Behavioural Calibration Diagnostics for LLMs
May 11, 2026Rafael C. T. OliveiraLanguage Model Calibration LLM Uncertainty Estimation
Independent Researcher
Beyond Language: Format-Agnostic Reasoning Subspaces in Large Language Models
May 10, 2026Aojie Yuan, Zhiyuan SuLLM Interpretability Representation Geometry in Language Models
University of Southern California · 2Duke University
Do Linear Probes Generalize Better in Persona Coordinates?
May 10, 2026Prasad Mahadik, Adrians SkaparsLanguage Model Safety Evaluation Deception in Language Models
Independent Researcher · University of Manchester
The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations
May 9, 2026Rania Elbadry, Ahmed Heakl, Fan Zhang +4LLM Interpretability LLM Reliability
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · INSAIT, Sofia University
Do LLMs Experience an Internal Polylogue? Investigating Reasoning through the Lens of Personas
May 9, 2026Nils A. Herrmann, Leander Girrbach, Kirill Bykov +1Language Model Steering LLM Interpretability
1Technical University of Munich · 2Helmholtz Munich · 3Munich

May 27, 2026, cs.AIJ/K move · Enter open · S save

Refusal Before Decoding: Detecting and Exploiting Refusal Signals in Intermediate LLM Activations

Matteo Gioele Collu, Riccardo Conte, Alberto Giaretta, Denis Kleyko+3

University of Padua, Italy · 2Örebro University, Sweden · 3Fondazione Bruno Kessler, Italy

LLM Interpretability Adversarial Attacks on LLMs LLM Refusal Behavior LLM Jailbreak Attacks Refusal Behavior in Language Models Language Model Probing

Open paper AskPDF arXiv

In this paper, we investigate whether refusal behavior can be predicted from LLM intermediate activations before decoding using linear probes trained on residual stream activations at each transformer block. We find that refusal is linearly decodable well before the final layer, indicating that safety-relevant behavior is represented in intermediate activations before output generation. To test whether this signal is actionable, we introduce Mechanistic AutoDAN, a probe-guided variant of AutoDAN that replaces full-model fitness evaluation with partial forward passes and probe-based scoring inside a genetic prompt search loop. Across the evaluated models, our method achieves attack success rates competitive with vanilla AutoDAN while reducing per-iteration search time by up to 72%, and probe-guided prompts match or exceed AutoDAN's cross-model transfer in several configurations. We further find that the usefulness of probe guidance increases with model scale. Our results suggest that refusal-relevant information is decodable from intermediate activations and can serve as an effective search signal in an AutoDAN-style discrete jailbreak optimization loop, especially for larger and more robust models.

Momentum

Related topics

Latest papers 192