Language Model Probing

Follow this topic Ask about this topic

Language model probing studies what models represent internally and what they can do, using controlled tasks to examine hidden activations, attention, logits, and generated behavior. Common approaches include training linear or nonlinear readouts on activations, comparing representations across layers and prompts, and testing causal effects with interventions such as activation patching or steering. Probes are used to investigate linguistic knowledge, reasoning, uncertainty, hallucination, safety, and model performance, and can support practical tools for monitoring, routing, and evaluation. A central open tension is that decodable information may not be used to produce an answer, while probe results can also reflect prompt structure, labels, or dataset artifacts rather than the intended capability.

Latest papers

280 papers35 papers in the last four weeks, up 35% on the four weeks before. 0.2% of all new papers.

Fast and Accurate Probing of In-Training LLMs' Downstream Performances
Apr 1, 2026Zhichen Liu, Tianle Lun, Zhibin Wen +7LLM Evaluation Language Model Probing
Southern University of Science and Technology · Huawei
LLM Probe: Evaluating LLMs for Low-Resource Languages
Mar 31, 2026Hailay Kidu Teklehaymanot, Gebrearegawi Gebremariam, Wolfgang NejdlMultilingual Language Model Evaluation Low-Resource Language Processing
L3S Research Center, Leibniz University Hannover, Germany · Aksum University, Ethiopia
WMF-AM: Probing LLM Working Memory via Depth-Parameterized Cumulative State Tracking
Mar 28, 2026Dengzhe Hou, Lingyu Jiang, Deng Li +3Working Memory in Language Models LLM Evaluation
Tohoku University · Lappeenranta-Lahti University of Technology LUT · Texas A&M University +1
ReLope: From Hidden-State Probing to a Decision Module for Multimodal LLM Routing
Mar 25, 2026Yaopei Zeng, Congchao Wang, Blake JianHang Chen +1Multimodal Large Language Models LLM Routing
Pennsylvania State University · Google · Amazon AGI
Probing Ethical Framework Representations in Large Language Models: Structure, Entanglement, and Methodological Challenges
Mar 24, 2026Weilun Xu, Alexander Rusnak, Frederic KaplanMoral Reasoning in Language Models Language Model Probing
School of Computer and Communication Sciences, École Polytechnique Fédérale de Lausanne · Digital Humanities Lab, École Polytechnique Fédérale de Lausanne
Riding Brainwaves in LLM Space: Understanding Activation Patterns Using Individual Neural Signatures
Mar 23, 2026Ajan Subramanian, Sumukh Bettadapura, Rohan SathishEEG Decoding Language Model Probing
PROMPT2BOX:Improving LLM Weakness Discovery and Specificity Estimation by Uncovering Entailment Structure among Prompts
Mar 22, 2026Neeladri Bhuiya, Shib Sankar Dasgupta, Andrew McCallum +1LLM Evaluation Representation Learning
CICS, University of Massachusetts Amherst, USA · A10 Networks · Amazon AWS AI
Is Evaluation Awareness Just Format Sensitivity? Limitations of Probe-Based Evidence under Controlled Prompt Structure
Mar 19, 2026Viliana DevbunovaEvaluation Awareness in Language Models Language Model Probing
Yandex Belgrade, Serbia
Geography According to ChatGPT -- How Generative AI Represents and Reasons about Geography
Mar 19, 2026Krzysztof Janowicz, Gengchen Mai, Rui Zhu +4Geospatial Reasoning Factual Knowledge in Language Models
University of Vienna, Austria · University of Texas at Austin, USA · University of Bristol, UK +4
Catching rationalization in the act: detecting motivated reasoning before and after CoT via activation probing
Mar 17, 2026Parsa Mirtaheri, Mikhail BelkinLLM Interpretability CoT Faithfulness
UC San Diego
Mind the Shift: Decoding Monetary Policy Stance from FOMC Statements with Large Language Models
Mar 15, 2026Yixuan Tang, Yi YangStance Detection Finance
The Hong Kong University of Science and Technology
Human-Centred LLM Privacy Audits: Findings and Frictions
Mar 12, 2026Dimitri Staufer, Kirsten Morehouse, David Hartmann +1Privacy Auditing LLM Auditing
TU Berlin Berlin, Germany · Columbia University New York, USA · Weizenbaum Institute for the Networked Society Berlin, Germany +1
Do Language Models Know Theo Has a Wife? Investigating the Proviso Problem
Mar 9, 2026Tara Azin, Daniel Dumitrescu, Diana Inkpen +1Pragmatic Reasoning in Language Models Language Model Probing
Carleton University, Canada · University of Ottawa, Canada
DepthCharge: A Domain-Agnostic Framework for Measuring Depth-Dependent Knowledge in Large Language Models
Mar 5, 2026Alexander SheppertLLM Evaluation Factual Knowledge in Language Models
Legacy Health · Capitol Technology University
Censored LLMs as a Natural Testbed for Secret Knowledge Elicitation
Mar 5, 2026Helena Casademunt, Bartosz Cywiński, Khoi Tran +3LLM Honesty Language Model Probing
Harvard University · Warsaw University of Technology · IDEAS Research Institute +2
Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought
Mar 5, 2026Siddharth Boppana, Annabel Ma, Max Loeffler +5Faithfulness of Language Model Reasoning Language Model Reasoning Evaluation
Goodfire AI · Harvard University, Cambridge, MA
World Properties without World Models: Distributional Associations and the Interpretation of Decoding Results from Language Models
Mar 4, 2026Elan BarenholtzLLM World Models Word Embeddings
Department of Psychology & Center for Complex Systems and Brain Sciences Florida Atlantic University
Eliciting Numerical Predictive Distributions of LLMs Without Autoregression
Mar 3, 2026Julianna Piskorz, Katarzyna Kobalczyk, Mihaela van der SchaarLLM Uncertainty Estimation LLM Inference
Department of Applied Mathematics and Theoretical Physics, University of Cambridge
Probing Materials Knowledge in LLMs: From Latent Embeddings to Reliable Predictions
Mar 2, 2026Vineeth Venugopal, Soroush Mahjoubi, Elsa OlivettiLLM Reliability LLM Evaluation
Department of Materials Science and Engineering, Massachusetts Institute of Technology, Cambridge, 02139, MA, USA.
Probing for Knowledge Attribution in Large Language Models
Feb 26, 2026Ivo Brink, Alexander Boer, Dennis UlmerLLM Interpretability Hallucination in Language Models
KPMG NL / University of Amsterdam Amsterdam, The Netherlands · KPMG NL Amsterdam, The Netherlands · University of Amsterdam Amsterdam, The Netherlands
Hidden Topics: Measuring Sensitive AI Beliefs with List Experiments
Feb 25, 2026Maxim ChupilkinProbability Elicitation from Language Models LLM Alignment
How Transformers Reject Wrong Answers: Rotational Dynamics of Factual Constraint Processing
Feb 25, 2026Javier MarínTransformer Interpretability LLM Interpretability
Groundlens-dev
The Truthfulness Spectrum Hypothesis
Feb 23, 2026Zhuofan Josh Ying, Shauli Ravfogel, Nikolaus Kriegeskorte +1LLM Honesty Deception in Language Models
Department of Psychology, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY · New York University, New York, NY · Department of Neuroscience, Columbia University, New York, NY +2
Latent Introspection: Models Can Detect Prior Concept Injections
Feb 23, 2026Theia Pearson-Vogel, Martin Vanek, Raymond Douglas +1Language Model Introspection Metacognition in Language Models
ACS Research, CTS, Charles University
What Do LLMs Associate with Your Name? A Human-Centered Black-Box Audit of Personal Data
Feb 19, 2026Dimitri Staufer, Kirsten MorehousePrivacy Leakage in Language Models LLM Auditing
TU Berlin, Berlin, Germany · Columbia University, New York, USA
Mechanistic Interpretability of Cognitive Complexity in LLMs via Linear Probing using Bloom's Taxonomy
Feb 19, 2026Bianca Raimondi, Maurizio GabbrielliLLM Interpretability Representation Geometry in Language Models
Department of Computer Science and Engineering, University of Bologna, Bologna, Italy
Probing the Limits of the Lie Detector Approach to LLM Deception
Feb 16, 2026Tom-Felix ThormannDeception in Language Models Language Model Probing
Ruhr-University Bochum Institute for Philosophy II
From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs
Jan 31, 2026Louis Schiekiera, Max Zimmer, Christophe Roux +2Representation Geometry in Language Models Computational Psycholinguistics
Computational Modelling Lab, Humboldt-Universität zu Berlin, Berlin, Germany · Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany · Department for AI in Society, Science, and Technology, Zuse Institute Berlin, Germany +1
Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry
Jan 30, 2026Zhuochun Li, Yong Zhang, Ming Li +8LLM Evaluation LLM-as-a-Judge
Ping An Technology (Shenzhen) Co., Ltd. · University of Pittsburgh · University of Maryland, College Park +1
Self-Aware Knowledge Probing: Evaluating Language Models' Relational Knowledge through Confidence Calibration
Jan 26, 2026Christopher Kissling, Elena Merdjanovska, Alan AkbikLanguage Model Probing Language Model Calibration
Humboldt-Universität zu Berlin · Science of Intelligence

Apr 1, 2026J/K move · Enter open · S save

Fast and Accurate Probing of In-Training LLMs' Downstream Performances

Zhichen Liu, Tianle Lun, Zhibin Wen, Hao An+6

Southern University of Science and Technology · Huawei

LLM Evaluation Language Model Probing

Open paper AskPDF arXiv

The paradigm of scaling Large Language Models (LLMs) in both parameter size and test time has pushed the boundaries of AI capabilities, but at the cost of making the traditional generative evaluation paradigm prohibitively expensive, therefore making the latency of LLM's in-training downstream performance evaluation unbearable. However, simple metrics like training loss (perplexity) are not always correlated with downstream performance, as sometimes their trends diverge from the actual task outcomes. This dilemma calls for a method that is computationally efficient and sufficiently accurate in measuring model capabilities. To address this challenge, we introduce a new in-training evaluation paradigm that uses a lightweight probe for monitoring downstream performance. The probes take the internal representations of LLM checkpoints (during training) as input and directly predict the checkpoint's performance on downstream tasks measured by success probability (i.e., pass@1). We design several probe architectures, validating their effectiveness using the OLMo3-7B's checkpoints across a diverse set of downstream tasks. The probes can accurately predict a checkpoint's performance (with avg. AUROC

>

0.75), have decent generalizability across checkpoints (earlier predicts later), and reduce the computation latency from

\sim

1 hr (using conventional generative evaluation method) to

\sim

3 min. In sum, this work presents a practical and scalable in-training downstream evaluation paradigm, enabling a more agile, informed, and efficient LLM development process.