Language Model Probing

Follow this topic Ask about this topic

Language model probing studies what models represent internally and what they can do, using controlled tasks to examine hidden activations, attention, logits, and generated behavior. Common approaches include training linear or nonlinear readouts on activations, comparing representations across layers and prompts, and testing causal effects with interventions such as activation patching or steering. Probes are used to investigate linguistic knowledge, reasoning, uncertainty, hallucination, safety, and model performance, and can support practical tools for monitoring, routing, and evaluation. A central open tension is that decodable information may not be used to produce an answer, while probe results can also reflect prompt structure, labels, or dataset artifacts rather than the intended capability.

Latest papers

280 papers35 papers in the last four weeks, up 35% on the four weeks before. 0.2% of all new papers.

Perturbation Probing: A Two-Pass-per-Prompt Diagnostic for FFN Behavioral Circuits in Aligned LLMs
Apr 30, 2026Hongliang Liu, Tung-Ling Li, Yuhao WuLLM Interpretability Mechanistic Interpretability
Palo Alto Networks
Do LLMs Capture Embodied Cognition and Cultural Variation? Cross-Linguistic Evidence from Demonstratives
Apr 28, 2026Yu Wang, Emmanuele Chersoni, Chu-Ren HuangMultilingual Language Model Evaluation LLM Grounding
Department of Language Science and Technology · The Hong Kong Polytechnic University · Hung Lok Road, Hung Hom, Kowloon, Hong Kong (China)
Evaluation without Generation: Non-Generative Assessment of Harmful Model Specialization with Applications to CSAM
Apr 28, 2026Vinith M. Suriyakumar, Ayush Sekhari, Lena Stempfle +5LLM Auditing Language Model Probing
MIT · BU · Thorn
Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacity
Apr 27, 2026Bojie LiParameter Estimation LLM Evaluation
Pine AI
The Pragmatic Persona: Discovering LLM Persona through Bridging Inference
Apr 27, 2026Jisoo Yang, Jongwon Ryu, Minuk Ma +2Personality Modeling in Language Models Language Model Probing
Department of Artificial Intelligence, Chung-Ang University, Seoul, 06974, Republic of Korea · Department of Computer Science, University of British Columbia,2026 Vancouver, BC V6T 1Z4, Canada · Van Lang University, Ho Chi Minh City, Vietnam
Revisiting Non-Verbatim Memorization in Large Language Models: The Role of Entity Surface Forms
Apr 23, 2026Yuto Nishida, Naoki Shikoda, Yosuke Kishinami +4LLM Evaluation Memorization in Language Models
Nara Institute of Science and Technology · Future Corporation
Measuring Opinion Bias and Sycophancy via LLM-based Persuasion
Apr 23, 2026Rodrigo Nogueira, Giovana Kerche Bonás, Thales Sales Almeida +7LLM Sycophancy LLM Auditing
Maritaca AI · JusBrasil
Slot Machines: How LLMs Keep Track of Multiple Entities
Apr 22, 2026Paul C. Bogdan, Jack LindseyLLM Interpretability Language Model Probing
Anthropic Fellows Program · Anthropic
Tracing Relational Knowledge Recall in Large Language Models
Apr 21, 2026Nicholas Popovič, Michael FärberLinear Probing Factual Knowledge in Language Models
TU Dresden & ScaDS.AI Dresden/Leipzig, Germany
Harmful Intent as a Geometrically Recoverable Feature of LLM Residual Streams
Apr 20, 2026Isaac Llorente-SaguerLanguage Model Safety Evaluation LLM Interpretability
Independent Researcher
Probing for Reading Times
Apr 20, 2026Eleftheria Tsipidi, Samuel Kiegeland, Francesco Ignazio Re +4Computational Psycholinguistics Language Model Probing
ETH Zürich · Toyota Technological Institute at Chicago · University College London
Dual Alignment Between Language Model Layers and Human Sentence Processing
Apr 20, 2026Tatsuki Kuribayashi, Alex Warstadt, Yohei Oseki +1Language Modeling Computational Psycholinguistics
MBZUAI · Tohoku University · UC San Diego +2
Exploring Concreteness Through a Figurative Lens
Apr 20, 2026Saptarshi Ghosh, Tianyu JiangFigurative Language Understanding LLM Interpretability
University of Cincinnati
STaD: Scaffolded Task Design for Identifying Compositional Skill Gaps in LLMs
Apr 20, 2026Sungeun An, Swanand Ravindra Kadhe, Shailja Thakur +2LLM Evaluation Reasoning Evaluation
IBM Research
SPENCE: A Syntactic Probe for Detecting Contamination in NL2SQL Benchmarks
Apr 20, 2026Mohammadtaher Safarzadeh, Hitesh Laxmichand Patel, Afshin Orojlooyjadid +2LLM Evaluation Benchmark Contamination
Oracle AI
Before You Interpret the Profile: Validity Scaling for LLM Metacognitive Self-Report
Apr 20, 2026Jon-Paul CacioliPsychometric Validation LLM Uncertainty Estimation
Independent Researcher, Melbourne, Australia
Introspection Adapters: Training LLMs to Report Their Learned Behaviors
Apr 18, 2026Keshav Shenoy, Li Yang, Abhay Sheshadri +4LLM Auditing Language Model Introspection
Anthropic Fellows · Ashwood Centre on AI Science and Policy, University of Cambridge · Anthropic
Segment-Level Coherence for Robust Harmful Intent Probing in LLMs
Apr 16, 2026Xuanli He, Bilgehan Sel, Faizan Ali +3LLM Safety LLM Jailbreak Attacks
University College London · Virginia Tech · Anthropic
Linear Probe Accuracy Scales with Model Size and Benefits from Multi-Layer Ensembling
Apr 15, 2026Erik Nordby, Tasha Pais, Aviel ParrackLanguage Model Probing Deception in Language Models
Georgia Institute of Technology, Atlanta, Georgia, USA · Independent Researcher · Stanford University, Stanford, California, USA
H-Probes: Extracting Hierarchical Structures From Latent Representations of Language Models
Apr 15, 2026Cutter Dawes, Aryan Sharma, Angelos Ioannis Lagos +1Representation Geometry in Language Models Hierarchical Representation Learning
Supervised Program for Alignment Research · Yale University · Harvard University
Do BERT Embeddings Encode Narrative Dimensions? A Token-Level Probing Analysis of Time, Space, Causality, and Character in Fiction
Apr 12, 2026Beicheng Bei, Hannah Hyesun Chun, Chen Guo +1Language Model Probing Representation Geometry in Language Models
Department of Linguistics, University of Rochester, 503 Lattimore Hall, Rochester, NY 14627
Nationality encoding in language model hidden states: Probing culturally differentiated representations in persona-conditioned academic text
Apr 11, 2026Paul Jackson, Ruizhe Li, Elspeth EdelsteinDemographic Bias in Language Models Cross-Cultural Language Model Evaluation
Language Centre, School of Language, Literature, Music and Visual Culture, University of Aberdeen, United Kingdom · Language Centre, School of Language, Literature, Music and Visual Culture, University of Aberdeen, Aberdeen AB24 3FX, United Kingdom. · School of Natural and Computing Sciences, University of Aberdeen, United Kingdom +3
Multilingual Embedding Probes Fail to Generalize Across Learner Corpora
Apr 8, 2026Laurits Lyngbaek, Ross Deans Kristensen-McLachlanLanguage Model Probing Cross-Dataset Generalization
Aarhus University, Denmark
MechELK: A Mechanistic Interpretability Framework for Eliciting Latent Knowledge in Large Language Models
Apr 7, 2026Ji-jun Park, Soo-joon Choi, Jiwon Jeong +2LLM Interpretability Language Model Probing
Dongguk University
Weakly Supervised Distillation of Hallucination Signals into Transformer Representations
Apr 7, 2026Shoaib Sadiq Salehmohamed, Jinal Prashant Thakkar, Hansika Aredla +2LLM Hallucination Detection Language Model Probing
LLM Lens
What are They Thinking? Delineation, Probing, and Tracking of Concepts in LLMs
Apr 7, 2026Mohamed Abdelwahab, Michelle Yu Collins, Sihan Chen +5Linear Probing LLM Interpretability
The Edward S. Rogers Sr. Department of Electrical and Computer Engineering University of Toronto
When Models Know More Than They Say: Probing Analogical Reasoning in LLMs
Apr 4, 2026Hope McGovern, Caroline Craig, Thomas Lippincott +1Analogical Reasoning for Language Models LLM Reasoning
Cambridge University · Northeastern University · Athenahealth +1
'Layer su Layer': Identifying and Disambiguating the Italian NPN Construction in BERT's family
Apr 4, 2026Greta Gorzoni, Ludovica Pannitto, Francesca MasiniLanguage Model Probing Representation Probing
Alma Mater Studiorum - University of Bologna
GRADE: Probing Knowledge Gaps in LLMs through Gradient Subspace Dynamics
Apr 3, 2026Yujing Wang, Yuanbang Liang, Yukun Lai +2LLM Interpretability LLM Uncertainty Estimation
Beihang University · Cardiff University · King’s College London
Beyond the Assistant Turn: User Turn Generation as a Probe of Interaction Awareness in Language Models
Apr 2, 2026Sarath Shekkizhar, Romain Cosentino, Adam EarleLLM Evaluation Language Model Probing
Salesforce AI Research

Apr 30, 2026J/K move · Enter open · S save

Perturbation Probing: A Two-Pass-per-Prompt Diagnostic for FFN Behavioral Circuits in Aligned LLMs

Hongliang Liu, Tung-Ling Li, Yuhao Wu

Palo Alto Networks

LLM Interpretability Mechanistic Interpretability LLM Safety Circuits in Language Models Causal Interventions in Language Models Language Model Probing

Open paper AskPDF arXiv

Perturbation probing generates task-specific causal hypotheses for FFN neurons in large language models using two forward passes per prompt and no backpropagation, followed by a one-time intervention sweep of about 150 passes amortized across all identified neurons. Across eight behavioral circuits, 13 models, and four architecture families, we identify two circuit structures that organize LLM behavior. Opposition circuits appear when RLHF suppresses a pre-training tendency. In safety refusal, about 50 neurons, or 0.014 percent of all neurons, control the refusal template; ablating them changes 80 percent of response formats on 520 AdvBench prompts while producing near-zero harmful compliance, 3 of 520 cases, all with disclaimers. Routing circuits appear for pre-training behaviors distributed through attention. For language selection, residual-stream direction injection switches English to Chinese output on 99.1 percent of 580 benchmark prompts in the 3 of 19 tested models that satisfy three observed conditions: bilingual training, FFN-to-skip signal ratio between 0.3 and 1.1, and linear representability. The same intervention fails on the other 16 models and on math, code, and factual circuits, defining the limits of directional steering. The FFN-to-skip signal ratio, computed from the same two forward passes, distinguishes the two structures and predicts the appropriate intervention. Circuit topology varies by architecture, from Qwen's concentrated FFN bottleneck to Gemma's normalization-shielded circuit. In Qwen3.5-2B, ablating 20 neurons eliminates multi-turn sycophantic capitulation, while amplifying 10 related neurons improves factual correction from 52 percent to 88 percent on 200 TruthfulQA prompts. These results show that perturbation probing offers mechanistic insight into RLHF-organized behavior and a practical toolkit for precision template-layer editing.