Language Model Probing

Ask about this topic

Overview

Language model probing studies what models represent internally and what they can do, using controlled tasks to examine hidden activations, attention, logits, and generated behavior. Common approaches include training linear or nonlinear readouts on activations, comparing representations across layers and prompts, and testing causal effects with interventions such as activation patching or steering. Probes are used to investigate linguistic knowledge, reasoning, uncertainty, hallucination, safety, and model performance, and can support practical tools for monitoring, routing, and evaluation. A central open tension is that decodable information may not be used to produce an answer, while probe results can also reflect prompt structure, labels, or dataset artifacts rather than the intended capability.

Latest papers

CardsList

279 papers35 papers in the last four weeks, up 35% on the four weeks before. 0.2% of all new papers.

  1. Activation Probes Are Reliable With a Handful of Positive Examples

    Oct 29, 2026Riya Tyagi, Stefan HeimersheimLLM Safety MonitoringLanguage Model Probing

  2. Value Probes: Learned Value Functions for Beam-Search Red Teaming

    Oct 28, 2026Jonathan F. Jao, Andrew Kyle LampinenLLM AuditingLanguage Model Probing

    Anthropic Fellows Program · Anthropic

  3. Robust LLM-Generated Text Detection through Matched Conditional-Response Probing

    Oct 28, 2026Junfeng Hao, Wenxian Wang, Tao Leng +3AI-Generated Text DetectionRobustness of Machine-Generated Text Detection

    School of Cyber Science and Engineering, Sichuan University · Cyber Science Research Institute, Sichuan University · Intelligent Policing Key Laboratory of Sichuan Province, Sichuan Police College +1

  4. Reva: Real-Time Vulnerability Detection through Contextual Probing of Large Language Models

    Oct 26, 2026Amine Lbath, Massih-Reza AminiSoftware Vulnerability DetectionLanguage Model Probing

    Université Grenoble Alpes, CNRS, France

  5. Linear Probes Transfer from Base to Distilled Language Models despite Rotation

    Oct 26, 2026Marios TsatsosLanguage Model ProbingLLM Safety Monitoring

    Independent researcher

  6. Scaling Laws for Safety Probes

    Oct 22, 2026Yuchen Wu, Jerry WeiLLM Safety MonitoringLanguage Model Probing

    University of Washington · Anthropic

  7. Same Beliefs, Different Policies: Learning Reusable Representations for Early Stopping in LLM Reasoning

    Oct 21, 2026Jingwei Ni, Dmitry Abulkhanov, Vladislav Smirnov +5Early Stopping for Language Model ReasoningEarly Stopping

    ETH Zürich · MBZUAI · University of Zürich +1

  8. Probing and Steering for Safety-Relevant Behaviours in Large Language Models

    Oct 21, 2026Nimun Bajwa, Massimo PoesioActivation SteeringLanguage Model Probing

    Queen Mary University of London

  9. Present but Hard to Read: Probing Latent Reasoning with Minimal Pairs

    Oct 20, 2026Aur Shalev MerinLanguage Model ProbingHidden-State Probing

    Independent researcher

  10. Relative Representations Enable Portable White-Box Monitoring Across Language Models

    Oct 20, 2026Tim Bakker, Christos LouizosLanguage Model ProbingRepresentation Geometry in Language Models

    Qualcomm AI Research

  11. Hearsay or Fact? How Framing and Elicitation Shape LLM Probability Updates

    Oct 20, 2026An-Sheng JhangProbability Elicitation from Language ModelsBelief Updating in LLMs

    Dongguan University of Technology

  12. How Language Models Organize Task Information

    Oct 20, 2026Kaiyang Wan, Yuxia WangLanguage Model ProbingLanguage Model Performance Prediction

    INSAIT, Sofia University “St. Kliment Ohridski” Sofia, Bulgaria

  13. Learned Readout Tokens for Extracting Large Language Model Confidence

    Oct 20, 2026Nima Seifi, Martin EsterConfidence Estimation in Language ModelsLanguage Model Probing

    School of Computing Science, Simon Fraser University, 8888 University Drive, Burnaby, BC V5A 1S6, Canada

  14. Behavioral Probes for Information Flow in LLM Swarms

    Oct 19, 2026Junhui Chen, Yu Zhang, Lantian Li +1Multi-Agent LLM SystemsLLM Agent Workflow Optimization

    School of Artificial Intelligence, Beijing University of Posts and Telecommunications, China · Center for Speech and Language Technologies, BNRist, Tsinghua University, China

  15. Measurement Validity and Representation Dynamics of Confidence Signals in Large Language Models

    Oct 19, 2026Debojeet Ghosh, Shreyansh Kumar, Ayushman GargConfidence Estimation in Language ModelsLanguage Model Calibration

  16. Do Knowledge and Reasoning Oriented Benchmarks Induce Stable Representation Signatures in LLMs?

    Oct 16, 2026Parsa Rahimi, Azim Dehghani AmirabadLLM InterpretabilityLanguage Model Probing

    EPFL, Idiap Research Institute · Independent Researcher

  17. FlashAttention with Statistics: Low-Overhead Attention Features for Compact Hallucination Detectors

    Oct 15, 2026Andrei Volodichev, Maksim Savichev, Nikita Sviridov +3LLM Hallucination DetectionLanguage Model Probing

    Applied AI Institute · Yandex · Risk AI Research Lab

  18. Narrative Lookahead in LLMs: Event-Level Decodability, Predictive Advantage, and Steering

    Oct 13, 2026Donghoon Jung, Min Kim, Seojin Choi +2Next-Event PredictionText Generation

    School of Digital Humanities and Computational Social Sciences, KAIST, South Korea

  19. From Candidate Tokens to Final Outcomes: Decoding Relative Long-Horizon Candidate Value from Reasoning Model Representations

    Oct 11, 2026Yixin Yang, Yue Wang, Fandong Meng +1LLM ReasoningLarge Reasoning Models

    State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · WeChat AI, Tencent Inc., China

  20. Do LLMs differentiate linguistic traits internally?: A layer-wise analysis

    Oct 10, 2026Samhita Bollepally, Takashi Yamauchi, Manju Muralee +1LLM InterpretabilityRepresentation Geometry in Language Models

  21. Telling FIBS: Probes Effectively Detect Sabotage and Catch Unverbalized Deception

    Oct 8, 2026Oskar J. Hollinsworth, Alex F. Spies, Tigist Diriba +2Deception in Language ModelsAI Agent Monitoring

  22. Probing for Long-Horizon Deductive Reasoning Capabilities in Language Models with Prolog

    Oct 8, 2026Hadeel Al-Negheimish, Jasna Ilieva, Yoon KimLLM ReasoningLong-Horizon LLM Reasoning

    King Saud University · Massachusetts Institute of Technology

  23. Coverage, Not Difficulty, Sets How Much Synthetic Data an Activation Probe Needs

    Oct 6, 2026Ankush CheckervartyLanguage Model ProbingSynthetic Data Evaluation

    Independent Researcher

  24. How High Is 0.6? Floors, Ceilings, and Headroom in Interpretability Probing

    Oct 6, 2026Pranjal GargRepresentation ProbingLanguage Model Probing

    Independent Researcher

  25. Knowing When Not to Answer: Cross-Domain and Multi-Turn Generalization of Latent Underspecification Signals

    Oct 6, 2026Jerzy Kamiński, Ilya Galyukshev, Artem Kuznetsov +6Multi-Turn Dialogue EvaluationLanguage Model Probing

    ITMO University, Saint Petersburg, Russia

  26. Structured but Silent: Probing Capability Requirements in LLM Hidden States

    Oct 6, 2026Kyojun Choo, Minsoo Song, Yunju Kang +1LLM Tool UseLanguage Model Probing

    Soongsil University

  27. Pseudowords as probes: Large Language Models show little of the sublexical sensitivity that governs human pseudoword processing

    Oct 6, 2026Jing Chen, Giulia Loca, Simona Amenta +1Language ModelingComputational Psycholinguistics

    Department of Psychology, University of Milano-Bicocca, Piazza dell’Ateneo Nuovo 1, 20126 Milano, Italy · Department of Informatics, Systems and Communication – DISCo, University of Milano-Bicocca, Milano, Italy

  28. Will the Judge Flip? Predicting Position-Sensitive LLM Judgments from Residual Stream Activations

    Oct 5, 2026Hashmath Shaik, Gnaneswar Villuri, Alex DoboliLLM EvaluationLLM-as-a-Judge

    Department of Electrical and Computer Engineering Stony Brook University Stony Brook, NY 11794, USA

  29. Cross-lingual Calibration of Pre-Generation Success Probes for Multilingual LLM Routing

    Oct 5, 2026Andrea Paganelli, Stefano Civelli, Pietro Bernardelle +1Confidence Estimation in Language ModelsLanguage Model Calibration

    The University of Queensland · Polytechnic University of Milan

  30. Correctness Is a Direction: Geometric Answer Selection in Language Models

    Oct 3, 2026Marcus Armstrong, Navid Ayoobi, Pradham Mummaleti +2Language Model CalibrationLanguage Model Probing

    Department of Computer Science University of Houston Houston, TX 77204