Linear Probing

Ask about this topic

Overview

In machine learning, linear probing means freezing a model’s representations and training a simple classifier or regressor—often logistic regression or ridge regression—to measure which task-relevant features they make accessible. Researchers apply it across layers, modules, and subspaces in vision, language, audio, medicine, and scientific models, using benchmarks for classification, geometry, factual knowledge, and out-of-distribution transfer. Methods now include layer fusion, low-rank and whitened probes, and probe-based steering, while comparisons with fine-tuning help assess label efficiency. A central tension is that easy linear decoding does not by itself show that a model uses that information causally, and probe results can be sensitive to label noise, task design, and distribution shift.

Latest papers

CardsList

38 papers5 papers in the last four weeks, up 67% on the four weeks before. 0.0% of all new papers.

  1. Controlled Evaluation of EEG Foundation Models: Random Initialisation, Band-Power and Validation-Selected Probes

    Oct 23, 2026Mingyu Jeon, Suwan Cho, Jae Young SuhEEG Foundation ModelsEEG Decoding

    Modulabs, Seoul, Republic of Korea

  2. When Models Know but Can't Linearly Say: Relational Encoding Limits Linear Vulnerability Classification in Code LLMs

    Oct 19, 2026Rui Melo, Andre Catarino, Claudia Mamede +2LLMs for CybersecuritySoftware Vulnerability Detection

    Carnegie Mellon University, Pittsburgh, PA, USA · Faculty of Engineering, University of Porto, Portugal

  3. Is a Linear Probe Evidence of a Linear Representation?

    Oct 15, 2026Nadir Kutluozen, Christian Internò, David KlindtRepresentation Geometry in VLMsLinear Representation Hypothesis

    Cold Spring Harbor Laboratory · Bielefeld University

  4. Whitening Improves Robustness to Spurious Correlations in Linear Probes

    Sep 30, 2026Floris Holstege, Bram Wouters, Noud van Giersbergen +1Linear ProbingFeature Whitening

    University of Amsterdam, Department of Quantitative Economics · Tinbergen Institute

  5. Capacity, Responsiveness and Alignment: What Makes a Latent Structure Actionable

    Sep 28, 2026Or Shafran, Mor GevaCausal Interventions in Language ModelsCausal Interpretability

    Blavatnik School of Computer Science and AI, Tel Aviv University

  6. CG-Probes: Recovering Guardrail Directions from Patient Query Embeddings

    Sep 25, 2026Marko Řeháček, Vítězslav Dušek, Martin Rusinko +1LLM GuardrailsLinear Probing

    Faculty of Informatics Masaryk University Brno, Czech Republic · Dept. of Pediatric Oncology University Hospital Brno Brno, Czech Republic · Faculty of Medicine Masaryk University +1

  7. Exact semantic readout from compressed vector representations

    Sep 16, 2026Daniel QuigleyLinear ProbingEmbedding Compression

    Center for Possible Minds Indiana University Bloomington Bloomington, IN 47408

  8. MedProb: Probing Internal Representations of Vision-Language Models for Medical Question Answering

    Sep 3, 2026Erfan Nourbakhsh, Ke Yang, Anthony RiosPosition Bias in VLMsMedical VQA

    The University of Texas at San Antonio

  9. The Hallucination Signal Is a Mean Shift: Why Simple Probes Suffice

    Aug 28, 2026Jungseob Lee, Jaehyung Seo, Heuiseok LimLLM Hallucination DetectionHidden-State Probing

    Korea University · Konkuk University

  10. How Do Linear Probes Emerge? A Circuit-Tracing Framework with Concept-Targeted Attribution

    Aug 27, 2026Vedant Palit, Florent Draye, Terry Jingchen Zhang +2Mechanistic InterpretabilityFeature Attribution

    Jinesis Lab, University of Toronto & Vector Institute · EuroSafeAI · Max Planck Institute for Intelligent Systems, Tübingen, Germany +2

  11. What AstroPT knows about galaxies, and what that can teach us about LLMs

    Aug 23, 2026UniverseTBD, :, Kshitij Duraphe +3Mechanistic InterpretabilityLanguage Model Probing

    IUCAA Pune, India

  12. Measuring Concept Content in Text from LLM Activations: ESG Evidence from Concept Vectors and Linear Probes

    Aug 7, 2026Luc Hazenoot, Zhaochun Ren, Amirhossein ZohrehvandLinear ProbingLLM Interpretability

    Leiden Institute of Advanced Computer Science, Leiden University

  13. Training-Free versus Training-Based Intent Classification in LLMs: Accuracy, Robustness, and Failure Modes

    Aug 3, 2026Nan Chen, Zhouhao Yang, Soufiane HayouZero-Shot LearningLLM Evaluation

    Department of Applied Mathematics and Statistics Johns Hopkins University

  14. Foundation Models for Face Presentation Attack Detection: A Unified Linear-Probing Benchmark

    Jul 29, 2026Peter Lorenz, Anjith George, Sébastien MarcelVision Foundation ModelsLinear Probing

    Idiap Research Institute, Martigny, Switzerland · University of Lausanne (UNIL), Lausanne, Switzerland

  15. Understanding Geometric Representations in Self-Supervised Vision Transformers via Subspace Intervention

    Jul 2, 2026Weichen Zhou, Yawen Zou, Chunzhi Gu +3Neural Representation GeometryLinear Probing

    University of Toyama, Toyama, Japan · University of Fukui, Fukui, Japan · Chukyo University, Aichi, Japan +1

  16. Primary ICD Category Prediction using LLM-based Probing

    Jun 27, 2026Chengyuan Liu, Xinyue Zhang, Yao Li +1Automated Medical CodingLinear Probing

    Department of Statistics, Pennsylvania State University, PA, USA · Department of Biostatistics, University of North Carolina at Chapel Hill, NC, USA · Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, NC, USA

  17. Comparing Linear Probes with Mahalanobis Cosine Similarity

    Jun 17, 2026Zhuofan Josh Ying, Peter Hase, Nikolaus KriegeskorteLinear ProbingMahalanobis Distance

    Departments of 1Psychology, 2Neuroscience, 3Electrical Engineering · 4Zuckerman Mind Brain Behavior Institute; Columbia University, New York, NY · 5Stanford University, Stanford, CA +1

  18. Where Do We (Not) Need Temporal Context in Low-Resource Video Task Adaptation?

    Jun 2, 2026Luc P. J. Sträter, Hazel DoughtyLinear ProbingVision Foundation Model Adaptation

    Leiden University

  19. Refit the Probe: Single-Direction Ablation Is Not a Necessity Test

    May 30, 2026Yuhang JiangTransformer InterpretabilityLinear Probing

    Independent Researcher

  20. When LLMs Learn to Be Consistently Wrong: A Multi-Model Study of Linear Representations of Synthetic Deception

    May 28, 2026Vahideh ZolfaghariLinear ProbingLanguage Model Safety Evaluation

    Algoverse AI Research, Algoverse AI Research Mentorship Program, USA. · Medical Sciences Education Research Center, Mashhad University of Medical Sciences, Mashhad, Iran. · Student Research Committee, Department of Health Information Technology and Management, Medical Informatics, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran.

  21. Optimal Representation Size: High-Dimensional Analysis of Pretraining and Linear Probing

    May 19, 2026Valentina Njaradi, Clémentine Dominé, Rachel Swanson +2Neural Network GeneralizationLinear Probing

    Gatsby Computational Neuroscience Unit · University College London · Institute of Science and Technology Austria +1

  22. Probing for Representation Manifolds in Superposition

    May 18, 2026Alexander ModellFeature SuperpositionNeural Representation Geometry

    Department of Mathematics Imperial College London, U.K.

  23. Polar probe linearly decodes semantic structures from LLMs

    May 13, 2026Pablo J. Diego-Simón, Pierre Orhan, Emmanuel Chemla +2Representation LearningLinear Probing

    LSCP, ENS, PSL, EHESS, CNRS, Paris, France · Paris Brain Institute, Paris, France · Earth Species Project, Berkeley, California, USA +1

  24. Closed-Form Linear-Probe Dataset Distillation for Pre-trained Vision Models

    May 8, 2026Bincheng Peng, Guang Li, Ping Liu +2Linear ProbingDataset Distillation

    Hokkaido University · University of Nevada, Reno

  25. Tracing Relational Knowledge Recall in Large Language Models

    Apr 21, 2026Nicholas Popovič, Michael FärberLinear ProbingFactual Knowledge in Language Models

    TU Dresden & ScaDS.AI Dresden/Leipzig, Germany

  26. FLiP: Towards understanding and interpreting multimodal multilingual sentence embeddings

    Apr 20, 2026Santosh Kesiraju, Bolaji Yusuf, Šimon Sedláček +2Linear ProbingSentence Embeddings

  27. What are They Thinking? Delineation, Probing, and Tracking of Concepts in LLMs

    Apr 7, 2026Mohamed Abdelwahab, Michelle Yu Collins, Sihan Chen +5Linear ProbingLLM Interpretability

    The Edward S. Rogers Sr. Department of Electrical and Computer Engineering University of Toronto

  28. Testing the Limits of Truth Directions in LLMs

    Apr 4, 2026Angelos Poulis, Mark Crovella, Evimaria TerziLLM InterpretabilityLinear Probing

    Department of Computer Science, Boston University · Faculty of Computing & Data Sciences, Boston University

  29. Label-efficient underwater image classification with logistic regression on frozen foundation model embeddings

    Mar 31, 2026Thomas Manuel Rost, Martina Figlia, F. Morgado-Dias +1Image ClassificationFew-Shot Image Classification

    University of Madeira, Faculty of Exact Sciences and Engineering, Funchal, Portugal · Interactive Technologies Institute (ITI/LARSyS), ARDITI, Funchal, Portugal · Wave Labs, Faculty of Arts and Humanities, University of Madeira, Portugal +4

  30. On the Dynamics & Transferability of Latent Generalization during Memorization

    Mar 20, 2026Simran Ketha, Venkatakrishnan RamaswamyNeural Network GeneralizationNeural Network Memorization

    Anuradha & Prashanth Palakurthi Centre for Artificial Intelligence Research, Department of Computer Science & Information Systems, Birla Institute of Technology & Science Pilani, Hyderabad 500078, India.