Hidden-State Probing

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, up 33% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 36

All topics

Deep Minds and Shallow Probes
May 12, 2026Su Hyeong Lee, Risi KondorNeural Representation Geometry Representation Probing
Department of Statistics, University of Chicago. · Department of Statistics and Department of Computer Science, University of Chicago.
Multi-layer attentive probing improves transfer of audio representations for bioacoustics
May 11, 2026Marius Miron, David Robinson, Masato Hagiwara +15Audio Representation Learning Bioacoustics
Earth Species Project
Do Linear Probes Generalize Better in Persona Coordinates?
May 10, 2026Prasad Mahadik, Adrians SkaparsLanguage Model Safety Evaluation Deception in Language Models
Independent Researcher · University of Manchester
Inference Time Causal Probing in LLMs
May 8, 2026Sadegh Khorasani, Saber Salehkaleybar, Negar Kiyavash +1Causal Interventions in Language Models Activation Steering
School of Computer and Communication Sciences, EPFL„ Lausanne, Switzerland · 2Leiden Institute of Advanced Computer Science (LIACS), Leiden University, , Leiden, The Netherlands · College of Management of Technology, EPFL, Lausanne, Switzerland
Probe-Geometry Alignment: Erasing the Cross-Sequence Memorization Signature Below Chance
May 3, 2026Anamika Paul Rupa, Anietie AndyMemorization in Language Models Neural Network Memorization
Howard University
Tracing the Dynamics of Refusal: Exploiting Latent Refusal Trajectories for Robust Jailbreak Detection
May 2, 2026Xulin Hu, Che Wang, Wei Yang Bryan Lim +2Adversarial Attacks on LLMs LLM Refusal Behavior
1Peking University · 2Nanyang Technological University · 3Beijing Jiaotong University

May 12, 2026, cs.LGJ/K move · Enter open · S save

Deep Minds and Shallow Probes

Su Hyeong Lee, Risi Kondor

Department of Statistics, University of Chicago. · Department of Statistics and Department of Computer Science, University of Chicago.

Neural Representation Geometry Representation Probing Hidden-State Probing

Open paper AskPDF arXiv

Neural representations are not unique objects. Even when two systems realize the same downstream computation, their hidden coordinates may differ by reparameterization. A probe family intended to reveal structure already present in a representation should therefore be stable under the relevant representation symmetries rather than be tied to a particular basis. We prove that, under standard nondegeneracy assumptions, behaviorally equivalent LLMs have hidden representations related by an invertible affine transformation. The resulting symmetry principle singles out a unique hierarchy of shallow coordinate-stable probes, with linear probes as its degree-1 member. We also show that a natural object for cross-model probe transfer is a shared probe-visible quotient--the representation modulo directions invisible to the probe family--rather than the full hidden state. Experiments on synthetic and real-world tasks support both predictions, showing where degree-2 probes help beyond linear ones and how quotient-based transfer enables coverage-aware monitor portability across model families. These results point toward a broader geometric representation theory of neural probing, with coverage-aware monitor transfer as a concrete operational consequence.

Momentum

Related topics

Latest papers 36

Deep Minds and Shallow Probes

Multi-layer attentive probing improves transfer of audio representations for bioacoustics

Do Linear Probes Generalize Better in Persona Coordinates?

Inference Time Causal Probing in LLMs

Probe-Geometry Alignment: Erasing the Cross-Sequence Memorization Signature Below Chance

Tracing the Dynamics of Refusal: Exploiting Latent Refusal Trajectories for Robust Jailbreak Detection