cs.CLNov 11, 2026

A Closer Look at Language Model Introspection: Probing Information and Reporting

Authors: Dustin Rubin, Vincenzo Timmel, Xufeng He, Aritra Saha, JooMan Kim, Vijay Madisetti

Organizations: Georgia Institute of Technology, Atlanta, GA 30332, USA

Abstract

One way to study language-model introspection is to test whether models can identify where their activations have been perturbed. We train a simple linear classifier to identify the perturbed sentence from the model's activations, then compare its predictions with the model's own answers across pretraining and injection strengths. This tests whether the model is being asked to report information that is already easy to recover. If so, successful localization may be less demanding than it first appears and may not extend to the broader uses that motivate introspection research, such as faithfully explaining a model's reasoning or helping researchers assess the reliability of its answers. With an injection four times the typical activation norm at layer 0 and probe readout at layer 24 in OLMo 3 32B, the supervised probe reaches 98.3 % accuracy after 8.4 billion training tokens, while the model's zero-shot answer remains at chance (11.1 %). At the released base checkpoint, these accuracies reach 99.1 % and 89.6 %, respectively. Across 7B and 32B, location is already decodable in randomly initialized networks, showing that the architecture can support these traces without pretraining. In these experiments, location information lies close enough to the surface for a simple linear probe to recover it, so successful reporting need not imply broader access to the model's internal computation.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. A mechanistic study of language model introspection

    Sep 28, 2026Jiahong Zou, Xiangkun Sun, Lingkai Kong +1Attention Head AnalysisLLM Interpretability

  2. Latent Introspection: Models Can Detect Prior Concept Injections

    Feb 23, 2026Theia Pearson-Vogel, Martin Vanek, Raymond Douglas +1Language Model IntrospectionMetacognition in Language Models