Linear Representation Hypothesis

The Linear Representation Hypothesis studies whether concepts and attributes can be represented by directions in neural activations, so that linear probes can detect them and interventions can steer them. Researchers test this in language and vision models using probing, activation steering, sparse autoencoders, and evaluations of attribute composition and cross-model alignment. Some findings support additive structure, but representations may be overlapping rather than orthogonal, and observed directions can reflect answer probabilities or training dynamics instead of concepts themselves. A central challenge is defining and measuring the hypothesis precisely across models, activation locations, features, and tasks.