Is a Linear Probe Evidence of a Linear Representation?

Nadir Kutluozen; Christian Internò; David Klindt

stat.MLOct 15, 2026

Is a Linear Probe Evidence of a Linear Representation?

Authors: Nadir Kutluozen, Christian Internò, David Klindt

Organizations: Cold Spring Harbor Laboratory · Bielefeld University

Abstract

The linear representation hypothesis holds that pretrained vision models encode generative factors as linearly decodable directions in feature space, and comes with a standard methodology: train a linear probe, declare a factor represented if accuracy is high. We show this test is too permissive. Asymptotically, any sufficiently rich feature map, including a random one, admits a linear readout, so high probe accuracy is consistent with two very different geometries: a true linear world model, where the latent-aligned subspace dominates the representation, or a kernel machine, where the latents lie linearly inside a much larger curved manifold. This distinction matters because the curved structure orthogonal to the probe is relevant for out-of-distribution generalization, robustness, and compositional transfer. We propose three asymmetric diagnostics that distinguish the two cases: reverse predictivity, spectral concentration, and out-of-distribution generalization. We validate them in a fully observed simulation where ground truth is known. Applied to six pretrained vision transformers on SVG-World, our deterministic generative benchmark with paired visual styles, and to two of them on CelebA, they show pretraining produces structure beyond a random kernel but well short of a linear world model. On dSprites, 3D Shapes and MPI3D, forward decodability and latent-aligned variance dissociate. Thus, forward decoding alone does not show that the probed factors align with the dominant geometry of the representation. In summary, our work cautions about the success of linear probes as opposed to truly linear and low dimensional representations.

Figures & tables

Figure 1: Forward Linear Decoding. A 3D latent trajectory $z$ (left) generates images $x$ (middle) through a renderer [ 1 ] . A linear probe on DINOv3-B/16 features recovers the trajectory in latent space (right), inverting the rendering pipeline with a linear map. We ask when recovery is evidence of a linear world model (Def. 1 ) rather than a generic property of any sufficiently rich feature map.

Figure 2: Linear World Model vs. Kernel Machine. (a) 2D latent grid. (b, c) Cartoons: a linear world model lifts the grid to a flat plane (Def. 1 ); a kernel-machine map might lift it to a curved surface. Forward

R^{2}

is high in both; reverse

R^{2}

separates them (Prop. 2 ). (d, e) Empirical realization at

h{=}1024

: horizontal axes are forward-decoded

\hat{z}_{1},\hat{z}_{2}

; vertical is the top variance direction in the decoder’s null space, residualized against

\hat{z}

. Trained is a plane, random a curved manifold.

Figure 3: Diagnostics on the simulation across hidden width $h$ . Random init (orange) reproduces the asymptotic random-features prediction: forward $R^{2}\to 1$ as $h$ grows, while reverse $R^{2}$ and the top-2 share stay low and OOD $R^{2}$ catches up only at large widths. Trained with weight decay $\lambda=1$ (dark blue), all four diagnostics sit flat at $\approx 1$ across widths. Without weight decay (light blue), reverse $R^{2}$ and the top-2 share dip at intermediate widths: the linear-world-model condition is sensitive to training pressure even when forward $R^{2}$ saturates. The third panel is the share of variance in the top two PCs that is linear in $z$ , without whitening (App. B ). Error bars over 3 seeds (initialization and batch order).

Figure 4: Spectral diagnostics on the simulation ( $h=1024$ ; random init vs. trained with $\lambda=1$ ). (a, b) Test points on the top-2 PCs of $f$ , colored by $z_{0}$ . Random folds the latent square into a non-axis-aligned shape (top-2 subspace not $z$ -aligned); trained gives an axis-aligned image, $z_{0}$ along PC 2 and $z_{1}$ along PC 1. (c) Normalized eigenspectrum of $f$ (log-log): trained drops by several orders of magnitude after PC 2, random decays smoothly over hundreds of PCs. (d) Reverse $R^{2}$ on the top $m$ PCs of $f$ , without whitening. Trained stays at 1: its top two PCs are linear in $z$ and carry almost all the variance, so later PCs barely change the ratio. Random falls from $\approx 0.7$ at $m=1$ to a plateau at its full $R^{2}_{\mathrm{rev}}\approx 0.2$ as PCs not linear in $z$ are added.

Figure 5: SVG-World, a deterministic two-style benchmark with a shared latent layout. Both panels show the canonical scene at $z=0$ ; arrows mark each object’s two position latents ( $+x$ horizontal, $+z$ depth, foreshortened by the oblique projection). AquaWorld and WestWorld share the same 32-dim layout (4 people with $(x,z)$ position, facing direction and skin hue; 2 animals with $(x,z)$ position) but differ in rendering, which enables the cross-domain transfer test of § 5.3 .

Figure 6: Forward Decodability Separates Pretrained from Random-Init at

d{=}768

. Per-latent

\rho

on CLS features; per-latent

R^{2}

in Table 4 (App. D ). Pretrained averages

\rho=0.94

(

R^{2}=0.88

); random averages

0.55

(

0.33

) (

\Delta\rho=0.38

, rising to

0.58

on animals).

Figure 7: Reverse Predictivity (D1) and Spectral Concentration (D2) on SVG-World.

R^{2}_{\mathrm{rev}}

vs. PCA-whitening cutoff

k

. Solid: top-

k

PCs whitened to unit variance. Dashed: raw features. Pretrained’s latent-aligned variance concentrates in the top

\sim

32–64 PCs; random’s is diffuse. The curve falls with

k

for any representation (App. E ); what separates the two is its value at small

k

Figure 8: Asymmetric Diagnostic D3: Cross-Domain Transfer. Per-latent test

\rho

on DINOv3-B/16 mean-pooled patch features; cross-domain values average both transfer directions. The pretrained-vs-random gap survives the style change (

\Delta\rho=0.41

within,

0.42

across domains). Animal positions transfer best (pretrained

\rho=0.68

–

0.84

cross-domain; random near zero).

Figure 9: CelebA. (a) Share of the ceiling,

R^{2}_{\mathrm{rev}}/C_{40}

, and (b) spectral concentration; pretrained vs. random init. Error bars: 95% bootstrap for pretrained in (a), s.d. for random (Table 9 ).

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Condition	Fwd $R^{2}$	$R^{2}_{\mathrm{rev}}$	$C_{2}$	Unwhitened top-2	SC	OOD $R^{2}$
Trained, $\lambda=1$	1.000	0.996	0.996	0.995	1.000	1.000
Trained, $\lambda=0$	1.000	0.786	0.797	0.781	0.983	0.996
Random init	1.000	0.213	0.258	0.181	0.713	0.892

Appendix

Table 1: Simulation diagnostics at

h=1024

(mean over 5 seeds). Unwhitened top-2: share of total variance in the top two PCs that is linear in

z

, the statistic plotted as spectral concentration in Fig. 3 . SC: whitened definition of § 3.3 . This run is independent of the one reported in § 4 ; reverse

R^{2}

differs by at most

0.003

Field	Value
Latent vector	$z\in[0,1]^{32}$ ; 20 active, 12 fixed by the renderer
Person $\times 4$	$(x,z)$ position in a fixed lane; facing $\in\{$ S, E, N, W $\}$ ; skin hue $0.05+0.06\,z$ (HSV)
Animal $\times 2$	$(x,z)$ position in a fixed lane (AquaWorld: dolphin, turtle; WestWorld: brown and white horse)
Fixed (12 dims)	shirt HSV per person; excluded from all probes
Sampling	$z\sim\mathcal{U}[0,1]^{32}$ i.i.d., seed 42
Occlusion	objects drawn back to front, so they may overlap

Appendix

Table 2: SVG-World dataset card.

Encoder	Checkpoint	Readout ( $d$ )
DINOv3-S/16	facebook/dinov3-vits16-pretrain-lvd1689m	CLS (384)
DINOv3-B/16	facebook/dinov3-vitb16-pretrain-lvd1689m	CLS or mean patch (768)
DINOv3-L/16	facebook/dinov3-vitl16-pretrain-lvd1689m	CLS (1024)
DINOv2-B/14	facebook/dinov2-base	CLS (768)
DeiT-B/16	facebook/deit-base-patch16-224	CLS (768)
CLIP-B/16	openai/clip-vit-base-patch16	pooled CLS, before projection (768)

Appendix

Table 3: Encoders, checkpoints and readouts.

	CLS token				Mean-pooled patches
	$\rho$		$R^{2}$		$\rho$		$R^{2}$
latent	pre	rand	pre	rand	pre	rand	pre	rand
p0_x	0.97	0.71	0.94	0.47	0.93	0.68	0.85	0.41
p0_z	0.95	0.80	0.89	0.61	0.90	0.76	0.80	0.55
p1_x	0.91	0.49	0.82	0.21	0.82	0.44	0.65	0.16
p1_z	0.95	0.72	0.89	0.50	0.90	0.70	0.80	0.45

Appendix

Table 4: DINOv3-B/16 Forward Decoding on SVG-World, per-dim within-domain. Pearson

\rho

and

R^{2}

from a ridge probe trained on each world separately, averaged over AquaWorld and WestWorld. Pre : pretrained. Rand : freshly initialized, no training. The pretraining contribution is clearest on small foreground objects (animal

\Delta\rho=0.58

on CLS,

0.68

on patches). Position latents only; fixed ridge penalty

\alpha=1

Encoder (readout)	Fwd $R^{2}$	$R^{2}_{\mathrm{rev}}$	Gap	$R^{2}_{\mathrm{rev}}/C_{r}$	SC	$\rho_{\times}$
DINOv3-S/16 (CLS)	0.86 / 0.45	0.257 / 0.023	0.235 [0.229, 0.242]	0.38 / 0.04	0.27 / 0.03	0.47 / 0.09
DINOv3-B/16 (CLS)	0.91 / 0.66	0.188 / 0.023	0.164 [0.160, 0.169]	0.31 / 0.04	0.26 / 0.03	0.46 / 0.10
DINOv3-B/16 (patches)	0.93 / 0.59	0.284 / 0.024	0.259 [0.253, 0.264]	0.47 / 0.05	–	0.57 / 0.15
DINOv3-L/16 (CLS)	0.91 / 0.74	0.169 / 0.024	0.145 [0.139, 0.151]	0.26 / 0.05	0.24 / 0.02	0.48 / 0.06
DINOv2-B/14 (CLS)	0.91 / 0.58	0.224 / 0.028	0.196 [0.190, 0.202]	0.35 / 0.05	0.26 / 0.04	0.34 / 0.07
DeiT-B/16 (CLS)	0.89 / 0.55	0.186 / 0.020	0.165 [0.159, 0.170]	0.27 / 0.04	0.23 / 0.02	0.26 / 0.02

Appendix

Table 5: Encoders on SVG-World (12 position latents,

r=12

; mean over the two styles; random control seed 42). Gap:

R^{2}_{\mathrm{rev}}

pretrained minus random, with 95% paired bootstrap interval. SC: spectral concentration

R^{2}_{\mathrm{rev}}(r)

\rho_{\times}

: mean cross-style correlation over both transfer directions. Each cell: pretrained / random.

Dataset ( $r$ )	Encoder	Fwd $R^{2}$	$R^{2}_{\mathrm{rev}}$	$C_{r}$	$R^{2}_{\mathrm{rev}}/C_{r}$	SC	PR	D3
dSprites (5)	DINOv3-B/16	0.42 / 0.16	0.09 / 0.61	0.81 / 0.82	0.11 / 0.74	0.08 / 0.41	4.4 / 1.9	0.42 / 0.17
	CLIP-B/16	0.30 / 0.04	0.17 / 0.09	0.67 / 0.46	0.25 / 0.20	0.12 / 0.14	6.1 / 18.7	0.31 / 0.04
3D Shapes (8)	DINOv3-B/16	0.96 / 0.96	0.11 / 0.67	0.70 / 0.92	0.15 / 0.72	0.26 / 0.24	8.7 / 3.8	0.95 / 0.96
	CLIP-B/16	0.96 / 0.97	0.15 / 0.90	0.61 / 1.00	0.25 / 0.91	0.17 / 0.32	16.6 / 2.3	0.95 / 0.97
MPI3D-real. (4)	DINOv3-B/16	0.85 / 0.61	0.19 / 0.40	0.35 / 0.96	0.53 / 0.42	0.26 / 0.27	19.8 / 3.1	0.85 / 0.61
	CLIP-B/16	0.76 / 0.51	0.14 / 0.09	0.35 / 0.97	0.41 / 0.09	0.22 / 0.19	20.8 / 2.6	0.77 / 0.51

Appendix

Table 6: Controlled benchmarks (canonical encoding, sample seed 1234; pretrained / mean of five random initializations). D3:

R^{2}

on held-out checkerboard combinations, against random seed 42 only. ResNet-50 uses the data-free control and is reported separately.

Dataset	Encoder	Canonical	Ordinal	With one-hot
dSprites	DINOv3-B/16	$-0.516$	$-0.398$	$-0.287$
	CLIP-B/16	$+0.077$	$+0.097$	$+0.128$
3D Shapes	DINOv3-B/16	$-0.559$	$+0.075$	$-0.094$
	CLIP-B/16	$-0.754$	$-0.073$	$-0.492$
MPI3D-realistic	DINOv3-B/16	$-0.213$	$-0.344$	$-0.478$
	CLIP-B/16	$+0.057$	$-0.213$	$-0.418$

Appendix

Table 7: Sensitivity to the factor encoding.

R^{2}_{\mathrm{rev}}

gap (pretrained minus mean of five random initializations), sample seed 1234.

Dataset	Fwd $R^{2}$	$R^{2}_{\mathrm{rev}}$	$C_{r}$	$R^{2}_{\mathrm{rev}}/C_{r}$
SVG-World (mean of styles)	0.995	0.166	0.412	0.42
dSprites	0.496	0.213	0.489	0.44
3D Shapes	0.998	0.836	0.885	0.94
MPI3D-realistic	0.636	0.389	0.676	0.58

Appendix

Table 8: Pixel-space control. Diagnostics computed on raw images downsampled to

32\times 32\times 3

(

d=3072

), canonical encoding.

Encoder	Features	Weights	Fwd $R^{2}$	$R^{2}_{\mathrm{rev}}$	Std.	$C_{r}$	$R^{2}_{\mathrm{rev}}/C_{r}$	SC	PR
DINOv3-B/16	CLS	pre	0.391	0.170 [0.168, 0.173]	0.160	0.606	0.281	0.153	48.2
		rand	0.069	0.067 $\pm$ 0.003	0.060	0.980	0.069	0.018	2.5
DINOv3-B/16	patches	pre	0.412	0.245 [0.242, 0.248]	0.226	0.680	0.360	0.174	32.1
		rand	0.114	0.090 $\pm$ 0.002	0.081	0.994	0.091	0.023	1.7
CLIP-B/16	image	pre	0.411	0.192 [0.191, 0.195]	0.152	0.694	0.277	0.155	45.6
		rand	0.115	0.106 $\pm$ 0.001	0.097	0.996	0.106	0.028	1.8

Appendix

Table 9: CelebA (20,000 images; 40 binary attributes,

r=40

). Pretrained: 95% bootstrap interval over held-out images; random: mean

\pm

s.d. over five initializations. Std.:

R^{2}_{\mathrm{rev}}

after standardizing each feature dimension. SC: spectral concentration,

R^{2}_{\mathrm{rev}}

on the top

r

whitened PCs. PR: participation ratio.

Figure 10: Two-face regime of SVG-Faces. (a) Example renders; each scene contains two faces with independently sampled latents, ordered left to right by

x

-coordinate. (b) Per-latent test

R^{2}

for pretrained and random-init DINOv3-B/16 (30 latents, 15 per face).

Figure 11: SVG-Faces. Each panel varies three latents of the single-face regime while the others are held fixed: (a) face center $(x,y)$ and radius, (b) eye spacing, vertical offset and radius, (c) mouth vertical offset, width and curvature, and (d) skin hue, saturation and value. Eye color (HSV, not shown) completes the 15 latents. The renderer is deterministic.

Figure 12: DINOv3-B/16 forward decoding on SVG-Faces. (a) Predicted vs. ground-truth values on held-out images for the 15 single-face latents. (b) Round-trip reconstructions: latents decoded from a held-out image and re-rendered through the SVG pipeline.

Figure 13: SVG-Faces: pretrained vs. random initialization. Left: per-latent test $R^{2}$ on the final block. Right: Pearson $\rho$ per transformer block. Random initialization matches the pretrained encoder on face position, radius and skin color, and fails on eye and mouth geometry and eye color.

Figure 14: SVG-Faces: reverse predictivity on DINOv3-B/16. (a) Conceptual setup. (b) Aggregate forward and reverse $R^{2}$ for pretrained and random initialization. (c) Contribution of each (feature dimension, latent) pair to reverse $R^{2}$ .

Figure 15: SVG-Faces: generalization within one style. Per-latent test $R^{2}$ for six latents in three regimes: IID (random split), CompGen (train on one half of a latent’s range, test on the other half) and Extrapolate (train within a sub-range, test outside it). The pretrained encoder extrapolates better on face radius and skin value; both encoders fail on skin hue outside the IID split, and random initialization is ahead on skin saturation.

Representation Geometry in VLMs

See all 21 papers

Feb 27, 2026

Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models

Compositional generalization, the ability to recognize familiar parts in novel contexts, is a defining property of intelligent systems. Although modern models are trained on massive datasets, they still cover only a tiny fraction of the combinatorial space of possible inputs, raising the question of what structure representations must have to support generalization to unseen combinations. We formalize three desiderata for compositional generalization under standard training (divisibility, transferability, stability) and show they impose necessary geometric constraints: representations must decompose linearly into per-concept components, and these components must be orthogonal across concepts. This provides theoretical grounding for the Linear Representation Hypothesis: the linear structure widely observed in neural representations is a necessary consequence of compositional generalization. We further derive dimension bounds linking the number of composable concepts to the embedding geometry. Empirically, we evaluate these predictions across modern vision models (CLIP, SigLIP, DINO) and find that representations exhibit partial linear factorization with low-rank, near-orthogonal per-concept factors, and that the degree of this structure correlates with compositional generalization on unseen combinations. As models continue to scale, these conditions predict the representational geometry they may converge to. Code is available at https://github.com/oshapio/necessary-compositionality.

Arnas Uselis, Andrea Dittadi, Seong Joon Oh

Tübingen AI Center, University of Tübingen · Helmholtz · Technical University of Munich

Neural Representation Geometry Visual Representation Learning Linear Representation Hypothesis Orthogonal Representation Learning Geometric Representation Learning Compositional Representation Learning+1

Ask PDF

Jun 17, 2026

Comparing Linear Probes with Mahalanobis Cosine Similarity

Linear probes are widely used in interpretability research and often compared by cosine similarity. The Mahalanobis cosine similarity (MCS) between two directions, which reweights the inner product by test data covariance, is a natural task-aware refinement. Ying et al. (2026) report that a probe's MCS to a reference probe trained on the out-of-distribution (OOD) data near-perfectly linearly predicts the probe's OOD AUROC (R^2 = 0.98). Here, we extend this empirical finding across models, layers, and concept domains, and prove this general phenomenon in closed form: For balanced classes whose projections are Gaussian, OOD AUROC and MCS to the reference probe are linear because both are sigmoid-shaped functions of the probe's signal-to-noise ratio (SNR) on the test data. The theory also predicts when this linearity fails, which we verify empirically. MCS offers a theoretically grounded and empirically effective alternative to Euclidean cosine similarity for comparing linear probes.

Zhuofan Josh Ying, Peter Hase, Nikolaus Kriegeskorte

Departments of 1Psychology, 2Neuroscience, 3Electrical Engineering · 4Zuckerman Mind Brain Behavior Institute; Columbia University, New York, NY · 5Stanford University, Stanford, CA +1

Linear Probing Mahalanobis Distance OOD Generalization Language Model Probing

Ask PDF

May 12, 2026

Deep Minds and Shallow Probes

Neural representations are not unique objects. Even when two systems realize the same downstream computation, their hidden coordinates may differ by reparameterization. A probe family intended to reveal structure already present in a representation should therefore be stable under the relevant representation symmetries rather than be tied to a particular basis. We prove that, under standard nondegeneracy assumptions, behaviorally equivalent LLMs have hidden representations related by an invertible affine transformation. The resulting symmetry principle singles out a unique hierarchy of shallow coordinate-stable probes, with linear probes as its degree-1 member. We also show that a natural object for cross-model probe transfer is a shared probe-visible quotient--the representation modulo directions invisible to the probe family--rather than the full hidden state. Experiments on synthetic and real-world tasks support both predictions, showing where degree-2 probes help beyond linear ones and how quotient-based transfer enables coverage-aware monitor portability across model families. These results point toward a broader geometric representation theory of neural probing, with coverage-aware monitor transfer as a concrete operational consequence.

Su Hyeong Lee, Risi Kondor

Department of Statistics, University of Chicago. · Department of Statistics and Department of Computer Science, University of Chicago.

Neural Representation Geometry Representation Probing Hidden-State Probing

Ask PDF

Abstract

Representation Geometry in VLMs

Explore similar work

Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models

Comparing Linear Probes with Mahalanobis Cosine Similarity

Deep Minds and Shallow Probes