Representation Geometry in VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Latest papers

18 papers5 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Is a Linear Probe Evidence of a Linear Representation?
Oct 15, 2026Nadir Kutluozen, Christian Internò, David KlindtRepresentation Geometry in VLMs Linear Representation Hypothesis
Cold Spring Harbor Laboratory · Bielefeld University
GeoReader: Learning Latent Geometry for Spatial Reasoning in Vision-Language Models
Oct 13, 2026Rong Li, Shijie Li, Jiangyong Huang +7Geospatial QA Visual Spatial Reasoning
The Hong Kong University of Science and Technology (Guangzhou) · Beijing Institute for General Artificial Intelligence · Agency for Science, A*STAR +3
Feature Competition and Its Representational Origins in Vision Language Models: Evidence from the Stroop Paradigm
Oct 8, 2026Nurbüke Teker, Rui Xiao, Zeynep Akata +1Representation Geometry in VLMs Vision-Language Models
Technical University of Munich · Munich Center for Machine Learning · University of Washington +1
GeoPID: Decomposing and Steering Visual Information in Vision-Language Models
Oct 6, 2026Seulgi Kim, Zhixiong Zhang, Xinwei Zhang +2Vision-Language Grounding Representation Geometry in VLMs
Georgia Institute of Technology · Work done as an intern at Amazon. · Amazon
Are In-Context Images Worth 10 Dimensions?
Sep 29, 2026Adhemar de Senneville, Xavier Bou, Jérémy Anger +2Vision-Language Models Multimodal ICL
Université Paris-Saclay, CNRS, ENS Paris-Saclay, Centre Borelli, Paris, France · Pôle recherche de l’AMIAD, Palaiseau, France · Institut Universitaire de France, Paris, France
Fisher-IRG: Fisher-Induced Local Invariant Representation Geometry across Language and Vision Models
Sep 29, 2026Abdullah All Tanvir, Xin ZhongInformation Geometry Neural Representation Geometry
Department of Computer Science University of Nebraska Omaha, Omaha, NE, USA
Geometric Encoding for Spatial Reasoning in Vision-Language Models
Sep 28, 2026Antonio Jun, Haoshui Yu, Zhengyi Lu +2Visual Spatial Reasoning Representation Geometry in VLMs
School of Arts and Sciences New York University New York City, United States · Dept. of Engineering and Computer Science Oakland University Rochester, United States
Seeing Before Answering: Training-Free Visual Layer Profiling for Vision-Language Models
Aug 17, 2026Ruchen Liu, Yi Yang, Yiming Xu +3Vision-Language Models VLM Evaluation
Leibniz Universität Hannover, Germany · University of Bath, UK
Through the LENS: Local Geometric Decomposition of Vision-Language Model Representations
Aug 1, 2026Shalom Kachko, Raz Lapid, Margarita Vald +2Vision-Language Models Representation Geometry in VLMs
1Ben-Gurion University of the Negev · 2Intuit Inc. · 3Deepkeep
Prior Directions: Why GUI Grounding Gets Locked in the Past
Jul 29, 2026Weile Gong, Zijian Lu, Mingcai Chen +3Vision-Language Models Representation Geometry in VLMs
Nanjing University of Posts and Telecommunications, Nanjing, China
Dive Into the Implicit Biases of Low-rank Vision-language Alignment
Jul 9, 2026Mingjia Shi, Shuo Wang, Xiaobo Wang +7Vision-Language Models Representation Geometry in VLMs
Mininglamp · Shenzhen University of Advanced Technology · National University of Singapore
HyperVis: Continuous Latent Visual Relational Graphs on the Lorentz Hyperboloid for Compositional Reasoning
Jun 4, 2026Moshiur Farazi, Sameera Ramasinghe, Mahbub Ahmed Turza +1Hyperbolic Representation Learning Visual Reasoning
Data Science and AI, University of Doha for Science and Technology, Qatar · Pluralis Research, Australia · Department of Electrical and Computer Engineering, North South University, Bangladesh
Why Far Looks Up: Probing Spatial Representation in Vision-Language Models
May 28, 2026Cheolhong Min, Jaeyun Jung, Daeun Lee +5VLM Evaluation Spatial Reasoning Benchmarks
Seoul National University · The Ohio State University · NVIDIA
3D Primitives are a Spatial Language for VLMs
May 12, 2026Junze Liu, Kun Qian, Florian Dubost +8Visually Grounded Reasoning 3D Spatial Reasoning
Unity Technologies
When Language Overwrites Vision: Over-Alignment and Geometric Debiasing in Vision-Language Models
May 7, 2026Harshvardhan Saini, Samyak Jha, Yiming Tang +1VLM Hallucination Mitigation Vision-Language Models
Indian Institute of Technology Dhanbad · National University of Singapore
Why MLLMs Struggle to Determine Object Orientations
Apr 14, 2026Anju Gopinath, Nikhil Krishnaswamy, Bruce DraperRepresentation Geometry in VLMs Vision-Language Models
Department of Computer Science, Colorado State University, Fort Collins, CO, USA
Do Foundation Models Know Geometry? Probing Frozen Features for Continuous Physical Measurement
Mar 6, 2026Yakov Pyotr ShkolnikovRepresentation Geometry in VLMs Vision-Language Models
Same Answer, Different Representations: Hidden instability in VLMs
Feb 6, 2026Farooq Ahmad Wani, Alessandro Suglia, Rohit Saxena +6VLM Evaluation VLM Robustness
Sapienza University of Rome · University of Edinburgh · University of Oxford +5

Oct 15, 2026J/K move · Enter open · S save

Is a Linear Probe Evidence of a Linear Representation?

Nadir Kutluozen, Christian Internò, David Klindt

Cold Spring Harbor Laboratory · Bielefeld University

Representation Geometry in VLMs Linear Representation Hypothesis Linear Probing Visual Representation Learning

Open paper AskPDF arXiv

The linear representation hypothesis holds that pretrained vision models encode generative factors as linearly decodable directions in feature space, and comes with a standard methodology: train a linear probe, declare a factor represented if accuracy is high. We show this test is too permissive. Asymptotically, any sufficiently rich feature map, including a random one, admits a linear readout, so high probe accuracy is consistent with two very different geometries: a true linear world model, where the latent-aligned subspace dominates the representation, or a kernel machine, where the latents lie linearly inside a much larger curved manifold. This distinction matters because the curved structure orthogonal to the probe is relevant for out-of-distribution generalization, robustness, and compositional transfer. We propose three asymmetric diagnostics that distinguish the two cases: reverse predictivity, spectral concentration, and out-of-distribution generalization. We validate them in a fully observed simulation where ground truth is known. Applied to six pretrained vision transformers on SVG-World, our deterministic generative benchmark with paired visual styles, and to two of them on CelebA, they show pretraining produces structure beyond a random kernel but well short of a linear world model. On dSprites, 3D Shapes and MPI3D, forward decodability and latent-aligned variance dissociate. Thus, forward decoding alone does not show that the probed factors align with the dominant geometry of the representation. In summary, our work cautions about the success of linear probes as opposed to truly linear and low dimensional representations.