cs.CVNov 14, 2026

Beyond the Training Cameras: Coverage-Controlled Probes of Viewing Direction in Frozen Vision Models

Authors: Peize Ding

Organizations: Columbia University

Abstract

Vision models are trained on 2D images of a 3D world. Do their hidden features encode the direction from which the camera views a scene, and if so, in what form? We study the relative viewing direction αα of two objects in rendered scenes where both object centres stay at fixed pixels while the camera moves, so that perspective, shading and shadows change but the projected layout does not. In eight pretrained models, from diffusion models to self-supervised and image-text encoders, αα is encoded in a strikingly simple form: a single two-dimensional plane in the hidden space, shared across scenes, in which the difference between the two objects' features traces (cos⁡α,sin⁡α)(\cosα, \sinα). The plane keeps its orientation when lighting, colours, floors and object shapes change. This encoding is more than a summary of a few visible measurements: readouts fitted on one set of camera paths stay accurate at camera positions outside their range in seven of the eight models, whereas decoders of the measured object and shadow geometry, equally accurate within that range, fail there. In the diffusion transformer PixArt-αα, the plane forms abruptly at a single block and carries direction information to later blocks, yet editing it changes the objects rather than the generated viewpoint. We also examine how this encoding carries over to scanned real objects.

Figures & tables

Explore similar work

CardsList
  1. Readout Blindness: VLM Scores Miss the Spatial Direction Their Frozen Encoders Retain

    Oct 5, 2026Guangyuan Li, Tianming Du, Yan Jiang +2VLM EvaluationVision-Language Grounding

  2. Do vision models know that three is three?

    Oct 26, 2026Ankit Sonthalia, Arnas Uselis, Seong Joon OhLinear Representation HypothesisVisual Representation Learning