cs.LGSep 26, 2026

Intuition vectors

Authors: Shahar Haim, Daniel C. McNamee

Organizations: Champalimaud Research, Centre for the unknown, Lisbon, Portugal

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large self-supervised vision models learn representations that support scene segmentation and the semantic decomposition of physical objects. We ask whether their representational geometry supports transfer to visual reasoning problems without any task-specific fine-tuning. We hypothesized that relational representations may bridge perception and abstract reasoning by encoding similarities and transformations among visual inputs such that an intuitive, implicit form of reasoning may be performed via latent vector arithmetic. Specifically, we examine DINOv3, MAE, and random pixel projections on abstract and naturalistic Bongard problems, ARC-AGI-1 and ARC-AGI-2, and novel ARC-GEN instances. On both Bongard benchmarks, the accuracy of a simple nearest-centroid readout of frozen visual embeddings is within four percentage points of the task-specific baselines reported with the original benchmarks. In ARC, latent difference vectors summarizing demonstration input-output transformations, which we refer to as intuition vectors, show greater alignment with test vectors from the same task, whereas those from unrelated tasks are near orthogonal. This latent geometry is operational: transporting a query along its intuition vector consistently improves exact-output retrieval, reaching 70.7 on ARC-AGI-2 evaluation. Across 397,000 ARC-GEN instances from 794 tasks, single-pair intuition vectors identify the generating task with approximately 87% leave-one-out accuracy. These findings suggest that latent vector arithmetic over frozen visual representations supports implicit rule inference across varied problem domains without a generative model component, indicating that inferring an abstract transformation and generating its instance-specific consequence may be separable capacities.

Figures & tables

Figure 1: Visual concept discrimination in frozen representation spaces on Bongard–LOGO and Bongard-OpenWorld. (a) The two support sets define class centroids, and a held-out query is assigned to its nearer centroid. (b, c) Representative Bongard-LOGO and Bongard-OpenWorld problems. (d, e) Best observed nearest-centroid accuracy for DINOv3, a size-matched MAE, and random-projection pixel controls at 224 and 480 px. For each encoder, dataset, and resolution, the displayed layer maximizes dataset-level evaluation accuracy. LOGO values are equal-weight means across its four disjoint test subsets; OpenWorld values pool 200 problems. Error bars show SEM across problems for encoders and the mean problem-level SEM across ten projections for pixel projections. Gray lines mark external published references (LOGO: 67.2%67.2\% ( Nie et al., 2020 ) ; OpenWorld: 64.0%64.0\% ( Wu et al., 2024 ) ); dashed lines mark 50%50\% chance. Per-subtask results and full layer profiles appear in Supp. Fig. S1 and S2 , respectively.
Figure 2: Intuition vector alignment in frozen DINOv3 and MAE representations of ARC-1 and ARC-2. (a) The intuition vector connects the support-input and support-output centroids; the test vector connects the held-out query and answer. (b) A task with three support pairs, its held-out test pair, and a cell-shuffled answer control, which preserves grid shape and color counts while disrupting spatial structure. (c,d) Intuition to test vector angles at layer 1919 for both encoders at 224 and 480 px, alongside resolution-matched random-projection pixel controls. Bars compare matched tasks with cross-task and shuffled-output controls, each averaged over 10 randomizations. Filled bars denote evaluation tasks and open hatched bars training tasks; ARC-1 has n=400n=400 tasks per dataset, and ARC-2 has n=1,000n=1{,}000 training and n=120n=120 evaluation tasks. Bars show mean ±\pm SEM across tasks; the dotted line marks orthogonality. Panel (c) provides the key for both panels. Layer 1919 was selected globally to maximize separation from both controls; it is within 1.9∘1.9^{\circ} of every setting’s individual optimum. Pixel controls are layer-independent. Full layer-wise results in Supp. Fig. S3 , S4 .
Figure 3: ARC output retrieval. (a) The inferred intuition vector transports the held-out query representation, and all outputs in the same dataset are ranked by distance to the transported query. (b) Candidate pool is the true output (outlined) competing with every other output in that dataset; four alternatives are shown. (c, d) Top-1 retrieval accuracy across the 24 transformer blocks on the ARC-1 and ARC-2 evaluation sets. Curves compare DINOv3 and a size-matched MAE at 224 and 480 px; flat lines are the corresponding layer-independent random-projection pixel baselines. Solid lines use the translated query; dotted lines use the original query, isolating the contribution of the intuition vector. Training set results appear in Supp. Fig. S6 . (e) ARC-GEN retrieval from a 368,442 grid candidate pool,across all 24 layers. Markers identify selected encoder layers. (f) ARC-GEN remaining distance between the transported query and answer, relative to the original query-answer distance: the median of task medians with 95%95\% task-bootstrap intervals. Lower is better, 11 denotes no improvement over the original query.
Figure 4: Task properties associated with intuition-vector gain and pixel advantage on ARC-GEN. All panels analyse 794 tasks with 100 generated queries each; encoder layers were selected on original ARC queries and held fixed. (a,b) Baseline retrieval (without the intuition vector) and its gain from adding the vector, in percentage points, against the fraction of canvas cells that change. Eight shared, fixed, near-equal-count bins use predictors computed from separate clean instances; shading shows marginal 95%95\% task-bootstrap intervals. (c,d) Spearman correlations between task properties and transported pixel advantage (pixel minus resolution-matched encoder retrieval); positive values favour pixels. Panel (c) uses task-level means and panel (d) between-instance variation. Whiskers show marginal 95%95\% task-bootstrap intervals, and ∗* denotes Bonferroni-adjusted two-sided Spearman p<.05p<.05 across the 40 correlations (10 task-property summaries ×\times four pixel-encoder comparisons).
Figure 5: Population-level organization of single instance ARC-GEN difference vectors (794 tasks; 500 instances each; 397,000 total) and its dissociation from exact-output retrieval. DINOv3, MAE, and random-projection pixel controls were evaluated at 224 and 480 px. (a) Cross tasks difference vector schematic. (b) Leave-one-out nearest-centroid accuracy across model layers. Markers denote layers selected independently by retrieval on the original ARC evaluation pairs; pixel controls are layer-independent. (c) Per-task accuracy distributions at these layers; legends report means. (d) Per-task retrieval and classification performance correlation.
Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1: Per-subtask nearest-centroid accuracy on the Bongard benchmarks, at the settings selected for Figure 1 (d,e). (a,b) The four disjoint Bongard–LOGO test subsets at 224 and 480 pixels. (c,d) The Bongard–OpenWorld subgroups at 224 and 480 pixels. Bars compare DINOv3, a MAE encoder of matched size, and a random-projection pixel control; the legend in (a) applies to every panel. The encoder layer is the one that maximizes the dataset-level mean within each dataset and resolution and is then held fixed across all subtasks, so no subtask is selected on its own; the layer choice is a descriptive maximum on the evaluation data and the intervals shown do not include selection uncertainty. Error bars are the SEM across problems for the encoders and, for pixels, a conservative upper bound on the SEM of the mean over ten projection seeds. The dashed line marks 50%50\% chance. No published reference line is drawn, because published values are reported per subset rather than for these groupings. The OpenWorld subgroups form two overlapping partitions of the same 200 problems – concept length (short/long) and commonsense requirement – and must not be averaged together. The count beneath each group is its number of problems; every problem contributes two held-out decisions, one per class.
Figure S2: Nearest-centroid accuracy across all 24 transformer blocks, for DINOv3 (left column) and a MAE encoder of matched size (right column). (a–d) Bongard–LOGO at 224 and 480 pixels, with one curve per disjoint test subset and a black curve for their equal-weight mean. (e–h) Bongard–OpenWorld at 224 and 480 pixels, with one curve per subgroup and a black curve for all 200 problems. The OpenWorld subgroups form two overlapping partitions of the same problems – concept length and commonsense requirement – so their curves are not independent strata. The legend in (a) applies to every LOGO panel and the one in (e) to every OpenWorld panel. Shading denotes the SEM across problems. The gray line marks the best published average (LOGO 67.2%67.2\% , the equal-weight mean of the per-subset maxima in ( Nie et al., 2020 ) ; OpenWorld 64.0%64.0\% ( Wu et al., 2024 ) ) and the dashed line marks 50%50\% chance. All panels share the same axes. The bars in Figure 1 (d,e) report the maximum of each black curve; because that layer is chosen on the same evaluation data, the peaks shown here are descriptive maxima rather than independently selected estimates.
Figure S3: Intuition–test-vector angles across all 24 transformer blocks on ARC-1. Green curves show matched tasks, red curves cross-task controls, and orange curves cell-shuffled answers. Solid and dashed curves denote the evaluation and training sets, respectively; shading shows the SEM across tasks and the dotted line marks 90∘90^{\circ} . Both sets contain n=400n=400 tasks. Cross-task and shuffled-output controls average ten randomizations. Figure 2 (c) reports the corresponding values at the globally selected layer 19.
Figure S4: Intuition–test-vector angles across all 24 transformer blocks on ARC-2, using the same conventions as Figure S3 . The training set contains n=1,000n=1{,}000 tasks and the evaluation set n=120n=120 , producing wider evaluation uncertainty. Figure 2 (d) reports the corresponding values at the globally selected layer 19.
Figure S5: Intuition–test-vector angles in random-projection pixel space for ARC-1 and ARC-2 at 224 and 480 px. Pixel representations have no layer axis and are therefore shown categorically. Green bars show matched tasks, red bars cross-task controls, and orange bars cell-shuffled answers. Filled bars denote evaluation tasks and open hatched bars training tasks. Bars average ten projection seeds; error bars combine the mean within-seed task SEM with uncertainty across projection seeds. Results at 224 and 480 px differ by at most 0.2∘0.2^{\circ} .
Figure S6: Output retrieval on the ARC training sets, matching Figure 3 (c,d) panel for panel. Top-1 accuracy across all 24 transformer blocks, comparing DINOv3 and a MAE encoder of matched size at both rendering resolutions, with the layer-independent random-projection pixel controls drawn as flat lines. Solid curves add the intuition vector to the held-out query and dotted curves rank against the unperturbed query. The training sets are larger than the evaluation set, at 400 against 400 tasks for ARC-1 and 1,000 against 120 for ARC-2, so these curves are the smoother estimate of the same effect; neither set was used to select a layer, an encoder or a resolution.
Figure S7: Additional ARC-GEN retrieval and classification diagnostics. DINOv3, MAE, and random-projection pixel controls are compared at 224 and 480 px. Retrieval uses 100 generated queries from each of 794 tasks and a 368,442-grid candidate pool; encoder layers were selected on original ARC queries and then held fixed. (a) Exact-output retrieval without the intuition vector (pale) and with it (solid) at each encoder’s selected layer, alongside the pixel controls. Whiskers are marginal 95%95\% task-bootstrap intervals. (b) Pairwise Spearman correlations between representations’ task-wise transported retrieval rates. (c) Standardized task properties for the two pixel-space extremes in Figure 5(d) : high classification with low retrieval ( ≥90%\geq 90\% and ≤10%\leq 10\% ; n=109n=109 ) and the reverse ( n=60n=60 ). Points and whiskers show group means and 95%95\% bootstrap intervals; asterisks denote Bonferroni-corrected two-sided Mann–Whitney p<.05p<.05 .
Figure S8: Nearest-neighbor discrimination after intuition-vector translation on noisy ARC-1 and ARC-2 for DINOv3 (480 px) and 10 random projections of pixel-space. (a,b) Top-1 retrieval of the true answer from a 100-grid pool containing the answer and corrupted variants, shown as a function of the fraction of corrupted cells. Colors identify sampled model layers; error bars are binomial SEM estimates. For pixelsbars are mean binomial SEM combined with the SEM across seeds.
Figure S9: Primitive transformations in DINOv3-Large at 480 px compose locally but depend on their source configuration. Layer numbers are zero-indexed. (a,d) Synthetic single-cell example designs without and with shared context. (b) Layer-17 geometry of four unit translations and eight recolorings from blue. (c) Layer-wise prediction of individual joint transformations from family-average translation and recoloring vectors, with shuffled controls. (e) Similarity between contextual primitives and their empty-grid counterparts by transformation family and color context. (f) Joint contextual transformations predicted using primitives measured in the same context, on an empty grid, or with an incorrect contextual recoloring. Blue and orange denote cosine similarity and relative residual, respectively; bands and error bars denote 95% confidence intervals.
Figure S10: Cell-addition vectors in DINOv3-Large at 480 px are context-dependent and directionally aligned, but not magnitude-additive. Layer numbers are zero-indexed. (a) The same cell is added to an empty or occupied grid. (b) Similarity between isolated and contextual addition vectors as occupancy increases, separated by color context. (c) Multiple additions measured separately and jointly from a shared source context. (d) The joint addition is compared with sums of context-conditioned vectors, isolated additions, and a wrong-location control. Blue and orange denote cosine similarity and relative residual, respectively; bands and error bars denote 95% confidence intervals.
Figure S11: Additional analyses of context-dependent single-cell geometry in DINOv3-Large at 480 px. Position decoding uses four-fold grouped cross-validation over independently sampled spatial layouts, with all color variants of a layout confined to the same fold. Layerwise analyses use a balanced subset containing every unit direction in each of four placement sets. The transfer analyses use five context cells; the multi-addition analysis uses five additions from a one-cell base. Bands and error bars denote 95% spatial-bootstrap confidence intervals. Layer numbers are zero-indexed.
More in this topic

Latent Visual Reasoning

Latent visual reasoning studies how multimodal models use compact internal visual states to solve perception, spatial, and temporal tasks.

See all 88 papers

Explore similar work

CardsList
Jul 31, 2026

TraceViT: Grounded Trace Supervision for Visual Abstract Reasoning

The Abstraction and Reasoning Corpus (ARC) tests whether a model can infer an unseen transformation from a few input-output examples and apply it to a new grid. Looped visual reasoners refine predictions over multiple iterations, but conventional training constrains only the final output, leaving intermediate refinements unconstrained. We propose that these refinements should instead follow the transformation step by step. We introduce TraceViT, a looped visual reasoner trained with semantically monotonic transformation chains. We obtain these chains by rewriting and verifying programmatic task implementations, decomposing each solution into intermediate grid states. Each iteration is grounded by a task reference derived from the few-shot demonstrations and an object workspace representing the current grid state. Because these chains may differ in length from the loop, soft trace alignment enforces only their ordering, letting the model allocate iterations freely. TraceViT achieves 67.8% pass@2 on ARC-AGI-1 and 24.3% on ARC-AGI-2. Controlled ablations on ARC-AGI-1 show that trace supervision becomes beneficial only when paired with grounding. Code and data will be available at https://github.com/LiuBinnan/TraceViT.

Binnan Liu, Yechi Ma, Tian Xie +1

1Zhejiang University · 2Zhejiang Lab

Visual ReasoningVisually Grounded Reasoning
AskPDF
May 4, 2026

Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs

Continuous latent-space reasoning offers a compact alternative to textual chain-of-thought for multimodal models, enabling high-dimensional visual evidence to be integrated without explicit reasoning tokens. However, we identify a previously overlooked optimization pathology in existing latent visual reasoning methods: although visual latents become semantically enriched during training, their contribution to final answer prediction is systematically suppressed. Within the shared parameter space, the autoregressive objective favors shortcut reliance on direct visual input, driving latent tokens toward transition-like states rather than informative reasoning content. We term this phenomenon Silenced Visual Latents. To address it, we disentangle the two conflicting objectives by directly optimizing the latent reasoning at inference time, keeping backbone parameters frozen. In Stage I, visual latents are warmed up via query-guided contrastive latent--visual alignment, improving semantic quality while preventing latent collapse. In Stage II, the latent reasoning is further optimized via a confidence-progression reward, which incentivizes predicted token distributions along the latent span to become progressively more concentrated, routing predictions through the latent reasoning rather than bypassing it. Experiments across eight benchmarks and four model backbones show that inference-time latent optimization, without any parameter updates, effectively unleashes the suppressed reasoning capacity of visual latents.

Xin Zhang, Qiqi Tao, Jiawei Du +2

Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore · Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore · Singapore University of Technology and Design +1

Inference-Time OptimizationTest-Time OptimizationMultimodal Large Language ModelsLatent Visual ReasoningVLM ReasoningContinuous Latent Reasoning+4
AskPDF
Feb 5, 2026

VRIQ: Benchmarking and Diagnosing the Visual-Reasoning IQ of Vision-Language Models

Vision-language model (VLM) errors on visual reasoning may come from missing visual structure, from mistakes after that structure is extracted, or from both. We introduce VRIQ, a benchmark of 1,391 human-verified IQ-style visual reasoning puzzles spanning five psychometric categories, each instantiated in category-matched abstract and natural-image domains. To study perception and reasoning more directly, we also introduce DiagVRIQ, a 209-item diagnostic subset paired with human-verified, rule-neutral textual descriptions. Across 21 VLMs, the 17 non-frontier models reach a 26.8% unweighted category mean on abstract puzzles, while three 2026 frontier models improve substantially but unevenly, with unweighted category means of 41.3--70.9% on abstract puzzles and at most 78.2% on natural-image puzzles; all remain below human performance. On DiagVRIQ, even when their descriptions are judged correct, the strongest models still answer approximately 9--12% of those items incorrectly. Two interventions, ground-truth description augmentation and tool-augmented inference, improve accuracy, with the largest tool gains on spatially demanding categories, but neither eliminates errors. VRIQ and DiagVRIQ provide a framework for studying visual reasoning through complementary end-to-end, description-based, and intervention-based analyses rather than treating it as a single monolithic capability.

Tina Khezresmaeilzadeh, Jike Zhong, Konstantinos Psounis

University of Southern California, Los Angeles, USA

Visual ReasoningVLM ReasoningVLM Evaluation
AskPDF