cs.LGSep 26, 2026
SaveIntuition vectors
Organizations: Champalimaud Research, Centre for the unknown, Lisbon, Portugal
Abstract
Large self-supervised vision models learn representations that support scene segmentation and the semantic decomposition of physical objects. We ask whether their representational geometry supports transfer to visual reasoning problems without any task-specific fine-tuning. We hypothesized that relational representations may bridge perception and abstract reasoning by encoding similarities and transformations among visual inputs such that an intuitive, implicit form of reasoning may be performed via latent vector arithmetic. Specifically, we examine DINOv3, MAE, and random pixel projections on abstract and naturalistic Bongard problems, ARC-AGI-1 and ARC-AGI-2, and novel ARC-GEN instances. On both Bongard benchmarks, the accuracy of a simple nearest-centroid readout of frozen visual embeddings is within four percentage points of the task-specific baselines reported with the original benchmarks. In ARC, latent difference vectors summarizing demonstration input-output transformations, which we refer to as intuition vectors, show greater alignment with test vectors from the same task, whereas those from unrelated tasks are near orthogonal. This latent geometry is operational: transporting a query along its intuition vector consistently improves exact-output retrieval, reaching 70.7 on ARC-AGI-2 evaluation. Across 397,000 ARC-GEN instances from 794 tasks, single-pair intuition vectors identify the generating task with approximately 87% leave-one-out accuracy. These findings suggest that latent vector arithmetic over frozen visual representations supports implicit rule inference across varied problem domains without a generative model component, indicating that inferring an abstract transformation and generating its instance-specific consequence may be separable capacities.
Figures & tables
Figure 1: Visual concept discrimination in frozen representation spaces on Bongard–LOGO and Bongard-OpenWorld. (a) The two support sets define class centroids, and a held-out query is assigned to its nearer centroid. (b, c) Representative Bongard-LOGO and Bongard-OpenWorld problems. (d, e) Best observed nearest-centroid accuracy for DINOv3, a size-matched MAE, and random-projection pixel controls at 224 and 480 px. For each encoder, dataset, and resolution, the displayed layer maximizes dataset-level evaluation accuracy. LOGO values are equal-weight means across its four disjoint test subsets; OpenWorld values pool 200 problems. Error bars show SEM across problems for encoders and the mean problem-level SEM across ten projections for pixel projections. Gray lines mark external published references (LOGO: ( Nie et al., 2020 ) ; OpenWorld: ( Wu et al., 2024 ) ); dashed lines mark chance. Per-subtask results and full layer profiles appear in Supp. Fig. S1 and S2 , respectively.
Figure 2: Intuition vector alignment in frozen DINOv3 and MAE representations of ARC-1 and ARC-2. (a) The intuition vector connects the support-input and support-output centroids; the test vector connects the held-out query and answer. (b) A task with three support pairs, its held-out test pair, and a cell-shuffled answer control, which preserves grid shape and color counts while disrupting spatial structure. (c,d) Intuition to test vector angles at layer for both encoders at 224 and 480 px, alongside resolution-matched random-projection pixel controls. Bars compare matched tasks with cross-task and shuffled-output controls, each averaged over 10 randomizations. Filled bars denote evaluation tasks and open hatched bars training tasks; ARC-1 has tasks per dataset, and ARC-2 has training and evaluation tasks. Bars show mean SEM across tasks; the dotted line marks orthogonality. Panel (c) provides the key for both panels. Layer was selected globally to maximize separation from both controls; it is within of every setting’s individual optimum. Pixel controls are layer-independent. Full layer-wise results in Supp. Fig. S3 , S4 .
Figure 3: ARC output retrieval. (a) The inferred intuition vector transports the held-out query representation, and all outputs in the same dataset are ranked by distance to the transported query. (b) Candidate pool is the true output (outlined) competing with every other output in that dataset; four alternatives are shown. (c, d) Top-1 retrieval accuracy across the 24 transformer blocks on the ARC-1 and ARC-2 evaluation sets. Curves compare DINOv3 and a size-matched MAE at 224 and 480 px; flat lines are the corresponding layer-independent random-projection pixel baselines. Solid lines use the translated query; dotted lines use the original query, isolating the contribution of the intuition vector. Training set results appear in Supp. Fig. S6 . (e) ARC-GEN retrieval from a 368,442 grid candidate pool,across all 24 layers. Markers identify selected encoder layers. (f) ARC-GEN remaining distance between the transported query and answer, relative to the original query-answer distance: the median of task medians with task-bootstrap intervals. Lower is better, denotes no improvement over the original query.
Figure 4: Task properties associated with intuition-vector gain and pixel advantage on ARC-GEN. All panels analyse 794 tasks with 100 generated queries each; encoder layers were selected on original ARC queries and held fixed. (a,b) Baseline retrieval (without the intuition vector) and its gain from adding the vector, in percentage points, against the fraction of canvas cells that change. Eight shared, fixed, near-equal-count bins use predictors computed from separate clean instances; shading shows marginal task-bootstrap intervals. (c,d) Spearman correlations between task properties and transported pixel advantage (pixel minus resolution-matched encoder retrieval); positive values favour pixels. Panel (c) uses task-level means and panel (d) between-instance variation. Whiskers show marginal task-bootstrap intervals, and denotes Bonferroni-adjusted two-sided Spearman across the 40 correlations (10 task-property summaries four pixel-encoder comparisons).
Figure 5: Population-level organization of single instance ARC-GEN difference vectors (794 tasks; 500 instances each; 397,000 total) and its dissociation from exact-output retrieval. DINOv3, MAE, and random-projection pixel controls were evaluated at 224 and 480 px. (a) Cross tasks difference vector schematic. (b) Leave-one-out nearest-centroid accuracy across model layers. Markers denote layers selected independently by retrieval on the original ARC evaluation pairs; pixel controls are layer-independent. (c) Per-task accuracy distributions at these layers; legends report means. (d) Per-task retrieval and classification performance correlation.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Per-subtask nearest-centroid accuracy on the Bongard benchmarks, at the settings selected for Figure 1 (d,e). (a,b) The four disjoint Bongard–LOGO test subsets at 224 and 480 pixels. (c,d) The Bongard–OpenWorld subgroups at 224 and 480 pixels. Bars compare DINOv3, a MAE encoder of matched size, and a random-projection pixel control; the legend in (a) applies to every panel. The encoder layer is the one that maximizes the dataset-level mean within each dataset and resolution and is then held fixed across all subtasks, so no subtask is selected on its own; the layer choice is a descriptive maximum on the evaluation data and the intervals shown do not include selection uncertainty. Error bars are the SEM across problems for the encoders and, for pixels, a conservative upper bound on the SEM of the mean over ten projection seeds. The dashed line marks chance. No published reference line is drawn, because published values are reported per subset rather than for these groupings. The OpenWorld subgroups form two overlapping partitions of the same 200 problems – concept length (short/long) and commonsense requirement – and must not be averaged together. The count beneath each group is its number of problems; every problem contributes two held-out decisions, one per class.
Figure S2: Nearest-centroid accuracy across all 24 transformer blocks, for DINOv3 (left column) and a MAE encoder of matched size (right column). (a–d) Bongard–LOGO at 224 and 480 pixels, with one curve per disjoint test subset and a black curve for their equal-weight mean. (e–h) Bongard–OpenWorld at 224 and 480 pixels, with one curve per subgroup and a black curve for all 200 problems. The OpenWorld subgroups form two overlapping partitions of the same problems – concept length and commonsense requirement – so their curves are not independent strata. The legend in (a) applies to every LOGO panel and the one in (e) to every OpenWorld panel. Shading denotes the SEM across problems. The gray line marks the best published average (LOGO , the equal-weight mean of the per-subset maxima in ( Nie et al., 2020 ) ; OpenWorld ( Wu et al., 2024 ) ) and the dashed line marks chance. All panels share the same axes. The bars in Figure 1 (d,e) report the maximum of each black curve; because that layer is chosen on the same evaluation data, the peaks shown here are descriptive maxima rather than independently selected estimates.
Figure S3: Intuition–test-vector angles across all 24 transformer blocks on ARC-1. Green curves show matched tasks, red curves cross-task controls, and orange curves cell-shuffled answers. Solid and dashed curves denote the evaluation and training sets, respectively; shading shows the SEM across tasks and the dotted line marks . Both sets contain tasks. Cross-task and shuffled-output controls average ten randomizations. Figure 2 (c) reports the corresponding values at the globally selected layer 19.
Figure S4: Intuition–test-vector angles across all 24 transformer blocks on ARC-2, using the same conventions as Figure S3 . The training set contains tasks and the evaluation set , producing wider evaluation uncertainty. Figure 2 (d) reports the corresponding values at the globally selected layer 19.
Figure S5: Intuition–test-vector angles in random-projection pixel space for ARC-1 and ARC-2 at 224 and 480 px. Pixel representations have no layer axis and are therefore shown categorically. Green bars show matched tasks, red bars cross-task controls, and orange bars cell-shuffled answers. Filled bars denote evaluation tasks and open hatched bars training tasks. Bars average ten projection seeds; error bars combine the mean within-seed task SEM with uncertainty across projection seeds. Results at 224 and 480 px differ by at most .
Figure S6: Output retrieval on the ARC training sets, matching Figure 3 (c,d) panel for panel. Top-1 accuracy across all 24 transformer blocks, comparing DINOv3 and a MAE encoder of matched size at both rendering resolutions, with the layer-independent random-projection pixel controls drawn as flat lines. Solid curves add the intuition vector to the held-out query and dotted curves rank against the unperturbed query. The training sets are larger than the evaluation set, at 400 against 400 tasks for ARC-1 and 1,000 against 120 for ARC-2, so these curves are the smoother estimate of the same effect; neither set was used to select a layer, an encoder or a resolution.
Figure S7: Additional ARC-GEN retrieval and classification diagnostics. DINOv3, MAE, and random-projection pixel controls are compared at 224 and 480 px. Retrieval uses 100 generated queries from each of 794 tasks and a 368,442-grid candidate pool; encoder layers were selected on original ARC queries and then held fixed. (a) Exact-output retrieval without the intuition vector (pale) and with it (solid) at each encoder’s selected layer, alongside the pixel controls. Whiskers are marginal task-bootstrap intervals. (b) Pairwise Spearman correlations between representations’ task-wise transported retrieval rates. (c) Standardized task properties for the two pixel-space extremes in Figure 5(d) : high classification with low retrieval ( and ; ) and the reverse ( ). Points and whiskers show group means and bootstrap intervals; asterisks denote Bonferroni-corrected two-sided Mann–Whitney .
Figure S8: Nearest-neighbor discrimination after intuition-vector translation on noisy ARC-1 and ARC-2 for DINOv3 (480 px) and 10 random projections of pixel-space. (a,b) Top-1 retrieval of the true answer from a 100-grid pool containing the answer and corrupted variants, shown as a function of the fraction of corrupted cells. Colors identify sampled model layers; error bars are binomial SEM estimates. For pixelsbars are mean binomial SEM combined with the SEM across seeds.
Figure S9: Primitive transformations in DINOv3-Large at 480 px compose locally but depend on their source configuration. Layer numbers are zero-indexed. (a,d) Synthetic single-cell example designs without and with shared context. (b) Layer-17 geometry of four unit translations and eight recolorings from blue. (c) Layer-wise prediction of individual joint transformations from family-average translation and recoloring vectors, with shuffled controls. (e) Similarity between contextual primitives and their empty-grid counterparts by transformation family and color context. (f) Joint contextual transformations predicted using primitives measured in the same context, on an empty grid, or with an incorrect contextual recoloring. Blue and orange denote cosine similarity and relative residual, respectively; bands and error bars denote 95% confidence intervals.
Figure S10: Cell-addition vectors in DINOv3-Large at 480 px are context-dependent and directionally aligned, but not magnitude-additive. Layer numbers are zero-indexed. (a) The same cell is added to an empty or occupied grid. (b) Similarity between isolated and contextual addition vectors as occupancy increases, separated by color context. (c) Multiple additions measured separately and jointly from a shared source context. (d) The joint addition is compared with sums of context-conditioned vectors, isolated additions, and a wrong-location control. Blue and orange denote cosine similarity and relative residual, respectively; bands and error bars denote 95% confidence intervals.
Figure S11: Additional analyses of context-dependent single-cell geometry in DINOv3-Large at 480 px. Position decoding uses four-fold grouped cross-validation over independently sampled spatial layouts, with all color variants of a layout confined to the same fold. Layerwise analyses use a balanced subset containing every unit direction in each of four placement sets. The transfer analyses use five context cells; the multi-addition analysis uses five additions from a one-cell base. Bands and error bars denote 95% spatial-bootstrap confidence intervals. Layer numbers are zero-indexed.
More in this topic
Latent Visual Reasoning
Latent visual reasoning studies how multimodal models use compact internal visual states to solve perception, spatial, and temporal tasks.
See all 88 papers