Vision Models Capture Complementary Representations in Children That Converge in Adults

Domenic Bersch; Timothy Schaumlöffel; Michela Proietti; Antonia Franaszek-Traczewska; Hannah Elisabeth Zwad; Siying Xie; Marlena Baldauf; Teresa Sylvester

cs.CVOct 12, 2026

Vision Models Capture Complementary Representations in Children That Converge in Adults

Authors: Domenic Bersch, Timothy Schaumlöffel, Michela Proietti, Antonia Franaszek-Traczewska, Hannah Elisabeth Zwad, Siying Xie, Marlena Baldauf, Teresa Sylvester,

Organizations: Department of Computer Science, Goethe University Frankfurt, Frankfurt am Main, Germany · The Hessian Center for Artificial Intelligence (hessian.AI), Darmstadt, Germany · Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany · Institute of Psychology, Developmental Cognitive Psychology, University of Regensburg, Regensburg, Germany ·

Abstract

How the relationship between image-text aligned and vision-only models changes across development remains unknown. We applied time-resolved representational similarity analysis to EEG data from 8-year-olds, 12-year-olds, and adults. CLIP showed significant model-brain alignment already in children, whereas vision-only models aligned more strongly later in development. Residual analyses revealed complementary structure, with CLIP retaining alignment beyond an architecture-matched vision-only model from age 8. Much of the model-brain alignment reflected semantic structure, with EEG representations of category and animacy strengthening toward adulthood, while alignment with low-level image properties remained weak across age. Together, these findings suggest that image-text aligned and vision-only models capture complementary aspects of developing visual representations and may provide useful constraints for AI systems that aim to learn and adapt over time.

Figures & tables

Figure 1: Model-brain alignment across development. (A) Time-resolved RSA between five vision models and EEG for 8-year-olds, 12-year-olds, and adults. Gray bands mark the 130-310 ms analysis window, shading indicates $\pm$ SEM, and bars mark significant timepoints from uncorrected stimulus-shuffle tests. (B) Residual alignment for CLIP and ViT-B/16 after removing shared model structure, evaluated at a layer selected independently of EEG from model-model RDM similarity. (C) Corresponding analysis for CLIP and SimCLR ViT-B/16 models trained on YFCC15M for 25 epochs. (D) Left, EEG alignment with category, animacy, luminance, and pixel-level hypothesis RDMs. Right, CLIP alignment after removing category and animacy structure, averaged across layers. Shading indicates SD across layers and dotted lines the reliability ceiling.

Vision Foundation Models

Vision foundation models are large pretrained visual systems adapted to recognition, segmentation, geometry, and video tasks.

See all 141 papers

Do Vision Model See Like the Brain? A Comparison Across EEG Encoding Model
Sep 22, 2026Shashank Baghel, Kshitij Dwivedi, Dinesh Singh +1Vision Transformer Visual Neuroscience
Centre for Human-Computer Interaction, Indian Institute of Technology, Mandi, Mandi, 175075, Himachal Pradesh, India · Freie Universität, Berlin, Germany · Goethe University, Frankfurt, Germany
LEVANTE-bench: Multi-Scale Comparison of VLMs to Children Using Cognitive Tasks (or, "Is Your VLM Smarter Than a 5th Grader?")
Jun 3, 2026Alvin Wei Ming Tan, David Cardinal, Tania Lorido-Botran +3VLM Evaluation AI Alignment
Stanford University
Brain-to-Image Retrieval and Reconstruction via Multimodal EEG Alignment
May 18, 2026Chi Kit Wong, Yan Liu, Haowen YanEEG-Based Visual Decoding Cross-Modal Alignment

Sep 22, 2026J/K move · Enter open · S save

Do Vision Model See Like the Brain? A Comparison Across EEG Encoding Model

Shashank Baghel, Kshitij Dwivedi, Dinesh Singh, Sanjeev Nara

Centre for Human-Computer Interaction, Indian Institute of Technology, Mandi, Mandi, 175075, Himachal Pradesh, India · Freie Universität, Berlin, Germany · Goethe University, Frankfurt, Germany

Vision Transformer Visual Neuroscience Convolutional Neural Networks Neural Encoding

Open paper AskPDF arXiv

Convolutional neural networks (CNNs) and vision transformers are both used to model the human visual system, but whether the two architectures diverge at a specific point in network depth is unclear. We compared six CNNs and two vision transformers by computing the Pearson correlation (r) between each model's predicted and measured EEG response at every layer or block, in ten participants viewing 200 natural images. For the transformer models, we also tested four token representations, from the classification (CLS) token alone to CLS combined with all patch tokens. CNNs showed strongest correspondence at the earliest layers, weakening at deeper layers, particularly later in the post-stimulus response. Transformers instead sustained strong correspondence at their deepest blocks, though not at their earliest ones. This advantage depended on token representation: pooled representations gave weaker peak correlations (r approx 0.48-0.51) than representations retaining all patch tokens (r=0.640 for CLIP-ViT-B/32, r=0.656 for DINOv2-ViT-B/14). Controlled comparisons showed architecture, not training objective, drove this effect: MoCo-v1 and ResNet-50 (matched architecture) performed nearly identically (r=0.673, 0.670), whereas CLIP-RN50 and CLIP-ViT-B/32 (matched objective) diverged until patch tokens were preserved. We propose that CNN training's classification bottleneck compresses brain-relevant information at depth, unlike transformers' self-attention and non-classification objectives. A spatial topography analysis showed a common occipital-dominant pattern across all models, indicating these differences reflect signal strength and persistence rather than distinct brain regions. Patch-preserving transformer representations sustain brain-predictive correspondence where CNNs collapse.

Abstract

Vision Foundation Models

Explore similar work

Do Vision Model See Like the Brain? A Comparison Across EEG Encoding Model

LEVANTE-bench: Multi-Scale Comparison of VLMs to Children Using Cognitive Tasks (or, "Is Your VLM Smarter Than a 5th Grader?")

Brain-to-Image Retrieval and Reconstruction via Multimodal EEG Alignment