Representational Similarity Analysis

Also known as RSA

Representational Similarity Analysis studies whether different systems organize the same inputs similarly, often by comparing representational dissimilarity matrices across layers, models, or brain regions. Researchers use methods such as RSA, centered kernel alignment (CKA), SVCCA, Procrustes alignment, and newer local-neighborhood, topological, and optimal-transport measures, applying them to vision and language models, brain recordings, and cross-modal benchmarks. These comparisons help map model development and brain alignment, and can inform model selection, transfer, and network stitching. A central open issue is what similarity means: scores depend on the metric, coordinates, stimuli, and calibration, and similar geometry does not necessarily imply shared concepts, computations, or behavior.

Latest papers

41 papers7 papers in the last four weeks, up 133% on the four weeks before. 0.0% of all new papers.

Similarity of Neural Network Representations in Superposition
Mar 31, 2026Sunny Liu, Habon Issa, André Longon +4Feature Superposition Representational Similarity Analysis
Cold Spring Harbor Laboratory · UC San Diego · Anthropic +2
Geometry-aware similarity metrics for neural representations on Riemannian and statistical manifolds
Mar 30, 2026N Alex Cayco-Gajic, Arthur PellegrinoNeural Representation Geometry Representational Similarity Analysis
Gatsby Unit, University College London
Gauge Freedom and Metric Dependence in Neural Representation Spaces
Mar 6, 2026Jericho CainNeural Representation Geometry Representational Similarity Analysis
Physics Department, Portland Community College, Portland, OR, USA
Neural Functional Alignment Space: Brain-Referenced Representation of Artificial Neural Networks
Feb 28, 2026Ruiyu Yan, Hanqi Jiang, Yi Pan +4Neural Representation Geometry Cross-Modal Alignment
Tandon School of Engineering, New York University, Brooklyn, NY 11201, USA · School of Computing, University of Georgia, Athens, GA 30602, USA · Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA +1
Partial Soft-Matching Distance for Neural Representational Comparison with Partial Unit Correspondence
Feb 22, 2026Chaitanya Kapoor, Alex H. Williams, Meenakshi KhoslaRepresentational Similarity Analysis Neural Representation Geometry
Center for Neural Science, New York University, New York City, NY, 10003 · Center for Computational Neuroscience, Flatiron Institute, New York City, NY, 10003
Logit Distance Bounds Representational Similarity
Feb 17, 2026Beatrix M. G. Nielsen, Emanuele Marconato, Luigi Gresele +2Representational Similarity Analysis Representation Learning
IT University of Copenhagen, Denmark · University of Trento, Italy · University of Copenhagen, Denmark +2
Revisiting the Platonic Representation Hypothesis: An Aristotelian View
Feb 16, 2026Fabian Gröger, Shuo Wen, Maria BrbićPermutation Testing Representational Similarity Analysis
1EPFL · University of Basel · 3HSLU
Geometric Stability: The Missing Axis of Representations
Jan 14, 2026Prashant C. RajuRepresentational Similarity Analysis Neural Representation Geometry
From Topology to Retrieval: Decoding Embedding Spaces with Unified Signatures
Nov 27, 2025Florian Rottach, William Rudman, Bastian Rieck +2Representation Geometry in Language Models Text Embedding Evaluation
University of Tübingen · Boehringer Ingelheim GmbH, Biberach (Riss), Germany · The University of Texas at Austin +1
Manifold Approximation leads to Robust Kernel Alignment
Oct 27, 2025Mohammad Tariqul Islam, Du Liu, Deblina SarkarKernel Methods Representational Similarity Analysis
MIT
A quantitative analysis of semantic information in deep representations of text and images
May 21, 2025Santiago Acevedo, Andrea Mascaretti, Riccardo Rende +3Representational Similarity Analysis Cross-Modal Representation Learning
Scuola Internazionale Superiore di Studi Avanzati (SISSA) · Universitat Pompeu Fabra (UPF) · Catalan Institute of Research and Advanced Studies (ICREA) and Universitat Pompeu Fabra (UPF)

Mar 31, 2026J/K move · Enter open · S save

Similarity of Neural Network Representations in Superposition

Sunny Liu, Habon Issa, André Longon, Liv Gorton+3

Cold Spring Harbor Laboratory · UC San Diego · Anthropic+2

Feature Superposition Representational Similarity Analysis Neural Representation Geometry Compressed Sensing Neural Encoding

Open paper AskPDF arXiv

Comparing internal representations is a central goal in neuroscience and machine learning, but standard linear alignment metrics (Representational Similarity Analysis, Centered Kernel Alignment, and linear regression) are frequently applied to neural activity coordinates rather than on the underlying features. We show this matters when neural systems operate in superposition, encoding more features than they have neurons via linear compression. Closed-form derivations prove that these metrics depend on the Gram matrices of each system's projection, not on the latent features themselves: alignment thus combines what a system represents with how it is encoded. For those interested in what features two systems share, this is a problem: Two networks can have identical feature content yet appear more dissimilar than networks exhibiting partial feature overlap. This apparent misalignment need not reflect lost information as compressed sensing guarantees sparse features remain recoverable from the compressed activity. We confirm this by training supervised TopK sparse autoencoders that realize solvable compressed sensing by construction, finding alignment on recovered latents restored even when raw-activation alignment remains deflated. We extend the result to unsupervised SAEs trained without ground-truth latents, and to pretrained vision and language model SAEs, where SAE-latent alignment exceeds raw-activation alignment, consistent with superposition in real systems.