Neural Representation Geometry

Follow this topic Ask about this topic

Momentum

18 papers in the last four weeks, up 125% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 182

All topics

Calibrating WEAT Against Anisotropy: ZCA Whitening as a Geometric Pre-Processing Step for Embedding Association Tests
Aug 7, 2026Seitaro Ono, Senna Ross, Jun SaikiNeural Representation Geometry Text Embedding Evaluation
Graduate School of Human and Environmental Studies, Kyoto University, Kyoto, Japan · Brock University, St. Catharines, Ontario, Canada
On the Geometry of Music Bandwidth Extension in Latent Spaces of Audio Codecs
Aug 4, 2026Hendrik Vincent Koops, Hao Hao Tan, Elio QuintonNeural Audio Codecs Neural Representation Geometry
Music & Audio Machine Learning Lab · Universal Music Group, London, U.K.
Transformer Geometry Observatory TGO-III: Semantic Geometry Observatory
Aug 3, 2026Kaustubh Kapil, Kishor P. UplaTransformer Interpretability Neural Representation Geometry
Department of Electronics Engineering Sardar Vallabhai National Institute of Technology (SVNIT), Surat, India
Do Music Foundation Models Embed Pitch in Helical Structure?
Jul 31, 2026Hayato Yagi, Shinnosuke Takamichi, Rin Sato +2Audio Representation Learning Neural Representation Geometry
Keio University, Japan · The University of Tokyo, Japan · Waseda University, Japan
More Data, Worse Decisions? Preference Reversals in Neural Networks under Gram Incompatibility
Jul 28, 2026Yanli Yan, Yuanzheng Li, Yong Zhao +2Neural Network Generalization Neural Representation Geometry
Huazhong University of Science and Technology Wuhan, China
Neural Representation of Minimal Surfaces
Jul 26, 2026Jiayin Sun, Albert ChernNeural Representation Geometry
University of Utah, USA · University of California San Diego, USA
Functional Equivalence and Geometric Diversity in Neural Network Approximations: An Empirical Characterization
Jul 21, 2026Anuragine S A, Prem JagadeesanMultilayer Perceptrons Neural Representation Geometry
Amrita School of Artificial Intelligence Amrita Vishwa Vidyapeetham, Coimbatore, India
Topological Signatures of Context-Level Reliability in TabPFN
Jul 20, 2026James Hu, Mahdi GhelichiTabular Foundation Models Neural Representation Geometry
Model Development Innovation, Risk Management TD Bank, Toronto, Canada
Quantifying Training Membership Information in the Hyperspherical Embedding Geometry of Face Recognition Models
Jul 16, 2026Ünsal Öztürk, Sébastien MarcelNeural Representation Geometry Membership Inference Attacks
Idiap Research Institute, Martigny, Switzerland · Universit´e de Lausanne (UNIL), Lausanne, Switzerland
From Geometric Recovery to Causal Validation: A Reproducible Audit of Sparse Autoencoder Features, from Superposition Geometry to Causal Inertness
Jul 13, 2026Mohamed Abdessalem BalFeature Superposition Neural Representation Geometry
Independent Researcher, Algiers, Algeria
How to Tame Grokking: Representation Geometry as a Control Signal
Jul 13, 2026Maksim A KazanskiiNeural Network Generalization Spectral Regularization
Independent Researcher
Beyond Coordinate Gauge: An Audited Protocol for Detecting Donor-Specific Functional Fingerprints after Neural Collapse
Jul 12, 2026Truong Xuan Khanh, Phan Thanh DucNeural Representation Geometry Neural Collapse
H&K Research Studio, Clevix LLC, Hanoi, Vietnam · Banking Academy of Vietnam, Hanoi, Vietnam
Contravariance Theory: Strong Alignment for Minimal Solutions to Hard Tasks
Jul 9, 2026Dan Yamins, Aran NayebiNeural Representation Geometry
Departments of Computer Science and Psychology, and Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA 94305 USA · Machine Learning Department, Neuroscience Institute, and Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA
Dynamic neural manifolds for flexible closed-loop control on neuromorphic hardware
Jul 8, 2026Oskar von Seeler, Christian Tetzlaff, Andrew LehrRobotic Control Neural Representation Geometry
Group of Computational Synaptic Physiology, Department of Neuro- and Sensory Physiology, University Medical Center Göttingen, Germany · Campus Institute Data Science, University of Göttingen, Germany · Circulant Labs, Bensheim, Germany
Language Models Represent and Transform Concepts with Shared Geometry
Jul 5, 2026Zhimin Hu, Lanhao Niu, Sashank VarmaNeural Representation Geometry Representation Geometry in Language Models
1Georgia Institute of Technology · University of Edinburgh
Transition Information Density: Morphological Trajectories, Synesthetic Perception, and Structured Interpolation in Neural Training (or: The Synesthetic AI)
Jul 3, 2026Sam MaoNeural Representation Geometry Trajectory Representation Learning
New York University | Interactive Media Arts
Observable- and Positional-Encoding-Dependent Symmetry Readout from Neural Network Weights
Jul 3, 2026Naoya Chiba, Satoshi Sugiyama, Yuki UranishiNeural Representation Geometry Implicit Neural Representations
D3 Center, The Unversity of Osaka
Transformer Geometry Observatory TGO-II: Representational Similarity Observatory
Jul 2, 2026Kaustubh Kapil, Kishor P. UplaIntrinsic Dimensionality Representational Similarity Analysis
Department of Electronics Engineering Sardar Vallabhai National Institute of Technology (SVNIT), Surat, India
Understanding Geometric Representations in Self-Supervised Vision Transformers via Subspace Intervention
Jul 2, 2026Weichen Zhou, Yawen Zou, Chunzhi Gu +3Neural Representation Geometry Linear Probing
University of Toyama, Toyama, Japan · University of Fukui, Fukui, Japan · Chukyo University, Aichi, Japan +1
Finite-Lag Operator Geometry of Recurrent Representations
Jul 2, 2026Kanishka ReddyNeural Representation Geometry Recurrent Neural Networks
Department of Applied Mathematics University of Washington Seattle, WA 98105
Radial Suppression Accelerates Algorithmic Generalization: A Geometric Analysis of Delayed Generalization
Jun 30, 2026Srijan Tiwari, Aditya Chauhan, Manjot SinghNeural Network Generalization Neural Representation Geometry
Indian Institute of Technology Roorkee
Resolving superposition in AI for interpretability and cross-modal alignment in patient-neuronal images
Jun 30, 2026Jisung Park, Seohyeon Kang, Daeun Yoo +8Feature Superposition Neural Representation Geometry
1KAIST · 2Konyang University · 4UCL Queen Square Institute of Neurology & The Francis Crick Institute +1
Learning as Observable Matrix Dynamics: Diffusive Relaxations versus Phase Transitions
Jun 29, 2026Igor HalperinNeural Representation Geometry Neural Network Training Dynamics
All calculations, numerical analysis, and manuscript preparation were performed by Claude Code with Opus 4.7 working as an AI assistant under author's supervision.
Geometric Reliability of Neural Population Codes: Sampling Calibration and Within-Session Nonstationarity
Jun 28, 2026Prashant C. RajuRepresentational Similarity Analysis Neural Representation Geometry
Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings
Jun 28, 2026V. S. Raghu ParupudiSemantic Textual Similarity Neural Representation Geometry
University of California, San Diego
Geometric Measurements of the Axiom of Choice in Neural Proof Embeddings
Jun 26, 2026Rodrigo Mendoza-SmithNeural Representation Geometry Automated Theorem Proving
Independent Researcher.
CANNs: A Toolkit for Research on Continuous Attractor Neural Networks
Jun 26, 2026Sichao He, Aiersi Tuerhong, Shangjun She +4Neural Representation Geometry Recurrent Neural Networks
School of Psychological and Cognitive Sciences, Peking–Tsinghua Center for Life Sciences, PKU-IDG/McGovern Institute for Brain Research, Center of Quantitative Biology, Peking University, Beijing, China · College of Mathematics and Statistics, Chongqing University, Chongqing, China
SubdivAR: Autoregressive Next-Scale Prediction for Neural Mesh Subdivision
Jun 25, 2026Huipeng Guo, Zikai Song, Hang Long +73D Mesh Generation Neural Representation Geometry
Huazhong University of Science and Technology
Exploring the Intrinsic Geometry of Diffusion Models with Constrained Inverse Kinematics
Jun 24, 2026Miguel Angel Rogel Garcia, Phone Thiha Kyaw, Jonathan KellyNeural Representation Geometry Conditional Diffusion Models
Institute for Aerospace Studies, University of Toronto, Toronto, Canada
Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds
Jun 23, 2026Thomas Fel, Matthew Kowal, Mozes Jacobs +22Neural Representation Geometry Activation Sparsity
•Goodfire · aHarvard University · bStanford University +1

Aug 7, 2026, cs.CLJ/K move · Enter open · S save

Calibrating WEAT Against Anisotropy: ZCA Whitening as a Geometric Pre-Processing Step for Embedding Association Tests

Seitaro Ono, Senna Ross, Jun Saiki

Graduate School of Human and Environmental Studies, Kyoto University, Kyoto, Japan · Brock University, St. Catharines, Ontario, Canada

Neural Representation Geometry Text Embedding Evaluation Language Model Bias Evaluation

Open paper AskPDF arXiv

We propose Zero-phase Component Analysis (ZCA) whitening as a geometric pre-processing step for the Word Embedding Association Test (WEAT). WEAT is a bias measurement method widely used in both computational social science and AI fairness research. It relies on cosine similarity as a measure of semantic association, which assumes that the embedding space is approximately isotropic. However, prior work has reported that many widely used language models do not satisfy this assumption, raising concerns about the reliability of bias measurements. ZCA whitening transforms the covariance of the embedding space into the identity matrix while minimizing perturbation to the original vectors. This transformation restores the isotropy condition on which WEAT relies. We evaluate our approach on ten standard WEAT test suites and seven models spanning three architectural families, yielding 70 model-task combinations. The results show that ZCA whitening substantially reduces the anisotropy of the embedding spaces across all models. Particularly for highly anisotropic models, we further observe improvements on standard semantic similarity benchmarks, indicating that the calibrated space better captures semantic associations. After calibration, over 30% of WEAT results change significance status, and effect sizes shift in both directions depending on bias category. These shifts suggest that uncalibrated measurements may both overestimate and underestimate the associations encoded in the embedding space. These findings indicate that previously reported bias measurements in anisotropic embedding spaces should be interpreted with caution and may benefit from re-evaluation with calibrated methods. Our approach contributes to restoring the measurement foundation of WEAT across both computational social science and AI fairness research.

Momentum

Related topics

Latest papers 182