Weight-Space Analysis

Ask about this topic

Overview

Weight-space analysis treats neural network parameters—not just their predictions—as objects whose geometry can reveal how models learn, differ, and store capabilities. Researchers study symmetries, spectra, and low-rank subspaces; compare checkpoint deltas and task vectors for merging or provenance; and train symmetry-aware encoders to retrieve or transform weights across architectures. These methods are tested on model zoos, language-model checkpoints and adapters, vision models, and implicit neural representations, for tasks such as model similarity, task composition, data inference, and compression. A central challenge is that equivalent functions can have very different parameters, while geometric similarity or a successful merge may depend on architecture, prompt, or training setup and need not provide a unique mechanistic explanation.

Latest papers

CardsList

41 papers3 papers in the last four weeks, down 25% on the four weeks before. 0.0% of all new papers.

  1. When Are Two Networks the Same? Tensor Similarity for Mechanistic Interpretability

    May 14, 2026ML Nissen Gonzalez, Melwina Albuquerque, Laurence Wroe +3Mechanistic InterpretabilityWeight-Space Analysis

    MARS & University Heidelberg · MARS · MARS & Stanford University +2

  2. Is Class Signal Clustered or Routed in Task-Induced Implicit Neural Representation Weight Spaces?

    May 8, 2026Xinyi Guo, Mingyi He, Haobin Ding +7Neural Representation GeometryImplicit Neural Representations

    South China Normal University · Beijing University of Chemical Technology · Tsinghua University +1

  3. Model Merging: Foundations and Algorithms

    May 2, 2026Donato CrisostomiMulti-Task LearningTask Vector Merging

    SAPIENZA UNIVERSITY OF ROME · GLADIA research lab · Department of Computer Science +1

  4. The Role of Symmetry in Optimizing Overparameterized Networks

    Apr 28, 2026Kusha Sareen, Mohammad Pedramfar, Sékou-Oumar Kaba +2Neural Network OptimizationWeight-Space Analysis

    McGill University · Mila - Quebec Artificial Intelligence Institute

  5. Gradient-Direction Sensitivity Reveals Linear-Centroid Coupling Hidden by Optimizer Trajectories

    Apr 28, 2026Yongzhong XuWeight-Space AnalysisGrokking

  6. Understanding and Enforcing Weight Disentanglement in Task Arithmetic

    Apr 18, 2026Shangge Liu, Yuehan Yin, Lei Wang +5Task VectorsWeight-Space Analysis

    State Key Laboratory for Novel Software Technology, Nanjing University, China · University of Wollongong, Australia · Nanyang Technological University, Singapore

  7. FLARE: A Data-Efficient Surrogate for Predicting Displacement Fields in Directed Energy Deposition

    Apr 17, 2026Kittipong Thiamchaiboonthawee, Ghadi Nehme, Ram Mohan Telikicherla +5Neural Surrogate ModelingImplicit Neural Representations

    Massachusetts Institute of Technology, Cambridge, MA, USA · GE Vernova Advanced Research Center, Niskayuna, NY, USA

  8. Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights

    Mar 12, 2026Yulu Gan, Phillip IsolaLLM Post-TrainingWeight-Space Analysis

    MIT CSAIL

  9. A Survey of Weight Space Learning: Understanding, Representation, and Generation

    Mar 10, 2026Xiaolong Han, Zehong Wang, Bo Zhao +8Weight-Space LearningWeight-Space Analysis

    University of Surrey · University of Notre Dame · University of California San Diego +4

  10. A LoRA is Worth a Thousand Pictures

    Dec 16, 2024Chenxi Liu, Towaki Takikawa, Alec JacobsonWeight-Space AnalysisLow-Rank Adaptation

    University of Toronto · Outerport · Adobe Research

  11. The Geometry of the Set of Equivalent Linear Neural Networks

    Apr 23, 2024Jonathan Richard Shewchuk, Sagnik BhattacharyaDeep Linear NetworksWeight-Space Analysis

    Department of Electrical Engineering and Computer Sciences University of California at Berkeley Berkeley, California 94720