Hierarchical Representation Learning

Follow this topic Ask about this topic

Momentum

18 papers in the last four weeks, up 200% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 155

All topics

Uncovering the Latent Potential of Deep Intermediate Representations
May 21, 2026Arnesh Batra, Arush Gumber, Aniket Khandelwal +2Neural Representation Geometry Hierarchical Representation Learning
SBILab, Indraprastha Institute of Information Technology Delhi, Delhi, India.
Efficient coding along the visual hierarchy
May 18, 2026Ananya Passi, Brian S. Robinson, Michael F. BonnerUnsupervised Learning Neural Processes
Department of Cognitive Science, Johns Hopkins University, Baltimore, MD, USA · Applied Physics Laboratory, Johns Hopkins University, Baltimore, MD, USA
Self-supervised local learning rules learn the hidden hierarchical structure of high-dimensional data
May 18, 2026Ariane Delrocq, Wu S. Zihan, Guillaume Bellec +1Self-Supervised Learning Hierarchical Representation Learning
School of Life Science and School of Computer and Communications Sciences, EPFL, Lausanne, Switzerland · Machine Learning Research Unit, TU Wien, Vienna, Austria
Graph Hierarchical Recurrence for Long-Range Generalization
May 18, 2026Stefano Carotti, Marco Pacini, Alessio Gravina +3Graph Neural Networks Graph Representation Learning
Department of Computer Science, University of Trento, Italy. · Fondazione Bruno Kessler, Italy. · Department of Computer Science, University of Pisa, Italy.
HyperPersona: A Multi-Level Hypergraph Framework for Text-Based Automatic Personality Prediction
May 17, 2026Sina Heydari, Majid RamezaniPersonality Modeling in Language Models Hierarchical Representation Learning
Department of Computer Science and Information Technology Institude for Advanced Studies in Basic Sciences (IASBS) Zanjan, Iran, 45137-66731
Neuroscience-inspired Staged Representation Learning with Disentangled Coarse- and Fine-Grained Semantics for EEG Visual Decoding
May 16, 2026Xiang Gao, Hui Tian, Yanming Zhu +2Disentangled Representation Learning Cross-Modal Representation Learning
School of Information and Communication Technology, Griffith University, Parklands Drive, Southport, 4222, QLD, Australia
Navigating the Emotion Tree: Hierarchical Hyperbolic RAG for Multimodal Emotion Recognition
May 16, 2026Zeheng Wang, Bo Zhao, Yijie Zhu +6Multimodal RAG Hierarchical Classification
Great Bay University Guangdong, Dongguan, China · Tencent Youtu Lab Shanghai, China · CUHK, shenzhen Guangdong, Shenzhen, China
Multi-level Self-supervised Pretraining on Compositional Hierarchical Graph for Molecular Property Prediction
May 15, 2026Xiayu Liu, Zhengyi Lu, Hou-biao LiSelf-Supervised Pre-Training Hierarchical Representation Learning
School of Mathematical Sciences, University of Electronic Science and Technology of China, No.2006, Xiyuan Avenue, West Hi-tech Zone,2026 611731, Chengdu, China · Department of Computer Science and Engineering, Oakland University, 201 Meadow Brook RD, Rochester Hills, 48309, MI, USA
Structure Abstraction and Generalization in a Hippocampal-Entorhinal Inspired World Model
May 15, 2026Tianqiu Zhang, Muyang Lyu, Xiao Liu +1Computational Cognitive Modeling World Model Learning
Peking-Tsinghua Center for Life Sciences, Academy for Advanced Interdisciplinary Studies, IDG/McGovern Institute for Brain Research, Center of Quantitative Biology, School of Psychological and Cognitive Sciences, Key Laboratory of Machine Perception (Ministry of Education), Peking University. · HHMI Janelia.
Compositional Sparsity as an Inductive Bias for Neural Architecture Design
May 14, 2026Hongyu Lin, Antonio Briola, Yuanrong Wang +1Hierarchical Representation Learning Structured Sparsity
Department of Computer Science, University College London, London, United Kingdom
Scaling Laws from Sequential Feature Recovery: A Solvable Hierarchical Model
May 14, 2026Arie Wortsman-Zurich, Hugo Tabanelli, Yatin Dandi +2Hierarchical Representation Learning Scaling Laws
Département d’Informatique, Ecole Normale Supérieure, PSL & CNRS · Information Learning and Physics Laboratory, École Polytechnique Fédérale de Lausanne (EPFL) · Statistical Physics of Computation Laboratory, École Polytechnique Fédérale de Lausanne (EPFL)
MoRe: Modular Representations for Principled Continual Representation Learning on Sequential Data
May 14, 2026Jiaqi Sun, Boyang Sun, Rasmy M. H. +2Representation Learning Hierarchical Representation Learning
1Carnegie Mellon University · 2Mohamed bin Zayed University of Artificial Intelligence
Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning
May 13, 2026Yatin Dandi, Matteo Vilucchio, Luca Arnaboldi +2Hierarchical Representation Learning Neural Network Training Dynamics
Information Learning and Physics Laboratory, École Polytechnique Fédérale de Lausanne (EPFL) · Statistical Physics of Computation Laboratory, École Polytechnique Fédérale de Lausanne (EPFL)
From Generalist to Specialist Representation
May 12, 2026Yujia Zheng, Fan Feng, Yuke Li +3Unsupervised Learning Disentangled Representation Learning
1CMU · 2UIUC · 3UCSD +3
H2G: Hierarchy-Aware Hyperbolic Grouping for 3D Scenes
May 12, 2026ByungHa Ko, Youngmin Lee, Dong Hwan KimClustering Hierarchical Representation Learning
Department of Computer Science and Engineering, Korea University · Intelligence and Interaction Research Center, Korea Institute of Science and Technology
Beyond Point-wise Neural Collapse: A Topology-Aware Hierarchical Classifier for Class-Incremental Learning
May 12, 2026Huiyu Yi, Zhiming Xu, Dunwei Tu +3Class-Incremental Learning Prototype-Based Classification
National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China · School of Computer Science, Nanjing University, Nanjing, China
M3Net: A Macro-to-Meso-to-Micro Clinical-inspired Hierarchical 3D Network for Pulmonary Nodule Classification
May 12, 2026Jinyue Li, Yuzhou Yu, Jingjing Yang +73D Medical Imaging Hierarchical Representation Learning
Hefei Cancer Hospital of CAS, Institute of Health and Medical Technology, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei, 230031, PR China · University of Science and Technology of China, Hefei, 230026, PR China · Graduate School, Bengbu Medical College, Bengbu, 233030, PR China +4
Online Continual Learning with Dynamic Label Hierarchies
May 12, 2026Xinrui Wang, Shao-Yuan Li, Bartłomiej Twardowski +2Class-Incremental Learning Hierarchical Classification
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics · MIIT Key Laboratory of Pattern Analysis and Machine Intelligence · Computer Vision Center, Spain +4
HH-SAE: Discovering and Steering Hierarchical Knowledge of Complex Manifolds
May 11, 2026Honghan Wu, Tianyan Wang, Jiacong Mi +2Hierarchical Representation Learning Sparse Autoencoders
University of Glasgow, Glasgow, UK · University of International Relations, Beijing, China · University College London, London, UK
Sharp feature-learning transitions and Bayes-optimal neural scaling laws in extensive-width networks
May 11, 2026Minh-Toan Nguyen, Jean BarbierNeural Network Generalization Hierarchical Representation Learning
ICTP Trieste, Italy
WavesFM: Hierarchical Representation Learning for Longitudinal Wearable Sensor Waveforms
May 9, 2026Peng Cao, Zhijian Yang, Tennison Liu +17Self-Supervised Time Series Representation Learning Hierarchical Representation Learning
Google Research · University of Washington · MIT
Objective-Specific Privileged Bases via Full-Prefix Matryoshka Learning
May 9, 2026Arghamitra Talukder, Philippe Chlenski, Itsik Pe'erRepresentation Learning Representation Identifiability
Computer Science, Columbia University
Semantic Alignment in Hyperbolic Space for Open-Vocabulary Semantic Segmentation
May 9, 2026Hoang M. Truong, Hai Nguyen-Truong, Dang HuynhVLM Adaptation Hierarchical Representation Learning
Fulbright University Vietnam
Hierarchical Multi-Scale Graph Neural Networks: Scalable Heterophilous Learning with Oversmoothing and Oversquashing Mitigation
May 8, 2026Md Sazzad Hossen, Avimanyu SahooGraph Neural Networks Over-Squashing
University of Alabama in Huntsville, Huntsville, AL, USA.
Tree SAE: Learning Hierarchical Feature Structures in Sparse Autoencoders
May 8, 2026Tue M. Cao, Hoang X. Nhat, Raed Alharbi +2Hierarchical Representation Learning Sparse Autoencoders
Hanoi University of Science and Technology, Hanoi, Vietnam · University of Florida, Florida, USA · Computer Science Department, Saudi Electronic University
MatryoshkaLoRA: Learning Accurate Hierarchical Low-Rank Representations for LLM Fine-Tuning
May 8, 2026Ionut-Vlad Modoranu, Mher Safaryan, Dan AlistarhLLM Fine-Tuning Hierarchical Representation Learning
Institute of Science and Technology Austria (ISTA). · Lancaster University, UK · ISTA
PPI-Net connects molecular protein interactions to functional processes in disease
May 8, 2026Kyle Higgins, Guadalupe Gonzalez, Dennis Veselkov +2Cancer Genomics Graph Neural Networks
Division of Cancer, Department of Surgery and Cancer, Faculty of Medicine, Imperial College London, London, UK. · Prescient Design, Genentech, South San Francisco, CA, USA. · Department of Computing, Imperial College London, U.K.
Classification Fields: Arbitrarily Fine Recursive Hierarchical Clustering From Few Examples
May 8, 2026Yicen Li, Ruiyang Hong, Anastasis Kratsios +2Hierarchical Classification Clustering
Department of Mathematics and Statistics, McMaster University, Canada · Vector Institute, Canada · University of Cambridge, United Kingdom
Hyperbolic Concept Bottleneck Models
May 7, 2026Daniel Uyterlinde, Swasti Shreya Mishra, Pascal MettesConcept Bottleneck Models Hierarchical Representation Learning
Informatics Institute, University of Amsterdam, The Netherlands
Graph Neural Network based Hierarchy-Aware Embeddings of Knowledge Graphs: Applications to Yeast Phenotype Prediction
May 5, 2026Filip Kronström, Alexander H. Gower, Daniel Brunnsåker +2Graph Neural Networks Representation Learning
Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, Sweden · Department of Life Sciences, Chalmers University of Technology, Sweden · Department of Industrial Biotechnology, KTH Royal Institute of Technology, Sweden +1

May 21, 2026, cs.LGJ/K move · Enter open · S save

Uncovering the Latent Potential of Deep Intermediate Representations

Arnesh Batra, Arush Gumber, Aniket Khandelwal, Jashn Khemani+1

SBILab, Indraprastha Institute of Information Technology Delhi, Delhi, India.

Neural Representation Geometry Hierarchical Representation Learning Geometric Representation Learning Transfer Learning

Open paper AskPDF arXiv

Foundational Models pretrained on huge amount of data learn representations that evolve across depth, forming a hierarchy of embeddings with distinct semantic content and geometric structure. Contrary to the widespread practice of using only the final layer or shallow mixtures, we show that task-relevant information is distributed non-monotonically across layers and cannot be recovered by naïve aggregation. Through a geometric and empirical study across multiple modalities, we show that effective transfer depends on identifying which layers encode task-discriminative structure and how their embeddings are geometrically organized. We introduce Layer-wise Optimal Embedding Selection (LOES), a constructive spectral method that identifies task-discriminative subspaces by minimizing residual error under orthogonality and isotropy constraints. To align fine-tuning with this selection principle, we further propose Geometric Regularization Loss (GeoReg), which enforces a simplicial structure on class manifolds and stabilizes representation geometry during fine-tuning. Across a wide range of architectures, depths, modalities, and data regimes, LOES consistently outperforms standard baselines, with gains that grow as model depth increases. Beyond accuracy, our method reveals how semantic factors are distributed across layers, thereby enabling cross-lingual and cross-modal interpretability analyses. Together, our results provide strong evidence that layerwise embedding geometry is not incidental but central to how deep models represent and transfer knowledge.

Momentum

Related topics

Latest papers 155