Low-Rank Representation Learning

Latest papers

13 papers2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Representations Have Trajectories: Characterizing Temporal Representation Dynamics in Neural Networks
Oct 17, 2026Shivaprasad B. Gowda, Sparsh MittalLow-Rank Representation Learning Neural Representation Geometry
Indian Institute of Information Technology, Nagpur · Indian Institute of Technology, Roorkee
Compactness and Consistency: A Conjoint Framework for Deep Graph Clustering
Oct 8, 2026Wei Ju, Siyu Yi, Kangjie Zheng +6Low-Rank Representation Learning Graph Clustering
College of Computer Science, Sichuan University · College of Mathematics, Sichuan University · Wellcome Sanger Institute +4
Fast Implicit Neural Light Field Representation via Geometric Decomposition and Multi-Resolution Low-Rank Features
Aug 14, 2026Yao Guo, Ligen Shi, Shuchen Sun +2Low-Rank Representation Learning Neural Rendering
Institute of Computational Imaging, Beijing Information Science and Technology University, Beijing 102206, China · College of Computer Science (College of Software), Inner Mongolia University, Hohhot 010021, China
Breaking the Periodicity Assumption: Robust Tensorial Multi-View Clustering via Graph-Spectral Low-Rank Learning
Jul 28, 2026Jintian Ji, Xingsu Li, Songhe FengRepresentation Learning Multi-View Learning
Computer Science and Engineering The Hong Kong University of Science and Technology Hong Kong, China · School of Computer and Information Technology Beijing Jiaotong University, Beijing, China
LCPNet: Latent Consistent Proximal Unfolding Network for Infrared Small Target Detection
Jul 6, 2026Tianfang Zhang, Lei Li, Chang Liu +3Remote Sensing Small Object Detection Proximal Optimization
Department of Automation, Tsinghua University, Beijing, 100190, China. · School of Artificial Intelligence, Beijing Institute of Technology, Beijing, 100081, China. · School of Information and Communication Engineering and the Laboratory of Imaging Detection and Intelligent Perception, University of Electronic Science and Technology of China, Chengdu, 610054, China.
Identifying Latent Concepts and Structures for Generalized Category Discovery
Jul 1, 2026Boyang Dai, Chaoqi Chen, Yizhou YuRepresentation Learning Visual Representation Learning
1The University of Hong Kong · 2Shenzhen University
DLR: Zero-Inference-Cost Latent Residuals for Low-Rank Pre-Training
Jun 27, 2026Dong Wang, Wenwu Tang, Yun Cheng +1Language Model Pretraining LLM Compression
Graz University of Technology, Austria · Swiss Data Science Center, Switzerland
Decompose Sparsely Where You Should, Absorb Densely Where You Should No
Jun 12, 2026Ruixuan Deng, Zehao Jin, Zekun Wang +1Transformer Interpretability Representation Learning
School of Psychology Georgia Institute of Technology · College of Computing Georgia Institute of Technology · School of Interactive Computing Georgia Institute of Technology
Riemannian Optimization for Hadamard Products of Low-Rank Matrices
May 31, 2026Pratik Jawanpuria, Ankish Chandresh, Bamdev MishraRiemannian Optimization Low-Rank Approximation
Centre for Machine Intelligence and Data Science, Indian Institute of Technology Bombay, India · Microsoft India
Spatial Adapter: Structured Spatial Decomposition and Closed-Form Covariance for Frozen Predictors
May 12, 2026Wen-Ting Wang, Wei-Ying Wu, Hao-Yun Huang +1Representation Learning Residual Learning
Institute of Statistics, National Chung Hsing University · Department of Applied Mathematics, National Dong Hwa University · Department of Applied Mathematics National Dong Hwa University
MatryoshkaLoRA: Learning Accurate Hierarchical Low-Rank Representations for LLM Fine-Tuning
May 8, 2026Ionut-Vlad Modoranu, Mher Safaryan, Dan AlistarhAdapter Tuning LLM Fine-Tuning
Institute of Science and Technology Austria (ISTA). · Lancaster University, UK · ISTA
Can We Really Learn One Representation to Optimize All Rewards?
Feb 11, 2026Chongyi Zheng, Royina Karegoudra Jayanth, Benjamin EysenbachZero-Shot RL Representation Learning
Princeton University
Transfer Denoising of Low-Rank Plus Sparse Matrices under Ambient and Rank Growth
Jan 29, 2026Elynn Chen, Jinhang Chai, Xuyuan Liu +1Transfer Learning Low-Rank Representation Learning
Department of Technology, Operations, & Statistics, New York University · Department of Computer Science, Dartmouth College

Oct 17, 2026J/K move · Enter open · S save

Representations Have Trajectories: Characterizing Temporal Representation Dynamics in Neural Networks

Shivaprasad B. Gowda, Sparsh Mittal

Indian Institute of Information Technology, Nagpur · Indian Institute of Technology, Roorkee

Low-Rank Representation Learning Neural Representation Geometry Neural Network Training Dynamics Representation Learning Neural Network Optimization Representation Collapse

Open paper AskPDF arXiv

Recent work has shown that deep neural networks exhibit an implicit low-rank simplicity bias. In this paper, we investigate the questions that naturally arise from this observation. What are the temporal consequences of low-rank simplicity bias during optimization? How does representation geometry evolve throughout optimization rather than only at convergence? Do different neural architectures exhibit distinct geometric optimization behaviors? To answer these questions, we conduct a large-scale empirical study tracking representational geometry throughout training. We find that neural networks follow three recurring geometric trajectories: persistent high-rank, collapse-and-recovery, and persistent low-rank. We further identify Destructive Representation Collapse (DRC) as a pathological regime in which representations collapse to rank-one and fail to recover, leading to a sustained loss of representational capacity. While collapse-and-recovery is consistently associated with successful optimization in controlled settings, experiments on real-world models show that the relationship between geometric trajectories and optimization outcomes is configuration dependent. Overall, our findings suggest that representation geometry evolves dynamically during training and provide a new perspective on how deep neural networks learn.