Subspace

Follow this topic Ask about this topic

Momentum

38 papers in the last four weeks, up 111% on the four weeks before. 0.5% of all new papers.

Jul 6Week of Sep 21

Latest papers 279

All topics

Learning in Low-Dimensional Subspaces: Orthogonal Bottlenecks for Reinforcement Learning
May 25, 2026Aleksandar Todorov, Matthia SabatelliNeural Network Representations Subspace
University of Groningen, Groningen, The Netherlands
Subspace-Guided Semantic and Topological Invariant Registration for Annotation-Free Ultrasound Plane Quality Control
May 25, 2026Chunzheng Zhu, Jianxin Lin, Feng Wang +5Ultrasound Image Registration
Hunan University, Changsha, China · Shenzhen Maternity and Child Healthcare Hospital, Shenzhen, China
Data-Specific Hyper-Parameter Design: A Paradigm Shift in Reservoir Computing
May 24, 2026G Manjunath, Juan-Pablo Ortega, Alma van der MerweReservoir Computing Reservoir
University of Pretoria. Department of Mathematics and Applied Mathematics. Pretoria 0002, South Africa. · Nanyang Technological University. Division of Mathematical Sciences. School of Physical and Mathematical Sciences. Singapore.
Steering Beyond the Support: Adversarial Training on Unsupervised Jailbroken Activation Simulation
May 23, 2026Luoyu Chen, Weiqi Wang, Zhiyi Tian +5Jailbreak Attacks Adversarial Training
University of Technology Sydney, Sydney, Australia · Xi’an Jiaotong University, Xi’an, China · School of Cyber Science and Engineering, Southeast University, Nanjing, China.
Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignment
May 22, 2026Haoyuan Wang, Xiaohao Liu, Jiajie Su +2Multimodal Knowledge Editing Multimodal Understanding
Zhejiang University · National University of Singapore · Jiangxi Normal University
Automated Random Embedding for Practical Bayesian Optimization with Unknown Effective Dimension
May 22, 2026Hong Qian, Xiang Shu, Xiang Xia +5Bayesian Optimization Subspace
Shanghai Institute of AI for Education, and School of Computer Science and Technology, East China Normal University, Shanghai 200062, China · Ant Group, Hangzhou 310013, China · Nanjing University, Nanjing 210023, China
Self-Policy Distillation via Capability-Selective Subspace Projection
May 21, 2026Guangya Hao, Yitong Shang, Yunbo Long +2Subspace
University of Cambridge · 2HKUST · University of Chicago
The ASE-LSE Disagreement Landscape: An End-to-End Characterisation of Extremes and Structural Drivers
May 21, 2026Minh Triet Pham, Ian GallagherGraph Edge Sparsification Heterogeneity
School of Mathematics and Statistics The University of Melbourne, Australia
Check Your LLM's Secret Dictionary! Five Lines of Code Reveal What Your LLM Learned (Including What It Shouldn't Have)
May 21, 2026Hisashi MiyashitaSingular Value Decomposition Subspace
Mgnite Inc.
Geometry-Adaptive Explainer for Faithful Dictionary-Based Interpretability under Distribution Shift
May 21, 2026Sungjun Lim, Heedong Kim, Andrew Lee +1Interpretability Mechanistic Interpretability
aYonsei University · bHarvard University
You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectories
May 20, 2026Zhepei Wei, Xinyu Zhu, Wei-Lin Chen +3Reinforcement Learning With Verifiable Reward Verifiable Rewards
University of Virginia · Washington University in St. Louis
Scale-Calibrated Median-of-Means for Robust Distributed Principal Component Analysis
May 20, 2026Kisung YouPrincipal Component Analysis Maximum Mean Discrepancy
Department of Mathematics, Baruch College · Department of Mathematics, The Graduate Center, City University of New York
Group-Aware Matrix Estimation and Latent Subspace Recovery
May 19, 2026Hamza Golubovic, Matthew Shen, Genevera I. Allen +1Tensor Completion Low-Rank Structure
Department of Statistics, Columbia University New York, NY 10027 · Department of Statistics, Irving Institute for Cellular Dynamics, Zuckerman Institute Columbia University New York, NY 10027 · School of Data and Information Sciences, University of North Carolina at Chapel Hill Chapel Hill, NC 27599
Axiomatizing Neural Networks via Pursuit of Subspaces
May 19, 2026Mehmet Yamac, Mert Duman, Ugur Akpinar +4Neural Network Subspace
Tampere University, Faculty of ITC, Finland · Department of Electrical Engineering, Qatar University, Qatar · Donders Institute, Radboud University, The Netherlands
iGSP:Implicit Gradient Subspace Projection for Efficient Continual Learning of Vision-Language Models
May 19, 2026Xuezhi Cui, Dongbo Zhou, Wang Guo +8Vision-Language Model Adaptation Subspace
School of Geosciences and Info-Physics, Central South University, Changsha 410083, China · Faculty of Artifi cial Intelligence in Education in Central China Normal University, Wuhan 430079, China · School of Earth Sciences and Spatial Information Engineering, Hunan University of Science and Technology, Xiangtan 411201, China
Modality-Decoupled Online Recursive Editing
May 19, 2026Siyuan Li, Youyuan Zhang, Fangming Liu +1Multimodal Knowledge Editing Multimodal Large Language Models
Harbin Institute of Technology, Shenzhen, China. · Peng Cheng Laboratory, China. · Huazhong University of Science and Technology, China.
Catching a Moving Subspace: Low-Rank Bandits Beyond Stationarity
May 18, 2026Hamed Khosravi, Xiaoming HuoContextual Bandit Framework Bandits
H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332
A Simplex Witness Certificate and Escape Force for Constant Collapse in Variational Autoencoders
May 18, 2026Zegu Zhang, Jian ZhangConditional Variational Autoencoder Collapse
Independent Researcher
Sample-efficient inductive matrix completion with noise and inexact side-information
May 16, 2026Yuepeng Yang, Cong MaTensor Completion Subspace
Department of Statistics and Data Sciences, Yale University · Department of Statistics, University of Chicago
The Geometry of Projection Heads: Conditioning, Invariance, and Collapse
May 16, 2026Faris ChaudhryProjection Heads Self-Supervised Learning
Department of Computing, Imperial College London, United Kingdom.
Contrastive Conceptor Activation Steering (COAST): Unlocking Vision-Language-Action Models through Hidden States
May 16, 2026Miranda Muqing Miao, Subin Kim, Brandon Yang +1Recent Vision-Language Models Linear Activation Steering
University of Pennsylvania
Scale Determines Whether Language Models Organize Representation Geometry for Prediction
May 16, 2026Weilun XuSubspace Representation Space
School of Computer and Communication Sciences ´Ecole Polytechnique F´ed´erale de Lausanne
Jacobian-Guided Anisotropic Noise Reshaping for Enhancing Representation Utility under Local Differential Privacy
May 16, 2026Youngmok Ha, Viktor Schlegel, Yidan Sun +1Subspace Distributional Fidelity
Department of Bioengineering, Imperial College London, United Kingdom · Artificial Intelligence Computing Research Laboratory, Electronics and Telecommunications Research Institute (ETRI), Daejeon, Republic of Korea · Imperial Global Singapore, Imperial College London, Singapore +1
Unsupervised Domain Shift Detection with Interpretable Subspace Attribution
May 15, 2026Sebastian Springer, Alessandro LaioMitigating Domain Shift Electrocardiography
SISSA
Average Gradient Outer Product in kernel regression provably recovers the central subspace for multi-index models
May 14, 2026Libin Zhu, Damek Davis, Dmitriy Drusvyatskiy +1Kernel Ridge Regression Kernel Method
Department of Mathematics, University of Washington, Seattle, WA 98195 · Wharton Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, PA 19104, USA · Department of Mathematics, U. Washington, Seattle, WA 98195 +1
nASR: An End-to-End Trainable Neural Layer for Channel-Level EEG Artifact Subspace Reconstruction in Real-Time BCI
May 14, 2026Shantanu Sarkar, Jose L. Contreras-VidalElectroencephalography Brain-Computer Interface
Electrical & Computer Engineering, Univ. of Houston, Houston, TX, USA · Faculty of Electrical & Computer Engineering, Univ. of Houston, Houston, TX, USA
Collaborative Yet Personalized Policy Training: Single-Timescale Federated Actor-Critic
May 14, 2026Leo Muxing Wang, Pengkun Yang, Lili SuSoft Actor-Critic Multi-Agent Reinforcement Learning
Northeastern University · Tsinghua University
Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimization
May 13, 2026Yang Bai, Kaiyuan Liu, Ziyuan Zhuang +5Reward-Only Policy Optimization Variants Subspace
Meituan, China
On the Limits of Latent Reuse in Diffusion Models
May 13, 2026Yifeng Yu, Lu YuDiffusion Models Latent Space
Department of Mathematical Sciences, Tsinghua University · Department of Data Science, City University of Hong Kong
In-Context Learning Operates as Concept Subspace Learning
May 12, 2026Wei Tang, Xinyan Jiang, Fakhri Karray +1In-Context Learning Subspace
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · Shanghai Advanced Research Institute, Chinese Academy of Science
BROS: Bias-Corrected Randomized Subspaces for Memory-Efficient Single-Loop Bilevel Optimization
May 11, 2026Hengrui Zhang, Boao Kong, Engao Zhang +1Bilevel Optimization Subspace
1Sichuan University · 2Peking University
Flag Varieties: A Geometric Framework for Deep Network Alignment
May 11, 2026Jingchuan Xiao, Xinyi Sui, Cihan RuanNeural Collapse Subspace
Department of Mathematics and Computer Studies, Mary Immaculate College, Ireland · Department of Computer Science and Engineering, Santa Clara University, USA
Beyond Language: Format-Agnostic Reasoning Subspaces in Large Language Models
May 10, 2026Aojie Yuan, Zhiyuan SuLLM Reasoning Strategies Subspace
University of Southern California · 2Duke University
Through the Lens of Character: Resolving Modality-Role Interference in Multimodal Role-Playing Agent
May 10, 2026Yihong Tang, Kehai Chen, Xuefeng Bai +1Multimodal Agents Modalities
Harbin Institute of Technology, Shenzhen, China · Shenzhen Loop Area Institute (SLAI), Shenzhen, China
Multi-scale Predictive Representations for Goal-conditioned Reinforcement Learning
May 10, 2026Valliappan Chidambaram Adaikkappan, David Meger, Sai Rajeswar +1Goal-Conditioned Reinforcement Learning Representation Learning
Mila, McGill University · Qualcomm Research
A Geometric Perspective on Next-Token Prediction in Large Language Models: Three Emerging Phases
May 9, 2026Gianfranco Lombardo, Giuseppe Trimigno, Stefano CagnoniNext-Token Prediction Layer-Wise
Department of Engineering and Architecture University of Parma Parma, IT 43125
Rotation-Preserving Supervised Fine-Tuning
May 8, 2026Hangzhan Jin, Tianwei Ni, Lu Li +3Model Fine-Tuning Subspace
1Mila - Quebec AI Institute · 2Polytechnique Montréal · 3Université de Montréal +3
Context-Gated Associative Retrieval: From Theory to Transformers
May 8, 2026Moulik Choraria, Argyrios Gerogiannis, Vidhata Jayaraman +2Dense Associative Memory In-Context Learning
UIUC · University of Minnesota, Twin Cities · Stony Brook University
Tree SAE: Learning Hierarchical Feature Structures in Sparse Autoencoders
May 8, 2026Tue M. Cao, Hoang X. Nhat, Raed Alharbi +2Improving Sparse Autoencoders Hierarchical
Hanoi University of Science and Technology, Hanoi, Vietnam · University of Florida, Florida, USA · Computer Science Department, Saudi Electronic University
NSPOD: Accelerating Krylov solvers via DeepONet-learned POD subspaces
May 8, 2026Francesc Levrero-Florencio, Youngkyu Lee, Jay Pathak +1Algebraic Multigrid Solver Iterations
Synopsys UK Ltd, UK · Division of Applied Mathematics, Brown University, Rhode Island, USA · Synopsys Inc, California, USA
Bayesian Fine-tuning in Projected Subspaces
May 8, 2026Viktar Dubovik, Patryk Marszałek, Jacek Tabor +1Parameter-Efficient Fine-Tuning Methods Low-Rank Structure
Jagiellonian University, Kraków, Poland · Doctoral School of Exact and Natural Sciences, Jagiellonian University, Kraków, Poland
Hierarchical Dual-Subspace Decoupling for Continual Learning in Vision-Language Models
May 8, 2026Mengxin Qin, Xiang Zhang, Kun Wei +2Continual Learning Vision-Language Model Adaptation
School of Electronic Engineering, Xidian University Xi’an 710071, China
Adaptive Subspace Projection for Generative Personalization
May 8, 2026Van-Anh Nguyen, Anh Tuan Bui, Tamas Abraham +5Personalization Subspace
Monash University, Australia · Defence Science and Technology Group, Australia
TAS-LoRA: Transformer Architecture Search with Mixture-of-LoRA Experts
May 8, 2026Jeimin Jeon, Hyunju Lee, Bumsub HamSelf-Supervised Vision Transformers Hip-Lora
Yonsei University · Articron Inc. · Korea Institute of Science and Technology (KIST)
Efficient Techniques for Data Reconstruction, with Finite-Width Recovery Guarantees
May 7, 2026Edward Tansley, Roy Makhlouf, Estelle Massart +1Subspace Infinities
Mathematical Institute, Woodstock Road, University of Oxford, Oxford, UK, OX2 6GG. · ICTEAM Institute, UCLouvain, Euler Building, Avenue Georges Lemaˆıtre, 4 - bte L4.05.01, Louvain-la-Neuve, B - 1348, Belgium.
From Coordinate Matching to Structural Alignment: Rethinking Prototype Alignment in Heterogeneous Federated Learning
May 7, 2026Xinghao Wu, Jianwei Niu, Guogang Zhu +3Heterogeneous Federated Learning Learnable Prototypes
State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering, Beihang University, Beijing 100091, China · Zhongguancun Laboratory, Beijing 100194, China · Center for AI Business Innovation, Department of Management Science and Systems, School of Management, University at Buffalo, NY 14260 USA
Jointly Learning Structured Representations and Stabilized Affinity for Human Motion Segmentation
May 7, 2026Xianghan Meng, Zhiyuan Huang, Zhengyu Tong +1Temporal Action Segmentation Latent Motion
School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100876, P.R. China
CPCANet: Deep Unfolding Common Principal Component Analysis for Domain Generalization
May 6, 2026Yu-Hsi Chen, Abd-Krim SeghouaneMultimodal Domain Generalization Subspace
The University of Melbourne
Conceptors for Semantic Steering
May 6, 2026Ilias Triantafyllopoulos, Young-Min Cho, Ren Tao +6Linear Activation Steering Steering
1New York University · University of Pennsylvania
CERSA: Cumulative Energy-Retaining Subspace Adaptation for Memory-Efficient Fine-Tuning
May 5, 2026Jingze Ge, Xue Geng, Yun Liu +6Parameter-Efficient Fine-Tuning Methods Pretraining
National University of Singapore · Institute for Infocomm Research (I2R), A*STAR · Nankai University +1
On the Spectral Structure and Objective Equivalence of Orthogonal Multilabel Fisher Discriminants
May 5, 2026Brian Keith-Norambuena, Juan Bekios-CalfaMulti-Label Classification Discriminative Models
Department of Computing & Systems Engineering, Universidad Cat´olica del Norte, Antofagasta, Chile.
Beyond Activation Alignment: The Geometry of Neural Sensitivity
May 4, 2026Amirhossein Yavari, Farnaz Zamani EsfahlaniRepresentational Similarity Analysis Subspace
Stephenson School of Biomedical Engineering, University of Oklahoma · Data Science and Analytics Institute, University of Oklahoma
Gated Subspace Inference for Transformer Acceleration
May 4, 2026Stephen J. ThomasTransformer Architectures Subspace
Department of Computer Science and Engineering, Lehigh University, Bethlehem, PA 18015
Compress Then Adapt? No, Do It Together via Task-aware Union of Subspaces
May 4, 2026Jingze Ge, Yun Liu, Xue Geng +4Subspace Low-Rank Structure
National University of Singapore · Nankai University · Institute for Infocomm Research (I2R), A*STAR
Bucketing the Good Apples: A Method for Diagnosing and Improving Causal Abstraction
May 4, 2026Li Puyin, Jiyuan Tan, Ahmad Jabbar +2Causal Inferences Abductive Reasoning
♢Stanford University · ♠Goodfire
A framework for analyzing concept representations in neural models
May 2, 2026Burin Naowarat, Hao Tang, Sharon GoldwaterSubspace Concept Erasure
The Centre for Speech Technology Research School of Informatics, University of Edinburgh, United Kingdom
Learning in the Fisher Subspace: A Guided Initialization for LoRA Fine-Tuning
May 1, 2026Zhi-Quan Feng, Ying-Jia Lin, Hung-Yu KaoSubspace Rapid Adaptation
Department of Computer Science and Information Engineering, National Cheng Kung University · Department of Artificial Intelligence, Chang Gung University · Department of Computer Science, National Tsing Hua University
Empowering Heterogeneous Graph Foundation Models via Decoupled Relation Alignment
May 1, 2026Ziyu Zheng, Yaming Yang, Zhe Wang +2Graph Foundation Model Subspace
School of Computer Science and Technology, Xidian University, Xi’an, China 710071
Do Sparse Autoencoders Capture Concept Manifolds?
Apr 30, 2026Usha Bhalla, Thomas Fel, Can Rager +9Manifolds Subspace
aHarvard University · bNortheastern University · cTechnion IIT +1
BoostLoRA: Growing Effective Rank by Boosting Adapters
Apr 30, 2026Raviteja Anantha, Nick Levato, Layne C. PriceParameter-Efficient Fine-Tuning Methods Multi-Lora
Amazon

May 25, 2026, cs.LGJ/K move · Enter open · S save

Learning in Low-Dimensional Subspaces: Orthogonal Bottlenecks for Reinforcement Learning

Aleksandar Todorov, Matthia Sabatelli

University of Groningen, Groningen, The Netherlands

Neural Network Representations Subspace Bottlenecks

Open paper AskPDF arXiv

Deep reinforcement learning (RL) agents commonly rely on high-dimensional neural representations, despite growing evidence that task-relevant value and policy structure may be intrinsically low-dimensional. In this work, we present a simple yet effective representation-level prior that inserts a fixed orthonormal projection to constrain encoder features to a low-dimensional subspace, requiring no auxiliary objectives, pretraining, or changes to the underlying RL algorithm. Under a linear realizability assumption, we prove that when the bottleneck dimension exceeds the intrinsic rank of the optimal value function in feature space, the bottleneck preserves expressivity and leaves the induced gradient dynamics unchanged up to an equivalent low-dimensional parameterization. Empirically, we find that across both single and multi-task benchmarks, baseline performance is either matched or improved once the bottleneck dimension exceeds a small task-dependent threshold; in many cases, value representations can be compressed to extremely low dimensions without loss, and the minimal sufficient dimension depends far more on environment complexity than encoder width. In addition, we analyze representation geometry and find that orthogonal bottlenecks stabilize feature norms and are associated with higher effective rank. Together, these results support a representation-space interpretation of the manifold hypothesis in reinforcement learning and position orthogonal bottlenecks as a lightweight, architecture-agnostic mechanism for shaping RL representations.

Momentum

Related topics

Latest papers 279