Subspace

Follow this topic Ask about this topic

Momentum

38 papers in the last four weeks, up 111% on the four weeks before. 0.5% of all new papers.

Jul 6Week of Sep 21

Latest papers 279

All topics

Decoupling Knowledge and Task Subspaces for Composable Parametric Retrieval Augmented Generation
Apr 29, 2026Weihang Su, Hanwen Zhang, Qingyao Ai +1Agentic Retrieval-Augmented Generation Systems Multimodal Retrieval Augmented Generation
Department of Computer Science and Technology, Tsinghua University
CoQuant: Joint Weight-Activation Subspace Projection for Mixed-Precision LLMs
Apr 29, 2026Zhe Ding, Su Pan, Duowei PanLarge Language Model Quantization Post-Training Quantization
School of Internet of Things, Nanjing University of Posts and Telecommunications, Nanjing, China · Amazon AGI, Seattle, WA, USA
Subspace Optimization for Efficient Federated Learning under Heterogeneous Data
Apr 28, 2026Shuchen Zhu, Zhengyang Huang, Yuqi Xu +1Heterogeneous Federated Learning Federated Learning
Peking University · Beihang University
Dynamic Regret for Online Regression in RKHS via Discounted VAW and Subspace Approximation
Apr 27, 2026Dmitry B. Rokhlin, Georgiy A. KarapetyantsReproducing Kernel Hilbert Spaces Kernel Method
Institute of Mathematics, Mechanics and Computer Sciences of the Southern Federal University
Operational Feature Fingerprints of Graph Datasets via a White-Box Signal-Subspace Probe
Apr 24, 2026Yuchen Xiong, Swee Keong Yeap, Zhen Hong BanGraph Representation Learning Fingerprint
China-ASEAN College of Marine Sciences · Xiamen University Malaysia · Sepang 43900, Selangor, Malaysia +1
Robust Fuzzy local k-plane clustering with mixture distance of hinge loss and L1 norm
Apr 24, 2026Junjun Huang, Xiliang Lu, Xuelin Xie +1Clustering Principal Component Analysis
China Electric Power Research Institute, Wuhan 430074, China, and also with the Key Laboratory of Measurement and Test of High Voltage and Heavy Current, State Administration for Market Regulation, Wuhan 430074, China · School of Mathematics and Statistics, Wuhan University, Wuhan 430072, China, the National Center for Applied Mathematics in Hubei, Wuhan University, Wuhan 430072, China, and also with the Hubei Key Laboratory of Computational Science, Wuhan University, Wuhan 430072, China · National Center for Applied Mathematics in Hubei, Wuhan University, Wuhan 430072, China, the Wuhan Institute for Math and AI, Wuhan University, Wuhan 430072, China, the School of Mathematics and Statistics, Wuhan University, Wuhan 430072, China, and the Hubei Key Laboratory of Computational Science, Wuhan University, Wuhan 430072, China
Logistic Bandits with $\tilde{O}(\sqrt{dT})$ Regret without Context Diversity Assumptions
Apr 24, 2026Seoungbin Bae, Dabeen Lee\Widetilde{\Mathcal{O}}(\Sqrt{T})$ Regret Contextual Bandit Framework
Department of Industrial & Systems Engineering, KAIST · Department of Mathematical Sciences, Seoul National University
Toward Efficient Membership Inference Attacks against Federated Large Language Models: A Projection Residual Approach
Apr 23, 2026Guilin Deng, Silong Chen, Yuchuan Luo +6Membership Inference Attacks Federated Learning
Colleague of Computer Science and Technology, National University of Defense Technology, Changsha, China · City University of Hong Kong, Hong Kong, China · Shenzhen University, Shenzhen, China
FASQ: Flexible Accelerated Subspace Quantization for Calibration-Free LLM Compression
Apr 22, 2026Ye Qiao, Yian Wang, Zhiheng Chen +2Large Language Model Compression LLM Inference Optimization
University of California, Irvine Irvine, California, USA
FB-NLL: A Feature-Based Approach to Tackle Noisy Labels in Personalized Federated Learning
Apr 21, 2026Abdulmoneam Ali, Ahmed ArafaFederated Learning Noisy Labels
Department of Electrical and Computer Engineering, University of North Carolina at Charlotte, NC 28223, USA
Discovering a Shared Logical Subspace: Steering LLM Logical Reasoning via Alignment of Natural-Language and Symbolic Views
Apr 21, 2026Feihao Fang, My T. Thai, Yuanyuan LeiLLM Reasoning Strategies Reasoning Chain
University of Illinois Urbana-Champaign, Champaign, IL · Computer & Information Science and Engineering, University of Florida, Gainesville, FL
VCE: A zero-cost hallucination mitigation method of LVLMs via visual contrastive editing
Apr 21, 2026Yanbin Huang, Yisen Li, Guiyao Tie +6Hallucination Mitigation Object Hallucination
1Huazhong University of Science and Technology · Institute of Artificial Intelligence (TeleAI), China Telecom
TLoRA: Task-aware Low Rank Adaptation of Large Language Models
Apr 20, 2026Weicheng Lin, Yi Zhang, Jiawei Dang +1Subspace Data-Driven
College of Computer Science and Software Engineering, Shenzhen University, China
RASP-Tuner: Retrieval-Augmented Soft Prompts for Context-Aware Black-Box Optimization in Non-Stationary Environments
Apr 20, 2026Enze PanBlack-Box Optimization Tuning
Department of Computer Science, The University of Hong Kong.
Evolutionary Negative Module Pruning for Better LoRA Merging
Apr 20, 2026Anda Cao, Zhuo Gou, Yi Wang +5Large Language Model Adaptation Large Language Model Compression
College of Computer Science and Technology, Zhejiang University · School of Software Technology, Zhejiang University · 3State Key Laboratory of Blockchain and Security, Zhejiang University +1
SafeAnchor: Preventing Cumulative Safety Erosion in Continual Domain Adaptation of Large Language Models
Apr 20, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuLarge Language Model Adaptation Domain Adaptation
The University of Hong Kong, Hong Kong, China · Brain Investing Limited, Hong Kong, China
Leveraging Kernel Symmetry for Joint Compression and Error Mitigation in Edge Model Transfer
Apr 19, 2026Anis Hamadouche, Mathini SellathuraiEquivariant Neural Networks Transmission
School of Engineering & Physical Sciences Heriot-Watt University Edinburgh EH14 4AS, UK
One Mask to Rule Them All: On Hidden Facts after Editing and How to Find Them
Apr 18, 2026Ali Holmov, Paul Youssef, Nandi Schoots +1Multimodal Knowledge Editing Modifications
Technical University of Munich · Marburg University · University of Oxford
Curvature-Aligned Probing for Local Loss-Landscape Stabilization
Apr 16, 2026Nikita Kiselev, Andrey GrabovoyAnisotropic Loss Landscapes Curvature-Aware Spectral Framework
Department of Intelligent Systems Moscow Institute of Physics and Technology
Algebraic Diversity: Group-Theoretic Spectral Estimation from Single Observations
Apr 4, 2026Mitchell A. ThorntonBeamforming Covariance
Richardson, TX 75080 USA
GRADE: Probing Knowledge Gaps in LLMs through Gradient Subspace Dynamics
Apr 3, 2026Yujing Wang, Yuanbang Liang, Yukun Lai +2Reasoning Gaps Gradient
Beihang University · Cardiff University · King’s College London
Over-Refusal and Representation Subspaces: A Mechanistic Analysis of Task-Conditioned Refusal in Aligned LLMs
Mar 29, 2026Utsav Maskey, Mark Dras, Usman NaseemRefusals Large Language Models Fail
Macquarie University
Project and Mix: Task-Semantic Prototypes for Few-Shot Image Classification
Mar 25, 2026Dipam Goswami, Simone Magistri, Gido M. van de Ven +4Few-Shot Learning Learnable Prototypes
Department of Computer Science, Universitat Autònoma de Barcelona, Spain · Computer Vision Center, Barcelona, Spain · Media Integration and Communication Center, University of Florence, Italy +3
Teacher-Guided Causal Interventions for Image Denoising: Orthogonal Content-Noise Disentanglement in Vision Transformers
Mar 1, 2026Kuai Jiang, Zhaoyan Ding, Guijuan Zhang +2Iterative Denoising Methods Self-Supervised Vision Transformers
China University of Mining and Technology · Shandong Normal University · Qilu University of Technology
Provable Subspace Identification of Nonlinear Multi-view CCA
Feb 27, 2026Zhiwei Han, Stefan Matthes, Hao ShenIdentifiability Subspace
Chair of Data Processing, Technical University of Munich, Germany · fortiss GmbH, Munich, Germany
NeuroWeaver: An Autonomous Evolutionary Agent for Exploring the Programmatic Space of EEG Analysis Pipelines
Feb 13, 2026Guoan Wang, Shihao Yang, Feng LiuSmall-Sample Electroencephalogram Datasets Electroencephalography
Department of Industrial and Systems Engineering, Rutgers, The State University of New Jersey, 96 Frelinghuysen Road, Piscataway, NJ 08854, USA
Functional Subspace, where language models can use vector algebra to solve problems
Feb 2, 2026Jung H. Lee, Sujith VijayanSubspace In-Context Learning
Pacific Northwest National Laboratory Seattle, WA · School of Neuroscience Virginia Tech Blacksburg, VA
XFACTORS: Disentangled Information Bottleneck via Contrastive Supervision
Jan 29, 2026Alexandre Myara, Nicolas Bourriez, Thomas Boyer +3Information Bottleneck Conditional Variational Autoencoder
ENS Paris
Linearized subspace refinement framework to expose hidden accuracy in trained neural networks
Jan 20, 2026Wenbo Cao, Weiwei ZhangLeast Squares Subspace
Institute of AI for Industries, Chinese Academy of Sciences, Nanjing 211135, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China · School of Aeronautics, Northwestern Polytechnical University, Xi’an 710072, China +2
Accelerated MR Elastography Using Learned Neural Network Representation
Jan 17, 2026Xi PengMagnetic Resonance Imaging Reconstruction Magnetic Resonance Imaging
Departments of Radiology, Biomedical Engineering, Electronic and Computer Engineering, and Mechanical Engineering, University of Iowa, Iowa City, IA 52246 · Department of Radiology, University of Iowa, Iowa City, IA, 52246
Provably Finding a Hidden Dense Submatrix among Many Planted Dense Submatrices via Convex Programming
Jan 7, 2026Valentine Olanubi, Phineas Agar, Brendan AmesConvex Relaxation Subgraphs
University of Alabama, Department of Mathematics · Department of Mathematics, University of Alabama, Tuscaloosa, AL, USA · University of Southampton, School of Mathematical Sciences +1
Geometrically Principled Randomized Optimization for Efficient LLM Training
Oct 2, 2025Sahar Rajabi, Nayeema Nonta, Sirisha RambhatlaLow-Rank Structure Large Language Model Training
CriticalML Lab, University of Waterloo
An Efficient Subspace Algorithm for Federated Learning on Heterogeneous Data
Sep 5, 2025Jiaojiao Zhang, Yizhao Fan, Yuqi Xu +1Heterogeneous Federated Learning Federated Learning
Intelligent Computing Research Center, Great Bay University, Dongguan, China · Center for Machine Learning Research, Peking University, Beijing, China
A Conditional GAN for Tabular Data Generation with Probabilistic Sampling of Latent Subspaces
Aug 1, 2025Leonidas Akritidis, Panayiotis BozanisSynthetic Tabular Data Conditional Gans
Department of Information and Electronic Engineering, International Hellenic University, Thessaloniki, Greece · School of Science and Technology, International Hellenic University, Thessaloniki, Greece
Kernel PCA for Out-of-Distribution Detection: Non-Linear Kernel Selection and Approximation
May 21, 2025Kun Fang, Qinghua Tao, Mingzhen He +6Out-Of-Distribution Detection In-Distribution
Department of Automation, Shanghai Jiao Tong University · Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University · School of Automation, Beijing Institute of Technology +2
The EM-algorithm and the Method of Moments in Softmax Mixture Models
Sep 16, 2024Xin Bing, Florentina Bunea, Jonathan Niles-Weed +1Mixture Models Expectation-Maximization
Department of Statistical Sciences, University of Toronto · Department of Statistics and Data Science, Cornell University · Courant Institute of Mathematical Sciences and Center for Data Science, New York University +1
Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributions
Sep 4, 2024Peng Wang, Huijie Zhang, Zekai Zhang +3Low-Dimensional Structure Diffusion Models
University of Macau · University of Michigan · University of California, Berkeley
Generalizing Adam to Manifolds for Efficiently Training Transformers
May 26, 2023Benedikt BrantnerAdam Manifolds
Max-Planck-Institut für Plasmaphysik Boltzmannstraße 2, 85748 Garching, Deutschland · Technische Universität München, Zentrum Mathematik Boltzmannstraße 3, 85748 Garching, Deutschland
DCReg: Decoupled Characterization for Efficient Degenerate LiDAR Registration
Date pendingXiangcheng Hu, Xieyuanli Chen, Mingkai Jia +3Point Cloud Registration Image Registration
Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong, China · Department of Mechanical Engineering at National University of Defense Technology, Changsha, Hunan · School of Intelligent Science and Technology, University of Science and Technology Beijing, Beijing, China +1

Apr 29, 2026, cs.CLJ/K move · Enter open · S save

Decoupling Knowledge and Task Subspaces for Composable Parametric Retrieval Augmented Generation

Weihang Su, Hanwen Zhang, Qingyao Ai, Yiqun Liu

Department of Computer Science and Technology, Tsinghua University

Agentic Retrieval-Augmented Generation Systems Multimodal Retrieval Augmented Generation Training-Side Stage-Aware Low-Rank Adaptation Search-Augmented Reasoning Subspace Orthogonality

Open paper AskPDF arXiv

Parametric Retrieval-Augmented Generation (PRAG) encodes external documents into lightweight parameter modules that can be retrieved and merged at inference time, offering a promising alternative to in-context retrieval augmentation. Despite its potential, many PRAG implementations train document adapters with task-supervised objectives, which may cause each adapter to encode both document-specific facts and reusable task-solving behavior. This entanglement may make adapter composition less reliable: when multiple adapters are merged at inference time, their overlapping task behaviors can accumulate together with document-specific updates, potentially making the merged adapter less stable and less focused on the intended document knowledge. To examine this issue, we explore Orthogonal Subspace Decomposition (OSD), an adapter-training setup that separates reusable task behavior from document-specific knowledge adapters. Concretely, we first train a Task LoRA to capture reusable task behavior, and then train document LoRAs to encode document-specific knowledge in a orthogonal subspace. This setup provides a controlled way to examine how orthogonalizing task and document LoRA updates affects adapter composition in multi-document PRAG. Experiments across multiple knowledge-intensive tasks and model scales suggest that this orthogonalization strategy can improve compositional robustness in parametric RAG, especially when multiple document adapters are merged.

Momentum

Related topics

Latest papers 279