Krylov Subspace Methods

Follow this topic Ask about this topic

Momentum

0 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 9

All topics

Subspace Levenberg Marquardt Algorithms in Training Neural Networks
Sep 1, 2026M. Duc HoangNeural Network Optimization Krylov Subspace Methods
Department of Mathematics, University of California, Davis, Davis, CA, USA
Eigenbasis-Independent Learnable Spectral Positional Encodings for Directed Graphs via Hermitian Block Krylov Subspaces
Jul 8, 2026Jiaqing Xie, Yuxin WangPositional Encoding Spectral Methods
Fudan University
Memory Is No Longer a Bottleneck: Memory-Efficient Graph Filtering for Scalable Collaborative Filtering
Jun 19, 2026Jin-Duk Park, Won-Yong ShinMemory-Efficient Inference Krylov Subspace Methods
School of Mathematics and Computing (Computational Science and Engineering), Yonsei University, Seoul 03722, Republic of Korea
Causal Unlearning in Collaborative Optimization: Exact and Approximate Influence Reversal under Adversarial Contributions
May 19, 2026Ali Mahdavi, Azadeh Zamanifar, Amirfarhad Farhadi +1Federated Unlearning Krylov Subspace Methods
Department of Computer Engineering SRC, Islamic Azad University Tehran, Iran · School of Computer Engineering Iran University of Science and Technology Tehran, Iran · Meta CA, USA
Adaptive Diagonal Loading using Krylov Subspaces for Robust Beamforming
May 11, 2026Manan Mittal, Ryan M. Corey, John R. Buck +1Krylov Subspace Methods
Stony Brook University1 · University of Illinois Chicago2 · University of Massachusetts Dartmouth3
NSPOD: Accelerating Krylov solvers via DeepONet-learned POD subspaces
May 8, 2026Francesc Levrero-Florencio, Youngkyu Lee, Jay Pathak +1PDE Solving Structural Mechanics
Synopsys UK Ltd, UK · Division of Applied Mathematics, Brown University, Rhode Island, USA · Synopsys Inc, California, USA
Sparse Random-Feature Neural Networks with Krylov-Based SVD for Singularly Perturbed ODE
May 8, 2026Kevin Kurian Thomas Vaidyan, Siddharth RoutNeural PDE Solvers Singular Value Decomposition
Institute of Applied Mathematics University of British Columbia Vancouver, BC V6T 1Z4
Towards Universal Convergence of Backward Error in Linear System Solvers
Apr 17, 2026Michał Dereziński, Yuji Nakatsukasa, Elizaveta RebrovaKrylov Subspace Methods
University of Michigan, Ann Arbor, MI · University of Oxford, Oxford, UK · Princeton University, Princeton, NJ
Scalable Krylov Subspace Methods for Generalized Mixed-Effects Models with Crossed Random Effects
Date pendingPascal Kündig, Fabio SigristLatent Variable Models Krylov Subspace Methods
Lucerne University of Applied Sciences and Arts · University of Basel · Seminar for Statistics, ETH Zurich

Sep 1, 2026, cs.LGJ/K move · Enter open · S save

Subspace Levenberg Marquardt Algorithms in Training Neural Networks

M. Duc Hoang

Department of Mathematics, University of California, Davis, Davis, CA, USA

Neural Network Optimization Krylov Subspace Methods

Open paper AskPDF arXiv

The Levenberg-Marquardt (LM) algorithm is a well-known second-order method for rapid convergence and strong robustness when training small- to medium-sized neural networks (NNs). However, its computational and memory costs increase significantly as the number of parameters in an NN grows. To address this limitation, subspace methods have been proposed, such as the Krylov subspace LM (KSLM) and the hybrid subspace LM (HSLM), making second-order algorithms more efficient. In this work, we evaluate the subspace Levenberg-Marquardt algorithms for regression and classification tasks in neural networks. We compare the performance of subspace LM variants with the classical LM method, as well as other popular first-order algorithms, such as stochastic gradient descent (SGD) and Adam.

Momentum

Related topics

Latest papers 9