Neural Network Training

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 40

All topics

Lock-in EP: An In-Situ Training Algorithm for Oscillatory Hardware
Oct 5, 2026Sowjanya Tammali, Wilkie Olin-AmmentorpAnalog Oscillatory Dynamics
Computer Science, Missouri University of Science and Technology, 500 West 15th Street, Rolla, 65409, MO, USA. · Mathematics & Computer Science Division, Argonne National Laboratory, 9700 S. Cass Avenue, Lemont, 60439, IL, USA.
Training Witnesses: Trusting the Training without Trusting the Trainer
Sep 27, 2026Houjun Liu, Pratyusha SharmaModel Training Witnesses
Microsoft Research
Training Neural Networks to Approach the Optimum Bayes Estimator in Dense Multi-Emitter Localization
Sep 17, 2026Yi Sun, Mona Sharifi, Muzna YummanNeural Network Training Neural Network
Electrical Engineering Department Applied and Embodied AI Lab The City College of City University of New York New York, NY 10031, USA
Subspace Levenberg Marquardt Algorithms in Training Neural Networks
Sep 1, 2026M. Duc HoangNeural Network Training Neural Network
Department of Mathematics, University of California, Davis, Davis, CA, USA
Threshold-Based Early Stopping of Accumulations in Neural Networks with Binary Activation
Aug 6, 2026Quentin Luquet de Saint-Germain, Massil Ait Abdeslam, Jean Pierre DavidEarly Stopping Neural Network Training
Polytechnique Montréal, Department of Electrical Engineering Montréal, Canada
Convergent Evolution in Neural Representation Space: Emergent Order in Deep Belief Networks
Aug 6, 2026Patrick Krauss, Achim Schilling, Andreas Maier +2Modern Deep Networks Neural Network Training
Cognitive Computational Neuroscience Group, Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nürnberg (FAU), Germany · Neuromodulation and Neuroprosthetics, University Hospital Mannheim, University Heidelberg, Germany · Neuroscience Lab, University Hospital Erlangen, Germany +1
How Much Reconstruction Does Quantum Machine Learning Need? Late Fusion of Independently Trained Quantum Subcircuits
Aug 6, 2026Prabhjot Singh, Adel N. Toosi, Rajkumar BuyyaQuantum Machine Learning Entanglement
Quantum Cloud Computing and Distributed Systems (qCLOUDS) Lab, School of Computing and Information Systems, The University of Melbourne, Melbourne, VIC, 3010, Australia
Predicting Deep Neural Network Training Outcomes from Early Training Telemetry
Aug 4, 2026Ranjita Naik, Anh D. Nguyen, Pankaj Kumar SinghNeural Network Training Epoch
Georgia Institute of Technology
CoRe-GNN: Multilevel Message passing on Coarsened graphs
Aug 3, 2026Antonin Joly, Nicolas Keriven, Aline RoumyGraph Neural Networks Gnn-Based Detectors
CNRS, IRISA, Rennes, FRANCE · INRIA, Rennes, FRANCE
A Comparison of Data Augmentation Methods for Training Deep Neural Networks on Synthetic Aperture Sonar
Jul 26, 2026C. J. Moore, Gregory D. Vetaw, Jordan MalofAugmentation Synthetic Aperture Radar
University of Missouri, Columbia, Missouri, USA · Naval Surface Warfare Center Panama City Division, Panama City, Florida, USA
Hardware-Software Co-Design for Float16 On-Device Training on RISC-V Single-Core
Jul 23, 2026Benjamin Hubinet, Pierre-Alain Moellic, Olivier Savry +2Risc-V Matched Fp16 Intermediate
CEA-Leti, Mines Saint-Etienne, Equipe Commune SAS, F-13541 Gardanne, France · Univ. Grenoble Alpes, CEA-Leti, F-38000 Grenoble, France · Mines Saint-Etienne, CEA-Leti, Centre CMP, Equipe commune SAS, F-13541 Gardanne, France
New Complexity-Theoretic Frontiers of Tractability for Neural Network Training
Jul 23, 2026Cornelius Brand, Robert Ganian, Mathis RoctonRectified Linear Unit Networks Neural Network Training
Algorithms & Complexity Group Vienna University of Technology Favoritenstraße 9-11, 1040 Vienna, Austria
Hyperparameter Transfer in Graph Neural Networks
Jul 6, 2026Gage DeZoort, Boris HaninGraph Neural Networks Hyperparameter
Department of Operations Research and Financial Engineering · School of Engineering and Applied Science · Princeton University +1
Unified convergence analysis for gradient descent optimization methods in the training of deep neural networks
Jul 5, 2026Shokhrukh Ibragimov, Arnulf JentzenNeural Network Optimization Gradient Descent
Applied Mathematics: Institute for Analysis and Numerics, University of Münster, Germany · School of Data Science and School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen), China
Escaping Iterative Parameter-Space Noise: Differentially Private Learning with a Hypernetwork
Jun 25, 2026Naoki Nishikawa, Shokichi Takakura, Satoshi HasegawaNeural Network Training Stochastic Gradient Descent
The University of Tokyo · LY Corporation
InTrain: Intrinsic Trainability for Zero-Cost Neural Architecture Search
Jun 17, 2026Qinqin Zhou, Fuhai Chen, Jipeng Wu +3Neural Architecture Search Neural Network Training
School of Computer and Data Science, Fuzhou University · School of Computer and Data Science, Minjiang University · School of Artificial Intelligence, Nanchang University +2
Optimizing Energy-based Neural Network Training with Coherent Ising Machine
Jun 8, 2026Chen-Rui Fan, Bo Lu, Zhi-Hong Zhang +3Neural Network Training Spatial Photonic Ising Machines
School of Artiﬁcial Intelligence, Beijing Normal University, Beijing, 100875, China. · Laboratory for Advanced Computing and Intelligence Engineering, Information Engineering University, Zhengzhou, 450001, China. · China Mobile (Suzhou) Software Technology Company Limited, Suzhou, 215163, China.
Rethinking Evaluation Paradigms in IBP-based Certified Training
Jun 1, 2026Konstantin Kaulen, Hadar Shavit, Holger H. HoosNeural Network Verification Certification
Chair of AI Methodology, RWTH Aachen University, Aachen, Germany · LIACS, Leiden University, Leiden, The Netherlands
On the Difficulty of Learning a Meta-network for Training Data Selection
May 30, 2026Zilin Du, Junqi Zhao, Boyang Albert LiMeta-Learning Training Data
College of Computing and Data Science, Nanyang Technological University, Singapore.
A2SG:Adaptive and Asymmetric Surrogate Gradients for Training Deep Spiking Neural Networks
May 29, 2026Yechan Kang, Yongjin Kweon, Mingyeong Seo +8Spiking Neural Networks Surrogate Gradient
Center for Semiconductor Technology, Korea Institute of Science and Technology (KIST), Seoul, Republic of Korea · Department of Computer Science and Engineering, Korea University, Seoul, Republic of Korea · LIG Defense&Aerospace, Seongnam, Republic of Korea +1
Do Deep Networks Forget Initialization? A Forgetting-Time View of Practical Inductive Bias
May 27, 2026Mohua Das, Pierfrancesco Beneventano, Shibshankar Dey +2Neural Network Training Batch
MIT · Northwestern University
Worker Disagreement Reveals Sharp Directions in Local SGD
May 26, 2026Tolga Dimlioglu, Kristi Topollai, Anna ChoromanskaHessian Stochastic Gradient Descent
New York University, New York, USA
Anytime Training with Schedule-Free Spectral Optimization
May 21, 2026Anuj Apte, Pranav Deshpande, Niraj Kumar +2Batch Adam
Global Technology Applied Research, JPMorganChase New York, NY 10001, USA
Training Neural Networks with Optimal Double-Bayesian Learning
May 19, 2026Vy Bui, Hang Yu, Karthik Kantipudi +2Stochastic Gradient Descent Batch
Lister Hill National Center for Biomedical Communications, National Library of Medicine National Institutes of Health, Bethesda, MD 20894, USA · Office of Cyber Infrastructure and Computational Biology, National Institute of Allergy and Infectious Diseases National Institutes of Health, Bethesda, MD 20892, USA
Replacement Learning: Training Neural Networks with Fewer Parameters
May 19, 2026Yuming Zhang, Peizhe Wang, Tianyang Han +5Neural Network Training Backpropagation
The University of Hong Kong, Hong Kong, China · Southeast University, Nanjing, China
NeuroTrain: Surveying Local Learning Rules for Spiking Neural Networks with an Open Benchmarking Framework
May 14, 2026Alessio Caviglia, Filippo Marostica, Roberta Bardini +2Spiking Neural Networks Neural Network Training
Politecnico di Torino, Control and Computer Engineering Department, Corso Duca degli Abruzzi 24, Torino, I-10129, , Italy
Gradient-Free Training of Spiking Neural Networks via Low-Rank Evolution Strategies
May 14, 2026Dhruv Patankar, Sachit Ramesha GowdaSpiking Neural Networks Neural Network Training
Shunya Research
Detecting overfitting in Neural Networks during long-horizon grokking using Random Matrix Theory
May 12, 2026Hari K. Prakash, Charles H MartinOverfitting Neural Network Training
University of California San Diego Data Science and Engineering · Calculation Consulting San Francisco, CA, USA
PowerStep: Memory-Efficient Adaptive Optimization via $\ell_p$ -Norm Steepest Descent
May 11, 2026Yao Lu, Dengdong Fan, Shixun Zhang +1Adam Neural Network Training
Pengcheng Laboratory · Peking University
Balancing Stability and Plasticity in Sequentially Trained Early-Exiting Neural Networks
May 6, 2026Alaa Zniber, Ouassim Karrakchou, Mounir GhoghoPlasticity Neural Network Training
⋆TICLab, International University of Rabat, Morocco · ‡ School of Electronic and Electrical Engineering, University of Leeds, UK
NeuroViz: Real-time Interactive Visualization of Forward and Backward Passes in Neural Network Training
May 3, 2026Tanvi Sharma, Reza RawassizadehNeural Network Training Visualization
Boston University · Boston, MA, USA
Scalable Hyperparameter-Divergent Ensemble Training with Automatic Learning Rate Exploration for Large Models
Apr 27, 2026Hailing Cheng, Tao Huang, Chen Zhu +1Batch Neural Network Training
LinkedIn Inc. Mountain View, CA 94043
Revisiting Neural Activation Coverage for Uncertainty Estimation
Apr 24, 2026Benedikt Franke, Nils Förster, Frank Köster +3Out-Of-Distribution Detection Uncertainty
DLR Institute for AI Safety and Security, Ulm - Germany · Ruhr University Bochum, Bochum, Germany
Statistical Properties of the King Wen Sequence: An Anti-Habituation Structure That Does Not Improve Neural Network Training
Apr 10, 2026Augustin ChanNeural Network Training Variance
Independent Researcher
$μ$ pscaling small models: Principled warm starts and hyperparameter transfer
Feb 11, 2026Yuxin Ma, Nan Chen, Mateo Díaz +3Hyperparameter Neural Network Training
Department of Applied Mathematics and Statistics Johns Hopkins University
Adaptive Momentum and Nonlinear Damping for Neural Network Training
Jan 30, 2026Aikaterini Karoni, Rajit Rajpal, Benedict Leimkuhler +1Momentum Stochastic Gradient Descent Neural Network Training
School of Mathematics University of Bristol · School of Mathematics University of Edinburgh · CERMICS, CNRS, École des Ponts, Institut Polytechnique de Paris; Inria Paris, France
Dropout Neural Network Training Viewed from a Percolation Perspective
Dec 15, 2025Finley Devlin, Jaron SandersNeural Network Training Neural Network
Second Foundation, Czechia · Eindhoven University of Technology, The Netherlands
Can Stationary Distributions of Scale-Invariant Neural Networks Be Described by the Thermodynamics of an Ideal Gas?
Nov 10, 2025Ildus Sadrtdinov, Ekaterina Lobacheva, Ivan Klimov +3Neural Network Training Stochastic Gradient Descent
Constructor University · Mila – Quebec AI Institute · Université de Montréal +2
Low-rank Orthogonalization for Large-scale Matrix Optimization with Applications to Foundation Model Training
Sep 15, 2025Chuan He, Zhanwang Deng, Zhaosong LuLow-Rank Structure Orthogonality
Department of Mathematics Linköping University, Sweden · Academy for Advanced Interdisciplinary Studies Peking University, Beijing, People’s Republic of China · Department of Industrial and Systems Engineering University of Minnesota, USA
Stochastic Engrams for Efficient Continual Learning
Mar 27, 2025Isabelle Aguilar, Luis Fernando Herbozo Contreras, Omid KaveheiReplay-Based Continual Learning Catastrophic Forgetting
School of Biomedical Engineering, The University of Sydney, NSW, 2006, Australia

Oct 5, 2026, cs.LGJ/K move · Enter open · S save

Lock-in EP: An In-Situ Training Algorithm for Oscillatory Hardware

Sowjanya Tammali, Wilkie Olin-Ammentorp

Computer Science, Missouri University of Science and Technology, 500 West 15th Street, Rolla, 65409, MO, USA. · Mathematics & Computer Science Division, Argonne National Laboratory, 9700 S. Cass Avenue, Lemont, 60439, IL, USA.

Analog Oscillatory Dynamics Neural Network Training

Open paper AskPDF arXiv

Analog hardware platforms offer the potential to reduce energy consumption over digital architectures, but in order to succeed, large-scale analog systems must also be able to operate with or recover from the variability of their components. Towards this goal, we derive and demonstrate the lock-in equilibrium propagation (LIEP) training method. LIEP provides local gradient information for each component in an oscillatory network without separate forward and backward sweeps, potentially allowing for in-situ learning capabilities on analog oscillatory hardware platforms. We demonstrate that LIEP can be used both for ab-initio training as well as recovering performance when pre-trained parameters are perturbed. We show that LIEP can be formulated as a three-factor update rule, and suggest that although the method is currently only validated on shallow networks, alternate architectures may allow it to extend to deep and large-scale networks addressing complex tasks.

Momentum

Related topics

Latest papers 40