Neural Network Optimization

Ask about this topic

Momentum

11 papers in the last four weeks, up 175% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 92

All topics
CardsList

  1. Neural Algorithmic Reasoning for Graph Saddle Point Problems

    Oct 5, 2026Samantha Chen, Jesse He, Coleman Clougherty +2Neural SolversNeural Network Optimization

    Oberlin College · Department of Computer Science, Oberlin College · University of California, San Diego +2

  2. Muon Is Theoretically Wrong For Convolutions, But Empirically Effective

    Oct 5, 2026Thibaut Boissin, Thomas Massena, Mathieu Serrurier +1MuonNeural Network Optimization

    IRIT · IRIT, DTIPG - SNCF, UT3 · Institut de Recherche Technologique Saint-Exupery, France

  3. Learning Pareto Stationary Fronts via Single-Pass Backpropagation

    Oct 5, 2026Elina Rojin Celik, Marcos Medeiros Raimundo, Isabel ValeraMulti--Task LearningNeural Network Optimization

    Department of Computer Science Saarland University, Germany · Instituto de Computação Universidade Estadual de Campinas, Brazil

  4. Bayesian Optimization in Sequence-to-Architecture Latent Space for Zero-Shot NAS

    Oct 5, 2026Ondrej Tybl, Lukas NeumannNeural Architecture SearchBayesian Neural Networks

    Department of Cybernetics FEE Czech Technical University

  5. Trust the Direction, Search the Step: Zero-and-First-Order Methods for LLM Fine-Tuning

    Oct 1, 2026Cristian McGee, El Houcine Bergou, Aritra DuttaZeroth-Order OptimizationZeroth-Order

    University of Central Florida Orlando, FL, USA · Mohammed VI Polytechnic University Ben Guerir, Morocco

  6. SoftServe: A Scalable Quasi-Newton Method for Deep Learning

    Oct 1, 2026Joohwan Ko, Tetiana Parshakova, Diana Cai +1Neural Network OptimizationNewton

    University of Massachusetts Amherst · CCM, Flatiron Institute · Cornell University

  7. MiLoop: Selective Memory Propagation for Neural Combinatorial Optimization

    Oct 1, 2026Changliang Zhou, Yuanyao Chen, Rongsheng Chen +2Neural Combinatorial OptimizationNeural Network Optimization

    School of Automation and Intelligent Manufacturing, Southern University of Science and Technology, Shenzhen, China · Guangdong Provincial Key Laboratory of Fully Actuated System Control Theory and Technology, Southern University of Science and Technology, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China

  8. Neural networks for spectral optimization

    Sep 28, 2026Alexis de Villeroché, Beniamin Bogosel, Stéphane Breuils +2Spectral Representation MethodNeural Partial Differential Equation Solvers

  9. DCEmbed: Scalable Optimization over Neural Surrogates

    Sep 27, 2026Akshay Sreekumar, Nicolas Christianson, Priya L. Donti +2Neural Network OptimizationStochastic Optimization

    Stanford University Stanford, CA, USA · Johns Hopkins University Baltimore, MD, USA · Massachusetts Institute of Technology Cambridge, MA, USA

  10. On the second-order optimization for spiking neural networks

    Sep 24, 2026Ngoc Phu Doan, Ihsen AlouaniSpiking Neural NetworksTime-To-First-Spike

    Centre for Secure Information Technologies Queen’s University Belfast

  11. KATOsuper: Surrogate-accelerated neural topology optimization with sensitivity-consistent Fourier neural operators

    Sep 23, 2026Shengyu Yan, Jasmin JelovicaTopology OptimizationFourier Neural Operators

    Department of Mechanical Engineering, The University of British Columbia, 6250 Applied Science Ln, Vancouver, V6T 1Z4, Canada · Department of Civil Engineering, The University of British Columbia, 6250 Applied Science Ln, Vancouver, V6T 1Z4, Canada

  12. AURA: Angular Update Rate Adaptation for training complex-valued neural networks

    Sep 22, 2026Enrico Ballini, Allan Peter Engsig-Karup, Tito AndriolloNeural Network OptimizationStep Accuracy

    Aarhus University, Aarhus, Denmark · Technical University of Denmark, Kongens Lyngby, Denmark

  13. Self-Supervised Combinatorial Optimization with Constraints via Frank-Wolfe

    Sep 22, 2026Akbar Rafiey, Yifei Xu, Nikolaos KaraliasNeural Combinatorial OptimizationCombinatorial Optimization

    NYU · MIT

  14. Dimension-Corrected Hitting Times for Heavy-Tailed Spectral Emergence in Neural Optimizer Dynamics

    Sep 14, 2026Zongmin LiuLong-Tailed DistributionSpectral

    Stanford University

  15. HyCO: A Hybrid Neural Solver for Combinatorial Optimization

    Sep 7, 2026Yuheng Li, Di Yang, Haipeng Chen +1Neural Combinatorial OptimizationNeural Solvers

    Department of Data Science College of William & Mary

  16. Stochastic Optimization of Tree Tensor Networks

    Sep 1, 2026Marius Willner, Maximilian Scharf, André Uschmajew +2Tensor NetworksNeural Network Optimization

    Institute of Mathematics, University of Augsburg, 86159 Augsburg, Germany · Centre for Advanced Analytics and Predictive Sciences, University of Augsburg, 86159 Augsburg, Germany · Tensor AI Solutions GmbH, 89284 Pfaffenhofen an der Roth, Germany +1

  17. Momentum as Residual-Driven Multiplier Correction for Deep Learning Optimization

    Aug 13, 2026Zhixin Ren, Yau Lyu, Congrong Li +2Neural Network OptimizationMomentum

    Department of Mathmatical Science, Tsinghua University · School of Vehicle and Mobility, Tsinghua University · College of AI, Tsinghua University

  18. Finding the Needle in a Haystack: Test-Time Analog Circuit Representation Adaptation for Bayesian Optimization

    Aug 13, 2026Fin Amin, Sounak Dutta, Paul D. FranzonBayesian OptimizationAnalog Design

    Department of Electrical and Computer Engineering, North Carolina State University, Raleigh, NC, USA

  19. Accelerated Learning of High Dimensional Functions with a Tensor-Featured Training Network

    Aug 11, 2026Karl Pierce, Yuehaw Khoo, Haizhao YangTensor DecompositionHigh-Dimensional

    † The University of Maryland, College Park, College Park MD 20742 USA · ∗The University of Chicago, Chicago, IL 60637 USA

  20. AOS: Adaptive Optimizer Switching via Training-State Signals for Faster Convergence and Better Generalization

    Aug 3, 2026Alok Kumar Pandey, Umang Chaturvedi, Aatish Rana +1Adaptive OptimizersNeural Network Optimization

  21. Geometric Self-Supervised Pre-training for Neural Combinatorial Optimization

    Jul 31, 2026David Aguado, Daniel Fuertes, Carlos R. del-Blanco +1Neural Combinatorial OptimizationNeural Network Optimization

    Grupo de Tratamiento de Imágenes, Information Processing and Telecomunications Center, ETSI Telecomunicación, Universidad Politécnica de Madrid, 28040, Madrid, Spain

  22. Which Hyperparameters Matter? A Game-Theoretic Framework for Interpretable Hyperparameter Sensitivity Analysis

    Jul 17, 2026Nyi Nyi Aung, Heepeom Shin, Abigail Lawlor +1HyperparameterShapley Value

    Department of Mechanical and Industrial Engineering Louisiana State University Baton Rouge, USA · Department of Electrical and Computer Engineering Louisiana State University Baton Rouge, USA

  23. Advancing Optimal Subset Oracle via Learning Relaxation of Neural Set Functions

    Jul 13, 2026Yongquan Shi, Zijing Ou, Shiping Wang +1Variational InferenceSurrogate Gradient

    Fuzhou University · Imperial College London · National University of Singapore

  24. An optimal control approach for neural network architecture adaptation with a posteriori error estimation

    Jul 8, 2026C G Krishnanunni, Thomas Scott, Tan Bui-ThanhNeural ApproximationsNeural Network Optimization

    Department of Aerospace Engineering & Engineering Mechanics, UT Austin · Oden Institute for Computational Engineering and Sciences, UT Austin.

  25. The Anatomy of Implicit Bias: Information Allocation in Neural Network Training

    Jul 8, 2026Zhang Gongyue, Wang Zhiyong, Liu Donghan +3Neural Network OptimizationOptimizer Design

    1State Key Laboratory of Robotics and Systems, Harbin Institute of Technology Shenzhen, Shenzhen, 518055, China.

  26. Unified convergence analysis for gradient descent optimization methods in the training of deep neural networks

    Jul 5, 2026Shokhrukh Ibragimov, Arnulf JentzenNeural Network OptimizationGradient Descent

    Applied Mathematics: Institute for Analysis and Numerics, University of Münster, Germany · School of Data Science and School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen), China

  27. Optimality-Informed Neural Networks for Lunar Landing Trajectory Optimization

    Jul 2, 2026Zhenbo WangIntent-Aligned Autonomous Spacecraft GuidanceOptimal Control

    Department of Mechanical and Aerospace Engineering The University of Tennessee 1512 Middle Drive, Knoxville, TN 37996

  28. Neural Certificate Pricing for Combinatorial Optimization Problems

    Jul 1, 2026Jingyi Chen, Xinyuan Zhang, Xinwu QianNeural Combinatorial OptimizationCombinatorial Optimization

    Department of Civil and Environmental Engineering, Rice University, Houston, TX

  29. GPU Parallelization Strategies for Forward and Backward Propagation in Shallow Neural Networks: A CUDA-Based Comparative Study

    Jun 29, 2026Rania Zitouni, Nadine Bousdjira, Sarah Hasnaoui +2CudaCpu-Gpu Hybrid Designs

    E´ cole Nationale Supe´rieure d’Informatique (ESI) · High Performance Computing (HPC)

  30. Bilevel Optimization for Neural Architecture Search

    Jun 28, 2026Abhishek Shukla, Ankur Sinha, Faiz HamidBilevel OptimizationNeural Architecture Search

    Department of Management Sciences, IIT Kanpur, India · Krishnamurthy Tandon School of AI, IIM Ahmedabad, India

  31. SOAP-Bubbles: Structured Weight Uncertainty for Neural Networks

    Jun 22, 2026Adrian Robert Minut, Nico Daheim, Marco Miani +3Bayesian Neural NetworksNeural Network Optimization

    Sapienza University of Rome, Rome, Italy · Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, Technical University of Darmstadt · Technical University of Denmark, Lyngby, Denmark +3

  32. Provably Sub-Linear Two-Timescale NeuroEvolution with Online Plasticity

    Jun 18, 2026Shishen Lin, Yixin ChenNeuroevolutionLearning-Based Control

    Department of Statistics, University of Warwick, Coventry, United Kingdom · 2Biozentrum, University of Basel, Basel, Switzerland

  33. Evolutionary Two-Stage Hyperparameter Optimization Strategies for Physics-Informed Neural Networks

    Jun 18, 2026Fedor Buzaev, Dmitry Efremenko, Egor Bugaev +4Parametric Physics-Informed Neural NetworkGeneral Grid Search Framework

    HSE University · 1HSE University · AXXX +1

  34. eCNNTO: A Highly Generalizable ConvNet for Accelerating Topology Optimization

    Jun 18, 2026Shengbiao Lu, Xiaodong WeiTopology OptimizationFinite Element Method

    Global college, Shanghai Jiao Tong University, 800 Dongchuan Road, Shanghai, 200240, Shanghai, China

  35. Deep-Unfolded Coordination

    Jun 18, 2026Hunter Kuperman, Minchan Jung, Rahul V. Ghosh +2Deep Unfolding NetworksNeural Network Optimization

    Autonomous Control and Decision Systems Laboratory Georgia Institute of Technology United States

  36. Interpreting Neural Combinatorial Optimization via Evolving Programmatic Bottlenecks

    Jun 18, 2026Haocheng Duan, Yuxin Guo, Jieyi Bi +4Neural Combinatorial OptimizationConcept Bottleneck Models

    1Carnegie Mellon University · 2Nanyang Technological University · 3Microsoft Research +1

  37. Schattor: Schatten-family methods for deep learning optimization

    Jun 14, 2026Bohao Ma, Junyu Zhang, Chuan HeNeural Network OptimizationStochastic Optimization

    School of Data Science, The Chinese University of Hong Kong (Shenzhen), Shenzhen, Guangdong, China · Department of Industrial Systems Engineering and Management, National University of Singapore, Singapore · Department of Mathematics, Linköping University, Sweden

  38. How Much Memory Do We Need? Adaptive Memory Gate for Neural Operators

    Jun 11, 2026Jihyeon Hur, Yongseok Kwon, Min-Gi Jo +2Neural OperatorsNeural Partial Differential Equation Solvers

    1KAIST, Republic of Korea.

  39. Different Layers, Different Manifolds: Module-Wise Weight-Space Geometry in Transformer Optimization

    Jun 11, 2026Kirato YoshiharaTransformer AttentionManifolds

    School of Engineering Science, The University of Osaka, Osaka, Japan.

  40. Accelerating SAV-based optimization via randomized low-rank Hessian approximation

    Jun 9, 2026Ryo Sagawa, Daisuke Furihata, Yuto MiyatakeHessianNewton

    Department of Pure and Applied Mathematics, Graduate School of Information Science and Technology, The University of Osaka · D3 Center, The University of Osaka

  41. Beyond the Thin-Layer Limit: Differentiable Volumetric Training for Visible-Range Diffractive Neural Networks

    Jun 5, 2026Dineth Jayakody, Dushan N. WadduwageModern Deep NetworksDifferentiable Optimization

    Department of Computer Science, Old Dominion University, Norfolk, VA 23529, USA · School of Data Science, Old Dominion University, Norfolk, VA 23529, USA · Department of Physics, Old Dominion University, Norfolk, VA 23529, USA

  42. Amortized Neural Optimization for Pre-Layout Signal Integrity Design Space Exploration using Differentiable Surrogates

    Jun 5, 2026Julian Withöft, Werner John, Emre Ecik +2Neural Network OptimizationOptimal Design

  43. LLM-Guided ANN Index Optimization for Human-Object Interaction Retrieval

    Jun 3, 2026Shahrzad Esmat, Chaunte W. Lacewell, Sameh Gobriel +2RetrieversGeneral Grid Search Framework

    Iowa State University · Intel Corporation

  44. Multi-Column RBF Neural Network Using Adaptive and Non-Adaptive Particle Swarm Optimization

    Jun 3, 2026Ammar Hoori, Yuichi MotaiRadial Basis FunctionParticle Swarm Optimization

    Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA · Department of Electrical and Computer Engineering, Virginia Commonwealth University, Richmond, VA 23284, USA

  45. Beyond Static Priors: Dynamic Neural Guidance for Large-Scale Ant Colony Optimization

    Jun 2, 2026Dat Thanh Tran, Van Khu Vu, Yining MaNeural Combinatorial OptimizationParticle Swarm Optimization

    Center for AI Research VinUniversity Hanoi, Vietnam · College of Engineering and Computer Science VinUniversity Hanoi, Vietnam · Laboratory for Information and Decision Systems Massachusetts Institute of Technology Cambridge, MA, USA

  46. Regularized Large Neighborhood Search

    Jun 1, 2026Germain Vivier-Ardisson, Laurent Demonet, Axel Parmentier +1Neural Combinatorial OptimizationLarge Neighborhood Search

    Google DeepMind & CERMICS · Google Research · CERMICS, ENPC, CNRS, IPP +1

  47. Beyond ℓ2\ell_2-norm and ℓ∞\ell_\infty-norm: A Curvature-Inspired ℓp\ell_p-Norm Scheme for Deep Neural Networks

    Jun 1, 2026Jianhao Xu, Zhuang YangNeural Network OptimizationAdaptive Optimizers

    School of Computer Science and Technology, Soochow University, Suzhou, 215006, China

  48. Singularity-aware Optimization via Randomized Geometric Probing: Towards Stable Non-smooth Optimization

    May 28, 2026Ruoran Xu, Borong She, Xiaobo Jin +1Stochastic Convex OptimizationAdam

    Xi’an Jiaotong-Liverpool University

  49. GeneralThinker: Domain-General Reasoning through Likelihood-Guided Answer-Conditioned Optimization

    May 27, 2026Shengmin Piao, Sanghyun ParkThinkCredit Assignment

    Yonsei University Seoul, South Korea

  50. SparseOpt: Addressing Normalization-induced Gradient Skew in Sparse Training

    May 26, 2026Mohammed Adnan, Rohan Jain, Tom Jacobs +4Batch NormalizationNeural Network Optimization

    University of Calgary · 2Vector Institute · 3CISPA Helmholtz +2

  51. EMA-Nesterov: Stabilizing Nesterov's Lookahead for Accelerated Deep Learning Optimization

    May 25, 2026Chung-Yiu Yau, Dawei Li, Athanasios Glentis +3Momentum Stochastic Gradient DescentNeural Network Optimization

    University of Minnesota · 2Amazon AGI · 3The Chinese University of Hong Kong

  52. Optimization of randomized neural networks for transfer operator approximation

    May 22, 2026Mohammad Tabish, Stefan KlusNeural ApproximationsNeural Network Optimization

    Maxwell Institute for Mathematical Sciences, University of Edinburgh and Heriot–Watt University, Edinburgh, UK · School of Mathematical & Computer Sciences, Heriot–Watt University, Edinburgh, UK

  53. Non-normal spectral signatures of instability in neural network training dynamics

    May 22, 2026Souvik GhoshNeural Network OptimizationDeep Network

    Department of Physics, National Sun Yat-sen University, Kaohsiung 80424, Taiwan

  54. Hinge Regression Trees and HRT-Boost: Newton-Optimized Oblique Learning for Compact Tabular Models

    May 22, 2026Hongyi Li, Jun Xu, Hong YanDecision TreesTabular Learning

    School of Intelligence Science and Engineering, Harbin Institute of Technology, Shenzhen, 518055, China and with the Shenzhen Key Lab for Advanced Motion Control and Modern Automation Equipments, Shenzhen, 518055, China · Department of Electrical Engineering, City University of Hong Kong, Kowloon, Hong Kong

  55. WeCon: An Efficient Weight-Conditioned Neural Solver for Multi-Objective Combinatorial Optimization Problems

    May 20, 2026Xuan Wu, Jinbiao Chen, Yang Li +7Neural Combinatorial OptimizationNeural Solvers

    Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, College of Computer Science and Technology, Jilin University, Changchun 130012, China · Department of Industrial Systems Engineering and Management, National University of Singapore, Singapore 117576, Singapore · College of Software, Jilin University, Changchun 130012, China +3

  56. StableGrad: Backward Scale Control without Batch Normalization

    May 19, 2026Jose I. Mestre, Alberto Fernández-Hernández, Cristian Pérez-Corral +2Batch NormalizationNeural Network Optimization

    Universitat Politècnica de València · Universitat Jaume I

  57. Scale-Invariant Neural Network Optimization: Norm Geometry and Heavy-Tailed Noise

    May 18, 2026Jiayu Zhang, Tianyi LinNeural Network OptimizationSpectral Norm

    Department of Industrial Engineering and Operations Research Columbia University

  58. An Amortized Efficiency Threshold for Comparing Neural and Heuristic Solvers in Combinatorial Optimization

    May 14, 2026Sohaib AfifiNeural Combinatorial OptimizationCombinatorial Optimization

    Univ. Artois, UR 3926, Laboratoire de Génie Informatique et d’Automatique de l’Artois (LGI2A), F-62400 Béthune, France

  59. NeuroRisk: Physics-Informed Neural Optimization for Risk-Aware Traffic Engineering

    May 13, 2026Yingming Mao, Ximeng Liu, Jingyi Cheng +8Bringing Network CodingNeural Network Optimization

    Xi’an Jiaotong University · Shanghai Innovation Institute · Shanghai Jiao Tong University +2

  60. Implicit Neural Optimal Transport via Fixed-Point Optimization

    May 11, 2026Yesom Park, Eric Gelphman, Stanley Osher +1Neural Network OptimizationPrimal-Dual Methods

    Department of Mathematics, University of California, Los Angeles · Department of Applied Mathematics and Statistics, Colorado School of Mines