Adam

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

6 new papers

A weekly snapshot of new work published in Adam.

Period ending 2026-09-14

4 new papers

A weekly snapshot of new work published in Adam.

Period ending 2026-09-07

4 new papers

A weekly snapshot of new work published in Adam.

Inside this field

Focused directions

293 papers

Latest in Adam

Browse all topics →
CardsList

  1. AdaMeZO: Adam-style Zeroth-Order Optimizer for LLM Fine-tuning Without Maintaining the Moments

    May 1, 2026Zhijie Cai, Haolong Chen, Guangxu ZhuAdamZeroth-Order Optimization

  2. NLPOpt-Net: A Learning Method for Nonlinear Optimization with Feasibility Guarantees

    Apr 30, 2026Bimol Nath Roy, Rahul Golder, MM Faruque HasanQuadratic ProgramsNeural Network Optimization

  3. On the Expressive Power of GNNs to Solve Linear SDPs

    Apr 30, 2026Chendi Qian, Christopher MorrisSemidefinite ProgrammingGraph Neural Networks

  4. NeuroPlastic: A Plasticity-Modulated Optimizer for Biologically Inspired Learning Dynamics

    Apr 29, 2026Douglas Jiang, Yuechen Wang, Jiayi Wang +3Neural Network OptimizationPlasticity

  5. From Cursed to Competitive: Closing the ZO-FO Gap via Input-to-State Stability

    Apr 28, 2026Amir Ali Farzin, Philipp Braun, Iman ShamesZeroth-Order OptimizationLyapunov Function

  6. The Role of Symmetry in Optimizing Overparameterized Networks

    Apr 28, 2026Kusha Sareen, Mohammad Pedramfar, Sékou-Oumar Kaba +2Over-Parameterized Neural NetworksLoss Landscape

  7. Gradient-Direction Sensitivity Reveals Linear-Centroid Coupling Hidden by Optimizer Trajectories

    Apr 28, 2026Yongzhong XuAdamSingular Value Decomposition

  8. A Layer Separation Optimization Framework for Cross-Entropy Training in Deep Learning

    Apr 25, 2026Yaru Liu, Michael K. Ng, Yiqi GuCross-Entropy LossesNeural Network Optimization

  9. Shape of Memory: a Geometric Analysis of Machine Unlearning in Second-Order Optimizers

    Apr 24, 2026Kennon StewartMachine UnlearningZeroth-Order Optimization

  10. On-Device Vision Training, Deployment, and Inference on a Thumb-Sized Microcontroller

    Apr 24, 2026Jeremy EllisCortexMobilenetv2

  11. Hidden Failure Modes of Gradient Modification under Adam in Continual Learning, and Adaptive Decoupled Moment Routing as a Repair

    Apr 24, 2026Yuelin Hu, Zhenbo Yu, Zhengxue Cheng +2AdamEfficient Continual Learning

  12. Incentivizing Neuro-symbolic Language-based Reasoning in VLMs via Reinforcement Learning

    Apr 23, 2026Karthic PalaniappanLarge Language Model ReasoningNeuro-Symbolic Framework

  13. Soft Anisotropic Diagrams for Differentiable Image Representation

    Apr 23, 2026Laki Iinbor, Zhiyang Dou, Wojciech MatusikDifferentiable RenderingAnisotropy

  14. DiffNR: Diffusion-Enhanced Neural Representation Optimization for Sparse-View 3D Tomographic Reconstruction

    Apr 23, 2026Shiyan Su, Ruyi Zha, Danli Shi +23D Latent Diffusion ModelDiffusion Refinement

  15. Beyond ZOH: Advanced Discretization Strategies for Vision Mamba

    Apr 22, 2026Fady Ibrahim, Guangjun Liu, Guanghui WangVision Foundation ModelsState Space Models

  16. Generalization at the Edge of Stability

    Apr 21, 2026Mario Tuci, Caner Korkmaz, Umut Şimşekli +1Generalization BoundsNeural Network Optimization

  17. What Makes an LLM a Good Optimizer? A Trajectory Analysis of LLM-Guided Evolutionary Search

    Apr 21, 2026Xinhao Zhang, Xi Chen, François Portet +1Evolutionary SearchOptimizer Design

  18. Neural Network Optimization Reimagined: Decoupled Techniques for Scratch and Fine-Tuning

    Apr 21, 2026Xin Ning, Qiankun Li, Xiaolong Huang +5Neural Network OptimizationDeep Learning

  19. Universally Empowering Zeroth-Order Optimization via Adaptive Layer-wise Sampling

    Apr 20, 2026Fei Wang, Li Shen, Liang Ding +3Zeroth-Order OptimizationUniversality

  20. Neural Adjoint Method for Meta-optics: Accelerating Volumetric Inverse Design via Fourier Neural Operators

    Apr 19, 2026Chanik Kang, Hyewon Suk, Haejun ChungGenerative Inverse DesignFourier Neural Operators

  21. A unified convergence theory for adaptive first-order methods in the nonconvex case, including AdaNorm, full and diagonal AdaGrad, Shampoo and Muo

    Apr 19, 2026S. Gratton, Ph. L. TointNonconvexZeroth-Order Optimization

  22. Benchmarking Optimizers for MLPs in Tabular Deep Learning

    Apr 16, 2026Yury Gorishniy, Ivan Rubachev, Dmitrii Feoktistov +1Tabular LearningOptimizer Design

  23. StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models

    Apr 16, 2026Dingzhi Yu, Rui Pan, Yuxing Liu +1Stochastic Gradient DescentOptimizer Design

  24. Prism: Symbolic Superoptimization of Tensor Programs

    Apr 16, 2026Mengdi Wu, Xiaoyu Jiang, Oded Padon +1Tensor ProgramsCode Optimization

  25. Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks

    Apr 16, 2026Kang An, Chenhao Si, Shiqian Ma +1Conventional Physics-Informed Neural NetworksCurvature-Aware Quadratic Banks

  26. Zeroth-Order Optimization at the Edge of Stability

    Apr 16, 2026Minhak Song, Liang Zhang, Bingcong Li +3Zeroth-Order OptimizationCondition Number

  27. Adam's Law: Textual Frequency Law on Large Language Models

    Apr 2, 2026Hongyuan Adam Lu, Z. L., Victor Wei +5High Bigram Repetition FrequencyCognitive Science

  28. Stochastic Dimension Zeroth-Order Estimator: Stable and Memory-Efficient Training of PINNs

    Mar 25, 2026Zhangyong Liang, Huanhuan GaoZeroth-Order OptimizationPartial Differential Equations

  29. Uniform a priori bounds and error analysis for the Adam stochastic gradient descent optimization method

    Mar 19, 2026Steffen Dereich, Thang Do, Arnulf JentzenAdamStochastic Smooth Convex Optimization

  30. To Use or not to Use Muon: How Simplicity Bias in Optimizers Matters

    Feb 28, 2026Sara Dragutinović, Yedi Zhang, Rajesh RanganathMuon OptimizerMuon

  31. DASH: Faster Shampoo via Batched Block Preconditioning and Efficient Inverse-Root Solvers

    Feb 2, 2026Ionut-Vlad Modoranu, Philip Zmushko, Erik Schultheis +2Spectral PreconditioningNewton

  32. Adaptive Momentum and Nonlinear Damping for Neural Network Training

    Jan 30, 2026Aikaterini Karoni, Rajit Rajpal, Benedict Leimkuhler +1Adam

  33. Why β1=β2β_1 = β_2 Is Dynamically Special in Adam

    Jan 29, 2026Alberto Fernández-Hernández, Cristian Pérez-Corral, Jose I. Mestre +2AdamResponse Magnitude

  34. Learning to Optimize by Differentiable Programming

    Jan 23, 2026Liping Tao, Xindi Tong, Chee Wei TanDifferentiable ProgrammingPrimal-Dual Methods

  35. Variance-Adaptive Muon: Pre-Orthogonalization Variance Modulation for Efficient Language Model Pretraining

    Jan 21, 2026Jingru Li, Yibo Fan, Huan LiMuon OptimizerMuon

  36. Enabling Population-Based Architectures for Neural Combinatorial Optimization

    Jan 13, 2026Andoni Irazusta Garmendia, Josu Ceberio, Alexander MendiburuMetaheuristicsNeural Network Optimization

  37. A General Neural Backbone for Mixed-Integer Linear Optimization via Dual Attention

    Jan 8, 2026Peixin Huang, Yaoxin Wu, Yining Ma +3Mixed-Integer ProgrammingCombinatorial Optimization

  38. Basic Inequalities for First-Order Optimization with Applications to Statistical Risk Analysis

    Dec 31, 2025Seunghoon Paik, Kangjie Zhou, Matus Telgarsky +1InequalitiesZeroth-Order Optimization

  39. Mean-Field Model for Two-Layer Neural Networks Trained with Consensus-Based Optimization

    Nov 26, 2025William De Deyn, Michael Herty, Giovanni SamaeyNeural Network OptimizationTwo-Layer Neural Networks

  40. On the Gradient Complexity of Private Optimization with Private Oracles

    Nov 17, 2025Michael Menart, Aleksandar NikolovDifferential PrivacyStochastic Gradient Descent

  41. Adam symmetry theorem: characterization of the convergence of the stochastic Adam optimizer

    Nov 10, 2025Steffen Dereich, Thang Do, Arnulf Jentzen +1AdamStochastic Smooth Convex Optimization

  42. Accelerated Frank-Wolfe Algorithms: Complementarity Conditions and Sparsity

    Nov 4, 2025Dan GarberConvex OptimizationSpectral Norm

  43. Understanding and Improving Shampoo and SOAP via Kullback-Leibler Minimization

    Sep 3, 2025Wu Lin, Scott C. Lowe, Felix Dangel +3Neural Network OptimizationKullback-Leibler Divergence

  44. L-SR1: Learned Symmetric-Rank-One Preconditioning

    Aug 17, 2025Gal Lifshitz, Shahar Zuler, Ori Fouks +1Zeroth-Order OptimizationDeep Learning

  45. On the Effectiveness of the z-Transform Method in Quadratic Optimization

    Jul 4, 2025Francis BachConvergence AnalysisZeroth-Order Optimization

  46. Favorability of Loss Landscape with Weight Decay Requires Both Large Overparametrization and Initialization

    May 28, 2025Etienne Boursier, Matthew Bowditch, Matthias Englert +1Loss LandscapeOver-Parameterized Neural Networks

  47. Minimisation of Quasar-Convex Functions Using Random Zeroth-Order Oracles

    May 4, 2025Amir Ali Farzin, Yuen-Man Pun, Philipp Braun +1Convex SetsZeroth-Order Optimization

  48. Mirror Descent Linearized Augmented Lagrangian Methods for Nonconvex Constrained Stochastic Zeroth-Order Optimization

    Apr 13, 2025Qiankun Shi, Han Yuan, Xiao Wang +1Stochastic Smooth Convex OptimizationMirror Descent

  49. On the Relationship Between CoCoA and ADMM for Distributed Empirical Risk Minimization

    Feb 1, 2025Runxiong Wu, Andi WangEmpirical Risk MinimizationPrimal-Dual Methods

  50. Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses

    Jun 20, 2024Steffen Dereich, Arnulf Jentzen, Adrian RiekertStochastic Gradient DescentAdam

  51. MEGO: Learning Mixture-of-Experts for General-Purpose Binary Optimization

    May 29, 2024Shengcai Liu, Zhiyuan Wang, Yew-Soon Ong +2Black-Box OptimizationCombinatorial Optimization

  52. Generalizing Adam to Manifolds for Efficiently Training Transformers

    May 26, 2023Benedikt BrantnerAdamManifolds

  53. Adapt or Forget: Provable Tradeoffs Between Adam and SGD in Nonstationary Optimization

    Date pendingSharan Sahu, Abir Sarkar, Cameron J. Hogan +1AdamStationarity