M+Adam

Follow this topic Ask about this topic

Recent momentum

emerging

1 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

16 papers

Latest in M+Adam

Open your feed →

AdamX: Cosine similarity meets gradient descent
Sep 10, 2026Francisco Caldas, Ruben Belo, Cláudia SoaresAdam M+Adam
AdaMTP: An Adaptive Training Paradigm for Multi-Token Prediction
Aug 1, 2026Ziqiang Cui, Han Shi, Bowei He +8Next-Token Prediction Large Language Model Backbones
Beyond Retrieval: Analytic Memory for Multimodal Agents
Jul 31, 2026Zhoujin Tian, Hao Zhang, Yao Tian +4Multimodal Memory Multimodal Retrieval
Adaptive Runge-Kutta Step Control Buys Training Loss, Not Generalization: An Honest Compute-Matched Study of RK-Adam Optimizers
Jul 16, 2026Akhilesh GogikarAdam Learning Rates
M+Adam: Low-Precision Training via Additive-Multiplicative Optimization
Jul 12, 2026Xiaoyuan Liang, Sebastian Loeschcke, Mads Toftrup +1Adam M+Adam
AdaMem: Learning What to Remember with Adaptive Memory Policies for Personalized Agents
Jun 19, 2026Xingyu Chen, Rui Wang, Zhaopeng Tu +1Long-Horizon Closed-Book Question Answering
NeuronFabric: A Software Reference Architecture for On-Chip Transformer Training with Local Adam
Jun 15, 2026Evgeny UkladchikovTransformer Architectures Field-Programmable Gate Arrays
AdaMame: A Training Recipe for Adaptive Multilingual Reasoning
Jun 13, 2026Dayeon Ki, Kevin Duh, Marine CarpuatMultilingual Reasoning Large Language Model Reasoning
A Knowledge Theory of Capital:The Value of Natural and Artificial Intelligence, Volume 1
Jun 12, 2026Jeffrey GardinerValue Governed
Beyond a Single Explanation of the Adam--SGD Gap
Jun 12, 2026Chenxiang Zhang, Rustem Islamov, Enea Monzio Compagnoni +3Adam M+Adam
AdMem: Advanced Memory for Task-solving Agents
Jun 5, 2026Runzhe Wang, Huilin Lu, Shengjie Liu +2Agent Memory Long-Term Memory
A Theoretical and Experimental Study of a Novel Adaptive Learning Algorithm
May 28, 2026Sakshi Kumari, Shyam Kumar M, Sushmitha PAdam Convergence Analysis
Understanding Dynamics of Adam in Zero-Sum Games: An ODE Approach
May 19, 2026Yi Feng, Weiming Ou, Xiao WangAdam M+Adam
Adam-SHANG: A Convergent Adam-Type Method for Stochastic Smooth Convex Optimization
May 13, 2026Yaxin Yu, Long Chen, Minfu FengStochastic Smooth Convex Optimization Adam
Incentivizing Neuro-symbolic Language-based Reasoning in VLMs via Reinforcement Learning
Apr 23, 2026Karthic PalaniappanLarge Language Model Reasoning Neuro-Symbolic Framework
Adam symmetry theorem: characterization of the convergence of the stochastic Adam optimizer
Nov 10, 2025Steffen Dereich, Thang Do, Arnulf Jentzen +1Adam Stochastic Smooth Convex Optimization

Sep 10, 2026 · cs.LGJ/K move · Enter open · S save

AdamX: Cosine similarity meets gradient descent

Francisco Caldas, Ruben Belo, Cláudia Soares

We introduce AdamX, a first-order optimizer that incorporates cosine similarity as an adaptive mechanism for controlling update magnitudes. The proposed method is scalable, model-agnostic, and straightforward to integrate into existing training pipelines. We further introduce a variance rectification scheme that promotes smoother optimization during the early stages of training. Overall, we provide empirical evidence that AdamX achieves competitive convergence rates across a range of benchmark datasets and architectures. Performance is evaluated in terms of the number of epochs required to reach predefined performance thresholds under a fixed hyperparameter budget. Code and Experiments available at: https://github.com/FranciscoCaldas/adamX.

Adam M+Adam Zeroth-Order Optimization Gradient Descent Parameter-Efficient Adaptation Response Magnitude Cosine Similarity Benchmark Datasets

Open paper Ask about it PDF

Recent momentum

Neighbouring directions

Latest in M+Adam