Convex Loss

Follow this topic Ask about this topic

Recent momentum

-29%

5 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

1 new paper

A weekly snapshot of new work published in Convex Loss.

Period ending 2026-09-07

1 new paper

A weekly snapshot of new work published in Convex Loss.

71 papers

Latest in Convex Loss

Open your feed →

Beyond Quadratic Loss: The Stability Phase Diagram of Adam
Sep 16, 2026Gaoxiang Tang, Huanran Chen, Ziming LiuAdam Convex Loss
Benign Loss Landscapes Can Coexist with Worst-Case Hardness
Sep 14, 2026Zach Furman, Stephan Wäldchen, Yangda Bei +1Convex Loss Two-Layer Neural Networks
Quantile-based Loss Filtering for Outlier-Robust Stochastic Gradient Descent
Sep 14, 2026Jamie Haddock, Anna Ma, Elizaveta RebrovaConvex Loss
Thompson Sampling for Non-Monotone Convex Ridge Bandits: Monotonicity Is Not Needed for Polynomial Regret
Sep 11, 2026Xuan LiThompson Sampling Dynamic Regret Bounds
Post-Training Science for Supervised Fine-Tuning
Sep 1, 2026Charles O'Neill, Mudith Jayasekara, Harry PartridgeSupervised Fine-Tuning Post-Training
Mode Connectivity Beyond Classifiers: Evidence from Generative and Contrastive Models
Aug 31, 2026Chengzheyi Yao, Yongzhao Zhang, Yongding TianContrastive Learning Convex Loss
Estimating Population-Risk Curves Along Nonconvex Gradient Flows from the Training Sample
Aug 31, 2026Mingzhi SongConvex Loss Wasserstein Gradient Flows
Kähler landscapes for complex neural network descents and guarantees including a search and destroy of the Calabi-Yau manifold
Aug 20, 2026Andrew GracykConvex Loss Manifolds
Tunneling the Loss Landscape: Bypassing Memorization with Monte Carlo Parameter Swapping
Aug 3, 2026Lai Shun Chan, Xiaotian Zhang, Yue Shang +2Grokking Parallel Liquid-Inspired Temporal Relaxation
The Role of Causality in Algorithmic Recourse
Jul 30, 2026Srikanth Avasarala, Varun Gupta, Shahin Jabbari +2Causal Structure and Reasoning Outcome
On the robustness of noisy solutions in non-convex neural networks
Jul 29, 2026Enrico M. Malatesta, Alessandra Passalacqua, Riccardo ZecchinaConvex Loss Parallel Liquid-Inspired Temporal Relaxation
A Defense of the Quadratic Model
Jul 23, 2026Alexandru Meterez, Pranav Ajit Nair, Depen Morwani +3Hessian Eigenvalue Neural Network Optimization
Loss Landscape Topology Reveals Why Simple Baselines are Competitive at 3D Point Cloud Segmentation Under Class Imbalance
Jul 23, 2026Antonis Savva, Christos Kyrkou, Theocharis TheocharidesPoint Clouds Class Imbalance
Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay
Jul 23, 2026Xiaolong Li, Zhangchen Zhou, Zhi-Qin John XuWeight Decay Convex Loss
Decentralized Online Riemannian Optimization for Strongly Geodesically Convex Functions
Jul 22, 2026Zhanyuan Cai, Emre Sahinoglu, Shahin ShahrampourRiemannian Optimization Convex Optimization
Effects of width-dependent model hyperparameters and $\ell_2$ -regularization on the loss landscape of two-layer ReLU networksEffects of width-dependent model hyperparameters and \ell_2-regularization on the loss landscape of two-layer ReLU networks
Jul 18, 2026Haruka Eshima, Makoto YamadaRectified Linear Unit Networks Weight Decay
How the Hessian-Spectrum of Neural Networks Depends on Data
Jul 15, 2026Jasraj Singh, Enea Monzio Compagnoni, Antonio OrvietoHessian Eigenvalue Hessian Matrix
Overcoming Fourier Locking in Quantum Data Re-uploading Classifiers via Spectral Homotopy
Jul 13, 2026Spencer TopelQuantum Circuits Quantum
Lower Bound on the Cumulative Constrained Violation for the OGD+Projection algorithm for Constrained Online Convex Optimization (COCO)
Jul 12, 2026Haricharan Balasundaram, Karthick Krishna Mahendran, Rahul VazeConvex Optimization O(T^Β)$ Simultaneous Regret
Avoiding unsafe sets when training with Langevin Dynamics
Jul 8, 2026Adam M. ObermanConvex Loss Langevin Dynamics
Constrained Online Convex Optimization without Slater's Condition
Jun 30, 2026Kihyun Yu, Junehee Lee, Dabeen LeeConvex Optimization Stochastic Optimization
Incremental Learning in Mirror Flows
Jun 22, 2026Raphaël Berthier, Loucas Pillaud-VivienMirror Descent Convex Loss
Generalized nonparametric regression in reproducing kernel Hilbert spaces: Consistency and rates of convergence
Jun 22, 2026Ioannis KalogridisKernel Hilbert Spaces Exponential Family
Bypassing Minimization Bias: A Shift-Invariant Variance Estimator for Off-Equilibrium Local Learning Coefficients
Jun 21, 2026Yingjia CaiConvex Loss Singular Learning Theory
Adversarial Bandit Optimization with Globally Bounded Perturbations to Convex Losses
Jun 18, 2026Zhuoyu Cheng, Kohei Hatano, Eiji TakimotoConvex Optimization Convex Loss
Functional Gradient Descent with Adaptive Representations
Jun 15, 2026Daniel Csillag, Rodrigo Schuller, Pedro Dall'Antonia +3Gradient Descent Convex Loss
Loss Landscape Poisoning: Targeted Extraction of Unseen Training Data from LLMs
Jun 15, 2026Md Abdullah Al Mamun, Ngoc Phu Doan, Pedram Zaree +2Poisoning Attacks Differential Privacy
Loss Landscape Diagnosis for Gradient-Based Gray-Scott System Inversion: Disentangling the Roles of PINN Components
Jun 9, 2026Yan YangPhysics-Informed Neural Network Framework Convex Loss
Lost in the Non-convex Loss Landscape: How to Fine-tune the Large Time Series Model?
Jun 7, 2026Xu Zhang, Peang Wang, Wei WangFinetuning Convex Loss
Noise-Adaptive High-Probability Regret Bounds for Online Convex Optimization
Jun 6, 2026Wentao Zhang, Yutong Zhang, Wentao MoConvex Optimization Dynamic Regret Bounds
Robust and sparse support vector machine via hybrid truncated loss for supervised classification
Jun 4, 2026Yuliang Yang, Chen Chen, Yuxiang Liu +1Loss Function Multi-View
Flatness and Generalization: Learning Multi-Index Models with Homogeneous Neural Networks
Jun 3, 2026Harsh Vardhan, Hossein Taheri, Arya MazumdarTwo-Layer Neural Networks Convex Loss
Quadratic integrate-and-fire neurons exhibit less fragmented loss landscapes and outperform leaky integrate-and-fire neurons in spike-based gradient descent
Jun 2, 2026Carlo Wenig, Raoul-Martin Memmesheimer, Christian KlosSpiking Neural Networks Neuromorphic Hardware
Exploiting weight-space symmetries for approximating curvature
May 30, 2026Artem Artemev, Rui Xia, Benjamin M. Boyd +4Curvature Prediction Quality Hessian Matrix
Singularity-aware Optimization via Randomized Geometric Probing: Towards Stable Non-smooth Optimization
May 28, 2026Ruoran Xu, Borong She, Xiaobo Jin +1Adam Neural Network Optimization
Convex Basins in Single-Index Model Loss Landscapes: Applications to Robust Recovery under Strong Adversarial Corruption
May 28, 2026Santanu Das, Sagnik Chatterjee, Jatin BatraConvex Loss Heavy-Tailed Noise
Unveiling Multi-regime Patterns in SciML: Distinct Failure Modes and Regime-specific Optimization
May 27, 2026Yuxin Wang, Yuanzhe Hu, Xiaokun Zhong +7Low-Data Regimes Machine Learning
How the Optimizer Shapes Learned Solutions in Equivariant Neural Networks
May 26, 2026Teodor-Mihai Stupariu, Andrei ManolacheConvex Loss Muon
Model Merging on Loss Landscape: A Geometry Perspective
May 26, 2026Juanwu Lu, Anand Bhaskar, Brian Axelrod +2Continual Model Merging Model Merging
Online Learning on Hidden-Convex Losses via Algorithmic Equivalence: Optimal Regret, Geometric Barrier, and Bandit Feedback
May 25, 2026Anas Barakat, Andreas Kontogiannis, Vasilis Pollatos +2O(T^Β)$ Simultaneous Regret Convex Loss
Spectral Asymptotics of Neural Network Loss Landscapes: An Exact Decomposition of the Curvature Exponent
May 22, 2026Anherutowa CalvoHessian Eigenvalue Hessian Matrix
A lift for input-convex neural network training
May 22, 2026Ali Siahkoohi, Anirudh ThatipelliConvex Loss Neural Network Optimization
Why SGD is not Brownian Motion: A New Perspective on Stochastic Dynamics
May 21, 2026Igor Ignashin, Anna Radovskaya, Andrew Semenov +7Stochastic Gradient Descent Stochastic Differential Equations
When Stronger Triggers Backfire: A High-Dimensional Theory of Backdoor Attacks
May 21, 2026Donald Flynn, Hadas Yaron Goldhirsh, Jonathan P. Keating +1Backdoor Attacks Convex Loss
A Geometric Approach to Constrained Online Learning
May 20, 2026Dhruv Sarkar, Abhishek SinhaConvex Optimization O(T^Β)$ Simultaneous Regret
LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series
May 19, 2026Alexis Roger, Prateek Humane, Zhenghan Tai +4Generative Pretrained Transformers Time Series
Landscape-Awareness for Geometric View Diffusion Model
May 19, 2026Yan-Ting Chen, Hao-Wei Chen, Tsu-Ching Hsiao +1Multi-View Diffusion Model Viewpoint
Bounded-Rationality, Hedging, and Generalization
May 14, 2026Pedro A. OrtegaGeneralization Bounds Convex Loss
Angel or Demon: Investigating the Plasticity Interventions' Impact on Backdoor Threats in Deep Reinforcement Learning
May 14, 2026Oubo Ma, Ruixiao Lin, Yang Dai +4Backdoor Attacks Safe Reinforcement Learning
Beyond Perplexity: A Geometric and Spectral Study of Low-Rank Pre-Training
May 13, 2026Namrata Shivagunde, Vijeta Deshpande, Sherin Muckatira +1Low-Rank Factorization Language Model Perplexity
DP-KFC: Data-Free Preconditioning for Privacy-Preserving Deep Learning
May 13, 2026Marc Molina Van den Bosch, Riccardo Taiello, Albert Sund Aillet +3Differential Privacy Dp-Sgd
Parameterized Complexity of Stationarity Testing for Piecewise-Affine Functions and Shallow CNN Losses
May 11, 2026Yuhan YeConvex Loss Convex Optimization
Fix the Loss, Not the Radius: Rethinking the Adversarial Perturbation of Sharpness-Aware Minimization
May 11, 2026Jinping Wang, Qinhan Liu, Zhiwu Xie +1Sharpness-Aware Minimization Convex Loss
Finite Volume-Informed Neural Network Framework for 2D Shallow Water Equations: Rugged Loss Landscapes and the Importance of Data Guidance
May 9, 2026Xiaofeng LiuShallow Water Equations Surrogate Modeling
Susceptibilities and Patterning: A Primer on Linear Response in Bayesian Learning
May 8, 2026Chris Elliott, Daniel MurfetBayesian Learning Bayesian Inference Approaches
Flatness and Gradient Alignment Are Both Necessary: Spectral-Aware Gradient-Aligned Exploration for Multi-Distribution Learning
May 8, 2026Aristotelis Ballas, Christos DiouDistributional Learning Gradient
Distributional simplicity bias and effective convexity in Energy Based Models
May 8, 2026Aurélien Decelle, Alfonso de Jesús Navas Gómez, Beatriz SeoaneEnergy Based Models Convex Loss
Weight-Decay Turns Transformer Loss Landscapes Villani: Functional-Analytic Foundations for Optimization and Generalization
May 7, 2026Abhijit Das, Sayantan DuttaWeight Decay Convex Loss
Gradient Regularized Newton Boosting Trees with Global Convergence
May 1, 2026Nikita Zozoulenko, Daniel Falkowski, Thomas Cass +1Gradient Boosted Tree Newton
The Role of Symmetry in Optimizing Overparameterized Networks
Apr 28, 2026Kusha Sareen, Mohammad Pedramfar, Sékou-Oumar Kaba +2Over-Parameterized Neural Networks Neural Network Optimization

Sep 16, 2026 · cs.LGJ/K move · Enter open · S save

Beyond Quadratic Loss: The Stability Phase Diagram of Adam

Gaoxiang Tang, Huanran Chen, Ziming Liu

Loss spikes are recurrent instabilities in neural-network training and can arise from multiple mechanisms. For Adam in particular, macroscopic loss spikes have been linked to optimizer dynamics, yet how its two momentum timescales govern them remains unclear. We investigate this dependence by mapping training dynamics across the

(β_1,β_2)

plane. Across a range of model--task settings, an approximately linear boundary,

1-β_2=C(1-β_1)

, separates spiky from non-spiky dynamics, whereas a one-dimensional quadratic loss produces approximately cubic slope. A one-dimensional superquadratic loss

L(x)\propto|x|^n

recovers the near-linear scaling and links the boundary coefficient to the effective loss exponent

n

. We further show that confident cross-entropy losses develop a core--wall landscape comprising a narrow quadratic core followed by a steep wall, which produces effective superquadratic behavior at the scale of an optimizer update. Together, these results connect Adam loss spikes to both the mismatch between momentum timescales and finite-scale superquadratic loss geometry beyond the Hessian.

Adam Convex Loss Plasticity

Open paper Ask about it PDF