Hypergradient Optimization

Latest papers

13 papers2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Newton-Based Methods for Bilevel Optimization Beyond Lower-Level Strong Convexity
Oct 19, 2026Youran Dong, Junfeng Yang, Wei Yao +1Bilevel Optimization Second-Order Optimization
School of Mathematics, Nanjing University, Nanjing, P. R. China. · School of Artificial Intelligence, Southern University of Science and Technology, Shenzhen, P. R. China. · Department of Mathematics and National Center for Applied Mathematics Shenzhen, Southern University of Science and Technology, Shenzhen, P. R. China.
LEAP: Gradient Flow Augmented Lagrangian Trajectory Optimization for Contact Constrained Robots
Oct 14, 2026César E. Ramos Chuquiure, Ryan Tsai, Yutong Zhou +8Hypergradient Optimization Trajectory Optimization
Department of Robotics, University of Michigan, Ann Arbor, MI, USA. · Computer Science and Engineering, University of Michigan, Ann Arbor, MI, USA. · Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA.
Bilevel Optimization of Topology and Hyperparameters (BOTH)
Sep 18, 2026Suryanarayanan Manoj Sanu, Miguel Anibal Bessa, Alejandro Marcos AragónHypergradient Optimization Topology Optimization
Faculty of Mechanical Engineering Delft university of technology Delft The Netherlands · School of engineering Brown University Providence United States of America
Efficient Hypergradient Descent for Inverse Reinforcement Learning
Aug 11, 2026Nikita Sevriukov, Anna Barabanova, Uliana Gagarina +4Reinforcement Learning Gradient Descent
HSE University Moscow, Russia
Generalized Convexity and Smoothness via Conjugate Duality, with Applications to Deep Network Training
Aug 10, 2026Binchuan QiGradient Descent Neural Network Optimization
College of Electronics and Information Engineering Tongji University Shanghai, 200092, China · Sihao Street Zhejiang Yuying College of Vocational Technology Zhejiang, 310018, China
Hypergradient-based Bilevel Reinforcement Learning with Improved Sample Complexity
Jul 30, 2026Naman Saxena, Mudit Gaur, Vaneet AggarwalReinforcement Learning Hypergradient Optimization
Purdue University West Lafayette, IN 47907
Parametric Rates for Nonparametric Conditional-Moment Bilevel Optimization
May 20, 2026Fares El Khoury, Houssam Zenati, Nathan Kallus +2Semiparametric Inference Hypergradient Optimization
Université Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK · Gatsby Computational Neuroscience Unit, University College London · Cornell University +1
Federated Learning with Hypergradient-based Online Update of Aggregation Weights
May 1, 2026Ayano Nakai-Kasai, Tadashi WadayamaFederated Learning Aggregation Hypergradient Optimization
Nagoya Institute of Technology, Gokiso-cho, Showa-ku, Nagoya, Aichi 466-8555, Japan
FlowAdam: Implicit Regularization via Geometry-Aware Soft Momentum Injection
Apr 8, 2026Devender Singh, Tarun SheelHypergradient Optimization Adaptive Gradient Methods
Department of Computer Science Memorial University of Newfoundland St. John’s, NL, Canada · Department of Mathematics and Statistics Memorial University of Newfoundland St. John’s, NL, Canada
Efficient Bilevel Optimization with KFAC-Based Hypergradients
Mar 31, 2026Disen Liao, Felix Dangel, Yaoliang YuKronecker-Factored Approximation Bilevel Optimization
University of Waterloo · Vector Institute
Sample-Efficient Hypergradient Estimation for Decentralized Bi-Level Reinforcement Learning
Mar 16, 2026Mikoto Kudo, Takumi Tanabe, Akifumi Wachi +1Hypergradient Optimization Decentralized MARL
University of Tsukuba · RIKEN AIP · LY Corporation
Understanding the Generalization of Bilevel Programming in Hyperparameter Optimization: A Tale of Bias-Variance Decomposition
Feb 20, 2026Yubo Zhou, Jun Shu, Junmin Liu +1Hypergradient Optimization Hyperparameter Optimization
School of Mathematics and Statistics and Ministry of Education Key Lab of Intelligent Networks and Network Security, Xi’an Jiaotong University, Xi’an, 100190, Shaan’xi Province, P. R. China. · Pazhou Lab (Huangpu), Street, Guangzhou, Guangdong Province, P. R. China.
Bilevel Learning via Strategically Inexact Stochastic Gradient Descent
Nov 10, 2025Mohammad Sadegh Salehi, Subhadip Mukherjee, Lindon Roberts +1Stochastic Bilevel Optimization Hypergradient Optimization
Independent Researcher, Bath, UK · Department of Electronics & Electrical Communication Engineering, Indian Institute of Technology (IIT), Kharagpur, India · School of Mathematics and Statistics, University of Melbourne, Parkville VIC 3010, Australia +1

Oct 19, 2026J/K move · Enter open · S save

Newton-Based Methods for Bilevel Optimization Beyond Lower-Level Strong Convexity

Youran Dong, Junfeng Yang, Wei Yao, Jin Zhang

School of Mathematics, Nanjing University, Nanjing, P. R. China. · School of Artificial Intelligence, Southern University of Science and Technology, Shenzhen, P. R. China. · Department of Mathematics and National Center for Applied Mathematics Shenzhen, Southern University of Science and Technology, Shenzhen, P. R. China.

Bilevel Optimization Second-Order Optimization Hypergradient Optimization

Open paper AskPDF arXiv

Bilevel optimization provides a framework for machine learning tasks such as hyperparameter optimization, data reweighting, and meta-learning. Existing Newton-based bilevel methods typically rely on the classical hypergradient formula, which is generally unavailable when lower-level solutions are nonunique and the lower-level Hessian is singular. We develop two Newton-based methods beyond lower-level strong convexity by exploiting curvature shared by the lower-level and multiplier updates. Newton-based GALET handles potentially singular lower-level curvature under a positive spectral-gap condition, exploiting its range structure to avoid the squared-Hessian multiplier system used in GALET. Newton-based BAMM uses a strongly convex aggregation model when the lower-level objective is convex and the upper-level objective is strongly convex in the lower-level variable. We establish convergence guarantees and improved complexity bounds relative to GALET and sl-BAMM under the corresponding assumptions. We further introduce GNBO, a generalized Newton-based framework that captures the shared-curvature structure of the proposed methods and several existing Newton and quasi-Newton bilevel methods. Experiments on synthetic problems, data hyper-cleaning, and GPT-2 data reweighting evaluate the proposed methods across different bilevel learning tasks.