Gauss-Newton Optimization

Ask about this topic

Overview

Gauss–Newton methods optimize objectives expressed through residuals by replacing the full Hessian with a Jacobian-based curvature matrix, often adding Levenberg–Marquardt damping to stabilize steps. Work spans nonlinear parameter fitting, neural-network training—especially physics-informed neural networks—and inverse problems, using matrix-free solvers, sketching, preconditioning, and structure-aware reductions to make updates affordable. These methods are useful when first-order optimizers struggle with ill-conditioning and are being adapted to PDE solvers, diffusion-based reconstruction, motion fitting, and hyperparameter search. A central tension is that approximate curvature can improve convergence but may be expensive or misleading outside suitable regimes, so scaling, damping, and guarantees about when it tracks true Hessian behavior remain important concerns.

Latest papers

CardsList

14 papers3 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

  1. Trust Your Targets: Is Target Propagation Block-Diagonal Gauss-Newton?

    Oct 19, 2026Giuseppe Castiglione, Christopher Buckley, Ivor SimpsonGauss-Newton OptimizationTrust-Region Optimization

    Department of Informatics University of Sussex

  2. Gauss-Newton Accuracy and Indefinite Hessians: Uniform Coexistence in Low-Cost Sets

    Oct 7, 2026Kihun Rhee, Hanjoon ByunGauss-Newton OptimizationSecond-Order Optimization

  3. Tensor-Train Compressed Separable PINNs: A Curvature-Aware Optimization Framework for Parametric PDEs in High Dimensions

    Sep 28, 2026Denis Korolev, Martin EigelNeural PDE SolversPhysics-Informed ML

    Weierstrass Institute for Applied Analysis and Stochastics (WIAS), Anton-Wilhelm-Amo-Straße 39, 10117 Berlin, Germany

  4. Beyond PINNs: A Unified Gauss--Newton and Petrov--Galerkin Framework for Neural and Hybrid PDE Solvers

    Sep 17, 2026Nilo Schwencke, Roland MaierGauss-Newton OptimizationNeural PDE Solvers

  5. DirtyMoCap: Robust Motion Capture from Unconstrained Markers

    Sep 17, 2026Long Wang, Shuting Zhao, Shen Yan +5Gauss-Newton Optimization3D Human Pose Estimation

    Zhejiang University, Hangzhou, China · Westlake University, Hangzhou, China · Fudan University, Shanghai, China +3

  6. Second-Order Muon Done Right: A Principled Marriage of Spectral Geometry and Curvature

    Aug 10, 2026Tong CheGauss-Newton OptimizationInformation Geometry

    NVIDIA Research

  7. Higher-Order Geometric Updates for Levenberg-Marquardt Method via Riemann Normal Coordinates

    Jul 8, 2026Jianing Liu, Dong H. ZhangGauss-Newton OptimizationRiemannian Optimization

    State Key Laboratory of Chemical Reaction Dynamics and Department of Chemical Physics, University of Science and Technology of China, Hefei, 230026, China · State Key Laboratory of Chemical Reaction Dynamics, Dalian Institute of Chemical Physics, Chinese Academy of Sciences, Dalian, 116023, China. · University of Chinese Academy of Sciences, Beijing, 100049, China +1

  8. An Optimisation Framework for the Well-Conditioned Training of Physics-Informed Neural Networks

    Jul 2, 2026Joseph Webb, Sadok Jerad, Coralia CartisGauss-Newton OptimizationPDE Solving

    Mathematical Institute, University of Oxford, Oxford, United Kingdom

  9. Geometry-Correct Diffusion Posterior Sampling with Denoiser-Pullback Curvature Guidance and Manifold-Aligned Damping

    May 27, 2026Seunghyeok Shin, Minwoo Kim, Dabin Kim +1Gauss-Newton OptimizationBayesian Inverse Problems

    Department of Electrical and Computer Engineering, Inha University, Incheon, 22212, South Korea.

  10. Coupling-Robust Accuracy in Multiphysics Physics Informed Neural Networks via Kronecker-Preconditioned Optimization

    May 22, 2026Youngjae Park, Jaemin Kim, Junghwa HongGauss-Newton OptimizationNeural Network Optimization

    Dept. of Control and Instrumentation Engineering, Korea University, Sejong, South Korea · BK21 FOUR Smart Mobility Education and Research Team, Korea University, Sejong, South Korea

  11. SCAMP: Sparse-anchor Control is One Small Projection

    May 14, 2026Pengcheng Fang, Tengjiao Sun, Xiaoyu Zhan +4Gauss-Newton OptimizationText-to-Motion Generation

    University of Southampton · Mogo AI Ltd. · Nanjing University

  12. Error whitening: Why Gauss-Newton outperforms Newton

    May 11, 2026Maricela Best McKay, Nathan P. Lawrence, Brian Wetton +1Gauss-Newton OptimizationDeep Learning Optimization

    University of British Columbia · University of California, Berkeley

  13. Fast Gauss-Newton for Multiclass Cross-Entropy

    May 7, 2026Mikalai Korbit, Mario ZanonGauss-Newton OptimizationNeural Network Optimization

    IMT School for Advanced Studies Lucca

  14. Multi-Objective Hyperparameter Search via Damped Gauss--Newton Optimization

    Jan 7, 2024Qinwu Xu, Yifan JiangGauss-Newton OptimizationSecond-Order Optimization