Overview

Gradient projection studies how to modify gradients or parameter updates so they satisfy constraints, such as avoiding directions that disrupt earlier tasks or violate physical and safety requirements. Common approaches include PCGrad and A-GEM for conflicting objectives, orthogonal or low-rank subspace projection for continual learning and unlearning, and retraction-based methods for optimization on manifolds. Applications range from multi-task learning and physics-informed neural networks to language-model adaptation, robotics, and diffusion models. A central challenge is that projecting a gradient may not constrain the update produced by momentum or adaptive optimizers, while stronger constraints can protect existing capabilities at the cost of learning new ones.

Latest papers

CardsList

24 papers3 papers in the last four weeks, down 40% on the four weeks before. 0.0% of all new papers.

  1. Near-Optimal Methods for Generalized Lipschitz Convex Optimization

    Nov 15, 2026Dmitry Kovalev, Grigory MalinovskyNonsmooth Convex OptimizationStochastic Optimization Convergence

  2. PRISM-AMC: Replay-Free Continual Learning for Automatic Modulation Classification

    Nov 4, 2026Usman Akram, Matthew Graham, Haris VikaloAutomatic Modulation ClassificationClass-Incremental Learning

  3. Factorized Gradients for Scalable Highly-expressive Parametric Diffeomorphisms

    Oct 26, 2026Amit Aflalo, Eran Treister, Chaim Baskin +1Gradient Projection

  4. UP-MOPD: Update Projection in Multi-Teacher On-Policy Distillation

    Oct 6, 2026Taojie Zhu, Jing Jin, Yuan Xia +8Language Model DistillationGradient Projection

  5. One-Step Curvature Probes Miss the Fitting Operator: Retained Capacity and Terminal Null-Space Correction for Continual Learning

    Oct 2, 2026Abu Sa-Adat Mohamed Moon-Im Al Ahsan, Ibne Farabi Shihab, Md Najmus SwaqeebContinual LearningIncremental Learning

  6. See it, Say it, Sorted: Mechanistic Diagnosis and Parameter-Space Mitigation of Emergent Misalignment in LLMs

    Sep 28, 2026Weiqiao Que, Ruizhe Li, Chengyu Wang +3Emergent Misalignment in Language ModelsLLM Safety Alignment

  7. Retraction-Based Gradient Projection Algorithms on Manifolds

    Sep 25, 2026Conglong Xu, Hao WuGradient ProjectionRiemannian Optimization

  8. Tackling Failure Modes of PINNs and PIKANs Using Conflict-Free Gradients

    Sep 13, 2026Sidharth S. Menon, Irina Tezaur, Ameya D. JagtapPDE SolvingGradient Interference

  9. Gradient-Update Mismatch: Rethinking Conflict-Free Training of Physics-Informed Neural Networks

    Sep 1, 2026Jing Xiao, Xinhai Chen, Qinglin Wang +5Neural Network OptimizationGradient Interference

  10. Denoising as Projection: Constrained Optimization with Gradient-Guided Diffusion

    Aug 30, 2026Runyu Zhang, Jiawei Zhang, Gioele Zardini +2Riemannian OptimizationDiffusion Guidance

  11. Unified Gradient Projection: Language-Balanced Continual Learning for Multilingual Low-Resource ASR

    Jul 13, 2026Ziang Ren, Guodong Lin, Yuchen Ai +2Automatic Speech RecognitionGradient Projection

  12. Rosetta: Composable Native Multimodal Pretraining

    Jul 1, 2026Xiangyue Liu, Zijian Zhang, Miles Yang +3Unified Multimodal ModelsMultimodal Pretraining

  13. Trajectory Optimization for Collision-Aware Redundant Robotic Multi-Axis Additive Manufacturing by Constrained Gradient Projection

    Jun 29, 2026Zhikai Shen, Jiasheng Qu, Chenyu Xu +4ManufacturingConstrained Motion Planning

  14. PURGE: Projected Unlearning via Retain-Guided Erasure

    Jun 2, 2026Vedant Jawandhia, Daksh Ahuja, Ghufran Alam Siddiqui +3Privacy-Preserving MLGradient Projection

  15. G2LoRA: Gradient Orthogonal Low-Rank Adaptation Framework for Graph Continual Learning on Text-Attributed Graphs

    Jun 1, 2026Yuhan Wang, Yibo Ding, Yutong Ye +4Continual LearningLow-Rank Adaptation

  16. Interference-Aware Multi-Task Unlearning

    May 18, 2026Ying-Hua Huang, Rui Fang, Hsi-Wen Chen +1Gradient ProjectionMachine Unlearning

  17. Projected Gradient Unlearning for Text-to-Image Diffusion Models: Defending Against Concept Revival Attacks

    Apr 22, 2026Aljalila Aladawi, Mohammed Talha Alam, Fakhri KarrayDiffusion Model Fine-TuningDiffusion Model Unlearning

  18. SafeAnchor: Preventing Cumulative Safety Erosion in Continual Domain Adaptation of Large Language Models

    Apr 20, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuContinual Learning for LLMsLLM Fine-Tuning

  19. Gradient Descent with Projection Finds Over-Parameterized Neural Networks for Learning Low-Degree Polynomials with Nearly Minimax Optimal Rate

    Mar 22, 2026Yingzhen Yang, Ping LiLow-Degree Polynomial MethodNeural Network Generalization

  20. Task-Agnostic Federated Continual Learning via Replay-Free Gradient Projection

    Sep 25, 2025Seohyeon Cha, Huancheng Chen, Haris VikaloFederated Continual LearningGradient Projection

  21. Projected gradient methods for nonconvex and stochastic smooth optimization: new complexities and auto-conditioned stepsizes

    Dec 18, 2024Guanghui Lan, Tianjiao Li, Yangyang XuGradient ProjectionNonconvex Optimization