Gradient Projection

Ask about this topic

Overview

Gradient projection studies how to modify gradients or parameter updates so they satisfy constraints, such as avoiding directions that disrupt earlier tasks or violate physical and safety requirements. Common approaches include PCGrad and A-GEM for conflicting objectives, orthogonal or low-rank subspace projection for continual learning and unlearning, and retraction-based methods for optimization on manifolds. Applications range from multi-task learning and physics-informed neural networks to language-model adaptation, robotics, and diffusion models. A central challenge is that projecting a gradient may not constrain the update produced by momentum or adaptive optimizers, while stronger constraints can protect existing capabilities at the cost of learning new ones.

Latest papers

CardsList

21 papers4 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

  1. Factorized Gradients for Scalable Highly-expressive Parametric Diffeomorphisms

    Oct 26, 2026Amit Aflalo, Eran Treister, Chaim Baskin +1Gradient Projection

    The Stein Faculty of Computer and Information Science, Ben-Gurion University of the Negev, Israel · School of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Israel

  2. UP-MOPD: Update Projection in Multi-Teacher On-Policy Distillation

    Oct 6, 2026Taojie Zhu, Jing Jin, Yuan Xia +8Language Model DistillationGradient Projection

    Tsinghua University · Ant Group · Zhejiang University

  3. One-Step Curvature Probes Miss the Fitting Operator: Retained Capacity and Terminal Null-Space Correction for Continual Learning

    Oct 2, 2026Abu Sa-Adat Mohamed Moon-Im Al Ahsan, Ibne Farabi Shihab, Md Najmus SwaqeebContinual LearningIncremental Learning

    Department of Computer Science and Engineering, BRAC University, Bangladesh · Department of Computer Science, Iowa State University, USA · BRAC University, Bangladesh

  4. See it, Say it, Sorted: Mechanistic Diagnosis and Parameter-Space Mitigation of Emergent Misalignment in LLMs

    Sep 28, 2026Weiqiao Que, Ruizhe Li, Chengyu Wang +3Emergent Misalignment in Language ModelsLLM Safety Alignment

    School of Computer Science and Technology, East China Normal University, China · School of Computer Science, University of Birmingham, UK · Alibaba Group, China +2

  5. PMOPD: Task Ordering, Cycling, and Parameter-Update Subspace Protection in Multi-Teacher On-Policy Distillation

    Sep 28, 2026Youzhi Liu, Ruobing Zheng, Boyuan Tong +5Gradient InterferenceMulti-Task Learning

    Ant Group

  6. Retraction-Based Gradient Projection Algorithms on Manifolds

    Sep 25, 2026Conglong Xu, Hao WuGradient ProjectionRiemannian Optimization

    Department of Mathematics, The George Washington University, Phillips Hall, Room 739, 801 22nd Street, N.W., Washington, DC 20052, USA.

  7. From Trainability Diagnostics to Optimization Claims: Boundaries and Controls in Variational Quantum Optimization

    Sep 18, 2026Pilsung KangVariational Quantum AlgorithmsGradient Projection

  8. Tackling Failure Modes of PINNs and PIKANs Using Conflict-Free Gradients

    Sep 13, 2026Sidharth S. Menon, Irina Tezaur, Ameya D. JagtapPDE SolvingGradient Interference

    Aerospace Engineering Department, Worcester Polytechnic Institute, Worcester, MA 01609, USA. · Sandia National Laboratories, 7011 East Ave, Livermore, CA 94550, USA.

  9. Gradient-Update Mismatch: Rethinking Conflict-Free Training of Physics-Informed Neural Networks

    Sep 1, 2026Jing Xiao, Xinhai Chen, Qinglin Wang +5Neural Network OptimizationGradient Interference

    Laboratory of Digitizing Software for Frontier Equipment, National University of Defense Technology, Changsha 410073, China · National Key Laboratory of Parallel and Distributed Computing, National University of Defense Technology, Changsha 410073, China · School of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China

  10. Denoising as Projection: Constrained Optimization with Gradient-Guided Diffusion

    Aug 30, 2026Runyu Zhang, Jiawei Zhang, Gioele Zardini +2Riemannian OptimizationDiffusion Guidance

    MIT · University of Wisconsin–Madison

  11. Unified Gradient Projection: Language-Balanced Continual Learning for Multilingual Low-Resource ASR

    Jul 13, 2026Ziang Ren, Guodong Lin, Yuchen Ai +2Automatic Speech RecognitionGradient Projection

    Department of Electronic Engineering, Tsinghua University, China

  12. Rosetta: Composable Native Multimodal Pretraining

    Jul 1, 2026Xiangyue Liu, Zijian Zhang, Miles Yang +3Unified Multimodal ModelsMultimodal Pretraining

    HKUST · Tencent Hunyuan

  13. Trajectory Optimization for Collision-Aware Redundant Robotic Multi-Axis Additive Manufacturing by Constrained Gradient Projection

    Jun 29, 2026Zhikai Shen, Jiasheng Qu, Chenyu Xu +4ManufacturingConstrained Motion Planning

    Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong, China · Centre for Perceptual and Interactive Intelligence (CPII), Hong Kong, China · Department of Robotics at the School of Mechanical Engineering, Southeast University, China

  14. PURGE: Projected Unlearning via Retain-Guided Erasure

    Jun 2, 2026Vedant Jawandhia, Daksh Ahuja, Ghufran Alam Siddiqui +3Privacy-Preserving MLGradient Projection

    1BITS Pilani, Pilani Campus, India

  15. G2LoRA: Gradient Orthogonal Low-Rank Adaptation Framework for Graph Continual Learning on Text-Attributed Graphs

    Jun 1, 2026Yuhan Wang, Yibo Ding, Yutong Ye +4Continual LearningLow-Rank Adaptation

    School of Computer Science and Engineering, Beihang University Beijing, China · Department of Statistics, Columbia University New York, NY, USA · College of Computer Science, Beijing University of Technology Beijing, China

  16. Interference-Aware Multi-Task Unlearning

    May 18, 2026Ying-Hua Huang, Rui Fang, Hsi-Wen Chen +1Gradient ProjectionMachine Unlearning

    National Taiwan University

  17. Projected Gradient Unlearning for Text-to-Image Diffusion Models: Defending Against Concept Revival Attacks

    Apr 22, 2026Aljalila Aladawi, Mohammed Talha Alam, Fakhri KarrayDiffusion Model Fine-TuningDiffusion Model Unlearning

    Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE. · University of Waterloo, Ontario, Canada.

  18. SafeAnchor: Preventing Cumulative Safety Erosion in Continual Domain Adaptation of Large Language Models

    Apr 20, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuContinual Learning for LLMsLLM Fine-Tuning

    The University of Hong Kong, Hong Kong, China · Brain Investing Limited, Hong Kong, China

  19. AEGIS: Anchor-Enforced Gradient Isolation for Knowledge-Preserving Vision-Language-Action Fine-Tuning

    Apr 17, 2026Guransh SinghVLM AdaptationRobotic Manipulation

    Independent Researcher

  20. Gradient Descent with Projection Finds Over-Parameterized Neural Networks for Learning Low-Degree Polynomials with Nearly Minimax Optimal Rate

    Mar 22, 2026Yingzhen Yang, Ping LiLow-Degree Polynomial MethodNeural Network Generalization

    School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ 85281, USA · VecML Inc., Bellevue, WA 98004, USA

  21. Task-Agnostic Federated Continual Learning via Replay-Free Gradient Projection

    Sep 25, 2025Seohyeon Cha, Huancheng Chen, Haris VikaloFederated Continual LearningGradient Projection

    Department of Electrical and Computer Engineering The University of Texas at Austin