Gradient Component

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 27

All topics

Preemptive LLM Unlearning against Forbidden Capability Acquisition via Gradient Sealing
Sep 30, 2026Kemou Li, Qizhou Wang, Yue Wang +6Exact Unlearning Data Leakage
State Key Laboratory of Internet of Things for Smart City, University of Macau · RIKEN Center for Advanced Intelligence Project · The University of Melbourne +5
G $^3$ -LoRA: Organizing Reward-Weighted Video Data with Gradient-Guided Grouped LoRA
Sep 28, 2026Jia Song, Wenhow Li, Lichen Bai +2Video Foundation Models Multi-Lora
The Hong Kong University of Science and Technology · Tencent
ORPG: Reconciling Multiple Reward Objectives through Objective-wise Policy Gradients
Sep 28, 2026Shicheng Fang, Yiwen Zhao, Wenbo Tian +4Reward-Only Policy Optimization Variants Gradient Component
Fudan University · Shanghai Innovation Institute
RoboDrop: Curating VLA Post-Training Data via Local Gradient Compatibility
Sep 9, 2026Runze Xu, Yuanfan Xu, Cuijie Xu +4Large-Scale Robot Demonstration Datasets Robocasa
Tsinghua University · Striding AI
M-Net: Integrating Spectral Features and Physical Field Operators into Deep Learning for Medical Image Segmentation
Aug 12, 2026Jing Zhu, Ye Wang, Fumin WangSemi-Supervised Medical Image Segmentation U-Net
School of Physics, Xi’an Jiaotong University, Xi’an 710049, China · University of Sydney, Sydney, NSW 2006, Australia
Gradient Concentration, Not Weight Saliency, Explains Representation-Level Class Unlearning
Jul 23, 2026Billel Habbati, Alessio Merlo, Luca Verderame +1Machine Unlearning Catastrophic Forgetting
DIBRIS – Department of Informatics, Bioengineering, Robotics and Systems Engineering, University of Genova, Via Dodecaneso 35, 16146 Genova, Italy · Centre for Defense Higher Studies (CASD), Palazzo Salviati, Piazza della Rovere 83, 00165 Roma, Italy
Information-Theoretically Secure Aggregation for Lightweight Federated Learning: Resilient to Dropouts and Adversaries
Jul 23, 2026Hyeong-Gun Joo, Songnam Hong, Dong-Joon ShinSecure Aggregation Federated Learning
Department of Electronic Engineering, Hanyang University, Seoul, South Korea
PRIME: Plasticity Recovery in Multi-Agent Environments for UAV-Assisted Emergency Communication Networks
Jul 20, 2026Wen Qiu, Zhiqiang He, Wei Zhao +1Multi-Agent Reinforcement Learning Multi-Agent Planning
Department of Information and Communication Engineering, Kitami Institute of Technology, Japan · Graduate School of Informatics and Engineering, University of Electro-Communications, Japan · School of Computer Science and Technology, Anhui University of Technology, China
FlexiGrad: Adaptive Gradient Modulation for Hierarchical Fine-Grained Classification
Jul 20, 2026Zilu Zhou, Dongliang Chang, Junhan Chen +1Fine-Grained Perception Gradient Component
Beijing University of Posts and Telecommunications, Beijing 100876, China · Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance, Beijing 100876, China
Heterogeneous Knowledge Distillation via Geometry Decoupling and Momentum-Aware Gradient Regulation
Jun 23, 2026Wuming Yang, Xiang Zhang, Hongmin ZhaoKnowledge Distillation Exponential Moving Average
School of Computer and Mathematics, Central South University of Forestry and Technology, Changsha, Hunan, China · College of Computer Science and Technology, and the Laboratory of Digitizing Software for Frontier Equipment, National University of Defense Technology, Changsha, Hunan, China
A Gradient Perspective on RLVR Stability and Winner Advantage Policy Optimization
Jun 15, 2026Prasanth YSS, Zhichen Ren, Rasa Hosseinzadeh +6Reinforcement Learning With Verifiable Reward Gradient Component
Layer 6 AI
HadBalance: A Plug-and-Play Unified Global Geometric Prior Framework for Generalizable Biomedical Segmentation
Jun 14, 2026Zhuangzhi Gao, Feixiang Zhou, He Zhao +11Semi-Supervised Medical Image Segmentation Geometric Priors
Department of Eye and Vision Science, University of Liverpool, Liverpool, UK · Department of Primary Care and Mental Health, University of Liverpool, Liverpool, UK · Computer Vision Research Group, University of Amsterdam, Netherlands +5
Bergson: An Open Source Library for Data Attribution
Jun 10, 2026Lucia Quirke, Louis Jaburi, David Johnston +6Interpretability Open-Source
EleutherAI · Independent
Adaptive directional gradients for parameterised quantum circuits
Jun 8, 2026Brian Coyle, Snehal Raj, Virag Umathe +2Quantum Circuits Quantum Approximate Optimization Algorithm
School of Informatics, University of Edinburgh, Edinburgh, UK · LIP6, CNRS, Sorbonne Université, Paris, France · QC Ware, Palo Alto, USA and Paris, France +1
Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors
Jun 4, 2026Naïl B. Khelifa, Richard E. Turner, Ramji VenkataramananScore-Based Diffusion Model Diffusion Models
University of Cambridge
OPD+: Rethinking the Advantage Design for On-Policy Distillation
May 31, 2026Hanyang Zhao, Haoxian Chen, Han Lin +3Uni-Opd Step-Level Advantage Estimation
Columbia University · Amazon · Meta +1
Local linear convergence of gradient methods for overparameterized Gaussian mixtures
May 29, 2026Jingxing Wang, Vasileios Charisopoulos, Maryam FazelGaussian Mixture Models Overparameterization
Electrical & Computer Engineering, University of Washington, Seattle, WA · National Institute for Theory and Mathematics in Biology, Chicago, IL · Amazon, Inc.
SPAR: Support-Preserving Action Rectification
May 27, 2026Jiaxin Zhao, Weihang Pan, Xun Liang +1Gradient Component Rectification
Zhejiang University.
When Gradients Collide: Failure Modes of Multi-Objective Prompt Optimization for LLM Judges
May 25, 2026Parth Darshan, Abhishek DivekarLarge Language Model Judges Gradient Component
♢IIT Jodhpur · ♠Amazon
PILOT: Policy-Informed Learned Optimization for Adaptive Deep Network Training
May 23, 2026Sattam Altuuaim, Lama Ayash, Muhammad Mubashar +1Parameter-Efficient Adaptation Gradient Component
King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia · University of Strathclyde, Glasgow, Scotland
Grokking as Structural Inference: Transformers Need Bayesian Lottery Tickets
May 15, 2026Kai Hidajat, Solden Stoll, Joseph AnTransformer Architectures Sequence Modeling
Department of Computer Science University of Washington Seattle, WA 98195
Not How Many, But Which: Parameter Placement in Low-Rank Adaptation
May 12, 2026Arijit Sehanobish, Charles LoveringLow-Rank Adaptation Adapters Gradient Component
Kensho Technologies
Emergence of Frontier Superposition: Möbius attractor and Cascade Supervision
May 12, 2026Hongyu Gu, Jingwen FuForward Propagation Gradient Component
University of Science and Technology of China · Zhongguancun Academy
Holder Policy Optimisation
May 12, 2026Yuxiang Chen, Dingli Liang, Yihang Chen +8Gradient Component Token-Level Uncertainty
University College London, London, United Kingdom. · 3The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · 2Shanghai Jiao Tong University, Shanghai, China.
Budget-aware Auto Optimizer Configurator
May 6, 2026Kang Liu, Wei Peng, Jianchen HuOptimizer Design Graphics Processing Unit Memory
School of Future Technology Xi’an Jiaotong University Xi’an, China · School of Automation Science and Engineering Xi’an Jiaotong University Xi’an, China
Efficient Reinforcement Learning using Linear Koopman Dynamics for Nonlinear Robotic Systems
Apr 21, 2026Wenjian Hao, Yuxuan Fang, Zehui Lu +1Model-Based Reinforcement Learning Koopman Operator
School of Aeronautics and Astronautics, Purdue University, West Lafayette, IN 47907, USA · independent researcher
Auditing Information Disclosure During Large-Scale Gradient-Based Training via Gradient Uniqueness
Oct 13, 2025Sleem Abdelghafar, Maryam Aliakbarpour, Christopher JermaineGradient-Based Attacks Membership Inference Attacks
Rice University Houston, USA

Sep 30, 2026, cs.LGJ/K move · Enter open · S save

Preemptive LLM Unlearning against Forbidden Capability Acquisition via Gradient Sealing

Kemou Li, Qizhou Wang, Yue Wang, Fengpeng Li+5

State Key Laboratory of Internet of Things for Smart City, University of Macau · RIKEN Center for Advanced Intelligence Project · The University of Melbourne+5

Exact Unlearning Data Leakage Gradient Component Acquisition Activation Functions

Open paper AskPDF arXiv

Open-weight LLMs are released not only as fixed products but also as substrates for downstream fine-tuning. This openness, however, creates legal and ethical risks because users may misuse fine-tuning to instill illicit knowledge or enable hostile operations. Model providers therefore need apre-release defense against such acquisition, motivating the problem of preemptive unlearning. Unlike retrospective unlearning, which removes capabilities already present in a fixed model, preemptive unlearning seeks to prevent their acquisition under unseen attack data and future fine-tuning procedures. Despite its practical importance, this setting remains largely unexplored, presents distinct challenges, and is therefore the central focus of our work. We first verify that existing retrospective methods provide insufficient pre-release protection. Even when forbidden capabilities are suppressed in current outputs, forbidden-domain data can still induce gradients through internal pathways, enabling later acquisition. Motivated by this finding, we propose a gradient-sealing principle that blocks these pathways by pushing relevant pre-activations into the negative region, where ReLU-family activations exhibit zero or near-zero derivatives. Experiments across multiple LLM families demonstrate our stronger resistance to downstream acquisition than retrospective baselines, validating gradient sealing as an effective mechanism for pre-release protection.

Momentum

Related topics

Latest papers 27