Gradient Alignment

Follow this topic Ask about this topic

Gradient alignment studies whether updates from different tasks, examples, models, or data sources point in compatible directions, and how that agreement affects learning and generalization. Common approaches measure gradient cosine similarity, select or reweight training examples, modify updates with gradient surgery or preconditioning, and analyze how curvature and gradient scale alter optimization. These ideas are applied in settings such as domain generalization, federated and multimodal learning, reasoning post-training, distillation, and memorization. A central tension is that alignment can support transfer and stability, but direction alone may be misleading when gradients are noisy, poorly scaled, or interact with sharp curvature.

Latest papers

21 papers2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Wrong Votes, Right Gradients: Why Majority-Vote Rewards Track Verifiable RL
Oct 14, 2026Dylan P. Wilson, Chengsong Huang, Jiaxin HuangRL for Language Model Reasoning Noisy-Label Learning
Dartmouth College · Washington University in St. Louis
Don't Forget! Decomposing the Training Dynamics of Memorization in Language Models
Sep 28, 2026Florian Eichin, Philipp Mondorf, Andrei Mircea +3Memorization in Language Models Data Repetition in Language Model Training
MaiNLP, Center for Information and Language Processing, LMU Munich & Munich Center for Machine Learning (MCML), Germany · Mila – Quebec AI Institute & University of Montreal, Canada · Saarland University, Germany
NAE: Normalizing AutoEncoder
Aug 12, 2026Muhammad Abdur Rafae, Niels LandwehrNormalizing Flows Autoencoders
University of Hildeshiem
When Do Surrogate Updates Improve Decisions? A Local Theory of Trajectory-Wise Transfer
Aug 2, 2026Yuyang ShenSurrogate Modeling Decision-Focused Learning
The Chinese University of Hong Kong, Shenzhen
Mechanisms of Width Scaling in Normalized Residual Networks: The Effective Alignment Dimension
Jul 27, 2026Jinhao Zhang, Zeyu Liu, Zicheng Yan +4Neural Network Generalization Residual Learning
Beijing University of Posts and Telecommunications · Institute of Computing Technology, Chinese Academy of Sciences · University of Science and Technology of China +1
What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility
Jun 19, 2026Yuren Hao, Xiang Wan, ChengXiang ZhaiCredit Assignment Gradient Alignment
Department of Computer Science University of Illinois at Urbana-Champaign · Department of Computer Science Stanford University
One-Step Generalization Ratio Guided Optimization for Domain Generalization
Jun 15, 2026Sumin Cho, Dongwon Kim, Kwangsu KimDomain Generalization Adaptive Gradient Methods
Department of Computer Science and Engineering, University of Sungkyunkwan, Suwon, Korea.
GRASP: Gradient-Aligned Sequential Parameter Transfer for Memory-Efficient Multi-Source Learning
Jun 12, 2026Mary Isabelle Wisell, Nicholas Jacobs, Aayush Manandhar +1Domain-Incremental Learning Cross-Domain Transfer Learning
San Diego State University, San Diego, CA, USA · University of Utah, Salt Lake City, UT, USA · University of Maine, Orono, ME, USA
Edge of Stability Selectively Shapes Learning Across the Data Distribution
Jun 2, 2026Shauna Kwag, Anakha Ganesh, Tomaso Poggio +1Edge of Stability Gradient Descent Dynamics
MIT
Radial-Angular Geometry for Reliable Update Diagnosis in Noisy-Label Learning
May 17, 2026Ningkang Peng, Jingyang Mao, Xiaoqian Peng +2Noisy-Label Learning Gradient Alignment
Nanjing Normal University · Nanjing University of Chinese Medicine
GRACE: Gradient-aligned Reasoning Data Curation for Efficient Post-training
May 13, 2026Junjie Li, Ziao Wang, NingXuan Ma +2Training Data Selection LLM Post-Training
Harbin Institute of Technology, Shenzhen, China · Hong Kong Baptist University, China
Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Why
May 11, 2026Mohammadreza Armandpour, Fatih Ilhan, David Harrison +6On-Policy Self-Distillation Language Model Distillation
Apple
Flatness and Gradient Alignment Are Both Necessary: Spectral-Aware Gradient-Aligned Exploration for Multi-Distribution Learning
May 8, 2026Aristotelis Ballas, Christos DiouSharpness-Aware Minimization Multi-Task Learning
Department of Informatics and Telematics Harokopio University of Athens Omirou 9, Athens, Greece
Directional Consistency as a Complementary Optimization Signal: The GONO Framework
May 7, 2026Victor Daniel GeraDeep Learning Optimization Stochastic Optimization Convergence
Department of Artificial Intelligence Anurag University, Hyderabad, India
Personalized Federated Learning for Gradient Alignment
May 4, 2026Dongwon Kim, Gyuejeong LeeFederated Learning Aggregation PAC-Bayesian Theory
SAKAK Inc., Korea · Div. of Artificial Intelligence and Data Science, Korea Cyber University, Korea
Sustained Gradient Alignment Mediates Subliminal Learning in a Multi-Step Setting: Evidence from MNIST Auxiliary Logit Distillation Experiment
Apr 28, 2026Chayanon Kitkana, Shivam AroraSubliminal Learning Teacher-Student Learning
Independent · Equivariant labs
Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima
Apr 10, 2026Huanran Chen, Huaqing Zhang, Xiao Li +3Language Model Pretraining Neural Network Generalization
Tsinghua University · ByteDance Seed
Harmonized Tabular-Image Fusion via Gradient-Aligned Alternating Learning
Apr 2, 2026Longfei Huang, Yang YangMultimodal Fusion Gradient Interference
Nanjing University of Science and Technology
GAPSL: A Gradient-Aligned Parallel Split Learning over Data-Heterogeneous Edge Computing Systems
Mar 19, 2026Zheng Lin, Ons Aouedi, Zihan Fang +4Split Federated Learning Gradient Alignment
Department of Electrical and Computer Engineering, University of Hong Kong, Pok Fu Lam, Hong Kong, China · Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, L-1855 Luxembourg, Luxembourg · Hong Kong JC STEM Lab of Smart City and Department of Computer Science, City University of Hong Kong, Kowloon, Hong Kong SAR, China +2
Not All Preferences Deserve Gradients: Understanding Gradient Utility in Offline Reasoning Alignment
Feb 1, 2026Hui Wu, Hengyi Cai, Jinman Zhao +6Pairwise Preference Learning Training Data Selection
Baidu Inc. · University of Toronto · School of Intelligence Science and Technology, Peking University +1
Spectral Gradient Descent Mitigates Anisotropy-Driven Misalignment: A Case Study in Phase Retrieval
Jan 30, 2026Guillaume Braun, Han Bao, Wei Huang +1Gradient Descent Gradient Descent Dynamics
RIKEN AIP · The Institute of Statistical Mathematics · Tohoku University +2

Oct 14, 2026J/K move · Enter open · S save

Wrong Votes, Right Gradients: Why Majority-Vote Rewards Track Verifiable RL

Dylan P. Wilson, Chengsong Huang, Jiaxin Huang

Dartmouth College · Washington University in St. Louis

RL for Language Model Reasoning Noisy-Label Learning Gradient Alignment Self-Consistency in Language Models Reinforcement Learning with Verifiable Rewards

Open paper AskPDF arXiv

Reinforcement learning with self-generated labels, such as majority vote over a model's own samples, often recovers most of the gain of training with verifiable rewards. This is puzzling from the usual prompt-level view, in which majority vote only returns a model's modal answer and cannot add information about correctness. We argue that the relevant unit is the batch-level update. Because parameters are shared, a prompt's own gradient accounts for less than 1% of the change in that prompt's output at practical batch sizes, so a wrong label matters only to the extent that label errors share a direction across prompts. We decompose the majority-vote update into shrinkage and misdirection relative to the gold update, and show that misdirection vanishes with batch size whenever errors are incoherent across prompts, however often the labels are wrong. Measuring per-response gradients on five models, we find that majority-vote errors are as incoherent as a random-reward null, that the majority-vote update agrees with the gold update as closely as two independent gold updates agree with each other, and that its noise-corrected cosine with the gold update is 1 within error. Randomly flipping up to 30% of gold labels leaves the update direction intact, while the same fraction flipped by a shared rule destroys it, and rewards that favor a kind of response on every prompt, such as a learned reward model, fall measurably short. The equivalence holds at every checkpoint we examine but is local: gold-reward training still ends ahead on held-out majority accuracy. Error coherence, not error rate, determines when self-generated labels can stand in for verification.