Overview

Gradient alignment studies whether updates from different tasks, examples, models, or data sources point in compatible directions, and how that agreement affects learning and generalization. Common approaches measure gradient cosine similarity, select or reweight training examples, modify updates with gradient surgery or preconditioning, and analyze how curvature and gradient scale alter optimization. These ideas are applied in settings such as domain generalization, federated and multimodal learning, reasoning post-training, distillation, and memorization. A central tension is that alignment can support transfer and stability, but direction alone may be misleading when gradients are noisy, poorly scaled, or interact with sharp curvature.

Latest papers

CardsList

23 papers1 paper in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

  1. AlignedHPO: Efficient Surrogate-Guided Loss Hyperparameter Optimization

    Nov 17, 2026Ivan Karpukhin, Andrey SavchenkoHyperparameter OptimizationHypergradient Optimization

  2. An Educated Guess: Deriving Statistically Aligned Gradient Estimates for Zeroth-Order Optimisation

    Nov 5, 2026Justus Huebotter, Serge Thill, Marcel van Gerven +1Zeroth-Order OptimizationNeural Network Optimization

  3. Wrong Votes, Right Gradients: Why Majority-Vote Rewards Track Verifiable RL

    Oct 14, 2026Dylan P. Wilson, Chengsong Huang, Jiaxin HuangRL for Language Model ReasoningNoisy-Label Learning

  4. Don't Forget! Decomposing the Training Dynamics of Memorization in Language Models

    Sep 28, 2026Florian Eichin, Philipp Mondorf, Andrei Mircea +3Memorization in Language ModelsData Repetition in Language Model Training

  5. NAE: Normalizing AutoEncoder

    Aug 12, 2026Muhammad Abdur Rafae, Niels LandwehrNormalizing FlowsAutoencoders

  6. Mechanisms of Width Scaling in Normalized Residual Networks: The Effective Alignment Dimension

    Jul 27, 2026Jinhao Zhang, Zeyu Liu, Zicheng Yan +4Neural Network GeneralizationResidual Learning

  7. What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility

    Jun 19, 2026Yuren Hao, Xiang Wan, ChengXiang ZhaiCredit AssignmentGradient Alignment

  8. One-Step Generalization Ratio Guided Optimization for Domain Generalization

    Jun 15, 2026Sumin Cho, Dongwon Kim, Kwangsu KimDomain GeneralizationAdaptive Gradient Methods

  9. GRASP: Gradient-Aligned Sequential Parameter Transfer for Memory-Efficient Multi-Source Learning

    Jun 12, 2026Mary Isabelle Wisell, Nicholas Jacobs, Aayush Manandhar +1Domain-Incremental LearningCross-Domain Transfer Learning

  10. Radial-Angular Geometry for Reliable Update Diagnosis in Noisy-Label Learning

    May 17, 2026Ningkang Peng, Jingyang Mao, Xiaoqian Peng +2Noisy-Label LearningGradient Alignment

  11. GRACE: Gradient-aligned Reasoning Data Curation for Efficient Post-training

    May 13, 2026Junjie Li, Ziao Wang, NingXuan Ma +2Training Data SelectionLLM Post-Training

  12. Flatness and Gradient Alignment Are Both Necessary: Spectral-Aware Gradient-Aligned Exploration for Multi-Distribution Learning

    May 8, 2026Aristotelis Ballas, Christos DiouSharpness-Aware MinimizationMulti-Task Learning

  13. Directional Consistency as a Complementary Optimization Signal: The GONO Framework

    May 7, 2026Victor Daniel GeraDeep Learning OptimizationStochastic Optimization Convergence

  14. Personalized Federated Learning for Gradient Alignment

    May 4, 2026Dongwon Kim, Gyuejeong LeeFederated Learning AggregationPAC-Bayesian Theory

  15. Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima

    Apr 10, 2026Huanran Chen, Huaqing Zhang, Xiao Li +3Language Model PretrainingNeural Network Generalization

  16. Harmonized Tabular-Image Fusion via Gradient-Aligned Alternating Learning

    Apr 2, 2026Longfei Huang, Yang YangMultimodal FusionGradient Interference

  17. GAPSL: A Gradient-Aligned Parallel Split Learning over Data-Heterogeneous Edge Computing Systems

    Mar 19, 2026Zheng Lin, Ons Aouedi, Zihan Fang +4Split Federated LearningGradient Alignment

  18. Not All Preferences Deserve Gradients: Understanding Gradient Utility in Offline Reasoning Alignment

    Feb 1, 2026Hui Wu, Hengyi Cai, Jinman Zhao +6Pairwise Preference LearningTraining Data Selection

  19. Spectral Gradient Descent Mitigates Anisotropy-Driven Misalignment: A Case Study in Phase Retrieval

    Jan 30, 2026Guillaume Braun, Han Bao, Wei Huang +1Gradient DescentGradient Descent Dynamics