Reweighting

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 133% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 44

All topics

Does Muon Need Fine-Grained Spectral Shaping?
Oct 5, 2026Meher Chaitanya, Tianyi Zhou, Aristides GionisMuon Reweighting
KTH Royal Institute of Technology Stockholm, Sweden
DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation
Oct 1, 2026Zhengming Yu, Junkun Yuan, Haotian Yang +8Distribution Matching Distillation Dataset Distillation
Texas A&M University · Intelligent Creation, ByteDance
Optimal Transport Reweighting for Robust Learning under Spurious Correlations and Label Noise
Oct 1, 2026Sung Ho Jo, Seonghwi Kim, Wonsang Yun +1Noisy Labels Reweighting
Pohang University of Science and Technology (POSTECH)
Rethinking Data Augmentation under Covariate Shift: Invariant-Guided Diffusion and Prototype Reweighting
Oct 1, 2026Hongyu Cao, Xinyuan Wang, Arun Vignesh Malarkkan +3Data Augmentation Covariate Shift
School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ 85281 USA
Reward-Aligned Reweighting for On-Policy Distillation
Sep 28, 2026Haofeng Xu, Junwei Su, Lansong Diao +2Reweighting Process-Level Supervision
The University of Hong Kong · Alibaba Group · University of Science and Technology of China
Rethinking Token Reweighting for SFT: Suppress, Reverse, and Extrapolate Learned Features
Sep 27, 2026Cunchun Li, Haonan He, Yifan Gao +4Supervised Finetuning Reweighting
Shanghai AI Laboratory · University of Science and Technology of China · Fudan University +2
Local Evidence and Geometric Readout Repair in Trained GNNs
Sep 22, 2026Nadi Tomeh, Hugo AttaliGnn-Based Detectors Graph Neural Networks
Université Sorbonne Paris Nord, CNRS, Laboratoire d’Informatique de Paris Nord, LIPN, F-93430 Villetaneuse, France
Correcting Within-Group Self-Selection Bias in Prioritized Replay
Sep 21, 2026Oscar Miró López-Feliu, Herke van HoofPrioritized Experience Replay Group-Based Reinforcement Learning
University of Amsterdam
Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation
Sep 3, 2026Xuanfa Jin, Zhijian Ma, Yongcheng Zeng +3Multi-Agent Debate Reweighting
Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University College London
Tracing Provenance and Detecting Tampering with Complementary LLM Watermarks
Aug 13, 2026Xiaoyan Feng, Yanjun Zhang, He Zhang +2Large Language Model Watermarks Tamper Localization
Griffith University
PAIR: Pairwise-Aware Inclusion Reweighting for Adaptive Rollout Allocation in RLVR
Aug 11, 2026Pixel Nomand, Elena Voss, Marcus Hale +1Reinforcement Learning With Verifiable Reward Verifiable Rewards
University of Wisconsin–Madison · University of Washington
VADER: Adaptive Debiasing for Hallucination Mitigation in Video Large Language Models
Aug 9, 2026Dong Xing, Jiaxin Chen, Hang Yang +3Video-Language Models Video Understanding
Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences Changchun, Jilin 130033, China · University of Chinese Academy of Sciences, Beijing 100049, China · City University of Hong Kong, Hong Kong, China
Stop Replacing Noise with Noise: Two-Source Reliability Assessment for Label Correction and Sample Reweighting in Label-Noise Learning
Aug 4, 2026Wenxiao Fan, Kan LiNoisy Labels Noise-Aware
School of Computer Science, Beijing Institute of Technology
Recurrent Contrastive Learning for Imbalanced Medical Image Classification
Aug 4, 2026Zhiyuan Zhu, Xinling Meng, Junxuan Yu +15Medical Image Classification Contrastive Learning
School of Biomedical Engineering, Medical School, Shenzhen University, China · Ultra-X AI Lab, Shenzhen University, Guangdong, China · School of Biomedical Engineering, South-Central Minzu University, Wuhan, China +4
Bayesian Data Reweighting Improves Multimodal Retrieval for Knowledge-Based Visual Question Answering
Aug 3, 2026Jingchen Sun, Shaobo Han, Ruiyi Zhang +5Knowledge-Based Visual Question Answering Multimodal Retrieval
University at Buffalo · NEC Laboratories America · Adobe Research +2
Rewriting or Reweighting? A Geometric Account in Language Models
Aug 3, 2026Juntong Wang, Shengkun Yang, Xiyuan Wang +1Behavioral Foundation Models Model Activations
Institute for Artificial Intelligence, Peking University, Beijing, China · School of Intelligence Science and Technology, Peking University, Beijing, China · School of Computer Science and Technology, Beijing Jiaotong University, Beijing, China
Harnessing the Potential of Optimizing Data Mixtures via Bayesian Domain Reweighting
Jul 30, 2026Xiang Yuan, Kaiqing Lei, Zhenyu Jin +3Reweighting Model Weights
School of Mathematics and Statistics, Xi’an Jiaotong University · Northwest University, Shaanxi, China
ReDiPPO: Reference-Guided Value Calibration and Discrepancy-Aware Token Reweighting for Mathematical Reasoning
Jul 30, 2026Zhenrong Zhang, Fei Wu, Jun Du +2Mathematical Reasoning Benchmarks Mathematical Reasoning
1IFLYTEK Research · University of Science and Technology of China
ReCo: Reweighting GRPO Against Distributional Concentration
Jul 29, 2026Junoh Park, Junseo Hwang, Wonguk Cho +1Reweighting Mathematical Reasoning Benchmarks
Graduate School of Data Science, Seoul National University
Chaos Is a LADDER: Domain Generalization Beyond Invariance via Reweighting
Jul 29, 2026Yuhang Jiang, Fengchuan Zhang, Sanguo Zhang +1Multimodal Domain Generalization Reweighting
School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, China · Key Laboratory of Big Data Mining and Knowledge Management, Chinese Academy of Sciences, Beijing, China
Online Variance Reduction for Domain Adaptation on Streaming Data
Jul 22, 2026Andrea NapoliVariance Reduction Domain Adaptation
Department of Electronics and Computer Science University of Southampton, UK
NFTR: From Provable Mode-Averaging to Geodesic Subgoal Selection in Offline Goal-Conditioned RL
Jul 8, 2026Erdemt Bao, Xing Lei, Jun Chen +2Goal-Conditioned Reinforcement Learning Reweighting
Huazhong University of Science and Technology · Xi’an Jiaotong University · University of Electronic Science and Technology of China
CenSynCMB: Centre Maps and Physics-Guided Synthesis for Microbleed Detection
Jul 6, 2026Lucas He, Hanyuan Zhang, Krinos Li +7Lesion Physics-Guided Machine Learning
Hawkes Institute, University College London, UK · Unit for Lifelong Health and Aging, University College London, UK · Bioengineering Department and Imperial-X, Imperial College London, UK +8
Target-Guided Selective Reweighting for Physics-Informed Neural Network Inverse Problems: A Transfer Learning Approach
Jul 6, 2026Qian Hu, Bin Fan, Yao Xiao +2Parametric Physics-Informed Neural Network Inverse Problem
School of Computing and Data Science, Fujian University of Technology, Fuzhou, 350118, Fujian, China
Robust Trajectory Distillation: Hybrid Reweighting Meets Teacher-Inspired Targets
Jun 29, 2026Kaifeng Chen, Lechao Cheng, Jiyang Li +6Diffusion-Based Dataset Distillation Dataset Distillation
Hefei University of Technology · Jianghuai Advance Technology Center · Anhui Provincial Key Laboratory of Humanoid Robots +2
Ribbon: Scalable Approximation and Robust Uncertainty Quantification
Jun 25, 2026Graham Gibson, John Tipton, Kellin Rumsey +1Uncertainty Quantification Uncertainty
Sample-efficient Transfer Reinforcement Learning via Adaptive Reward Shaping and Policy-Ratio Reweighting Strategy
Jun 25, 2026Wenjie Huang, Yang Li, Jingjia Teng +6Potential-Based Reward Shaping Offline Reinforcement Learning
State Key Laboratory of Advanced Design and Manufacturing Technology for Vehicle, College of Mechanical and Vehicle Engineering, Hunan University, Changsha 410082, China · Key Laboratory of Intelligent Air-Ground Cooperative Control for Universities in Chongqing, School of Automation, Chongqing University of Posts and Telecommunications, Chongqing 400065, China · Xi’an Institute of High-Tech, Xi’an 710025, China +1
Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot Imitation
Jun 23, 2026Xinyao Qin, Junjie Lu, Kaixin Wang +7Imitation Value Functions
Tsinghua University · Work done at Microsoft Research Asia · University of Technology Sydney +3
Neural Architecture Search of Sample Reweighting Networks for Complex Distribution Shift
Jun 22, 2026Keisuke Sugawara, Kento Uchida, Shinichi ShirakawaReweighting Neural Architecture Search
Yokohama National University, Kanagawa, Japan
GD $^2$ PO: Mitigating Multi-Reward Conflicts via Group-Dynamic reward-Decoupled Policy Optimization
Jun 15, 2026Haotian Liu, Yihao Liu, Jingwei Ni +11Offline Reinforcement Learning Reweighting
Alibaba · Renmin University of China · Peking University +3
TANDEM: Bi-Level Data Mixture Optimization with Twin Networks
Jun 3, 2026Jiaxing Wang, Deping Xiang, Jin Xu +9Mixture Models Reweighting
University of Oxford · Renmin University of China · University of Chinese Academy of Sciences
TIAR: Trajectory-Informed Advantage Reweighting for LLM Abstention Learning
May 25, 2026Muyu Pan, Shu Zhao, Nan Zhang +4Large Language Model Reliability Reweighting
Department of Computer Science, The Pennsylvania State University State College, PA, USA
CurveRL: Principled Distribution-Aware Context Reweighting for LLM Reasoning
May 23, 2026Ke Sun, Yizhou Zhao, Jiayi Xin +2Reinforcement Learning With Verifiable Reward Reweighting
University of Pennsylvania
GEAR: Granularity-Adaptive Advantage Reweighting for LLM Agents via Self-Distillation
May 12, 2026Sijia Li, Yuchen Huang, Zifan Liu +7Credit Assignment Self-Distillation Framework
Hong Kong University of Science and Technology · Microsoft Research Asia
Let the Target Select for Itself: Data Selection via Target-Aligned Paths
May 10, 2026Huitao Yang, Hengzhi He, Tung Sum Thomas Kwok +1Alignment-Aware Selection Adaptive Sampling
University of California, Los Angeles
Q-MMR: Off-Policy Evaluation via Recursive Reweighting and Moment Matching
May 7, 2026Xiang Li, Nan JiangValue Functions Reweighting
Nanjing University · UIUC
CredibleDFGO: Differentiable Factor Graph Optimization with Credibility Supervision
May 7, 2026Liang Qian, Penggao Yan, Penghui Xu +1Global Navigation Satellite System Factor Graphs
Department of Aeronautical and Aviation Engineering The Hong Kong Polytechnic University, Hong Kong, China
InfoLaw: Information Scaling Laws for Large Language Models with Quality-Weighted Mixture Data and Repetition
May 4, 2026Fengze Liu, Weidong Zhou, Binbin Liu +7Large Language Model Pretraining Scaling Laws
1ByteDance · 2UC Santa Cruz
Meta Additive Model: Interpretable Sparse Learning With Auto Weighting
Apr 22, 2026Xuelin Zhang, Xinyue Liu, Lingjuan Wu +1Generalized Linear Model Reweighting
College of Informatics, Huazhong Agricultural University, Wuhan, China
Rethinking Data Curation in LLM Training: Online Reweighting Offers Better Generalization than Offline Methods
Apr 19, 2026Wanru Zhao, Yihong Chen, Yuzhi Tang +6Data-Curation Large Language Model Training
University of Cambridge · OATML, University of Oxford · University of Toronto +2
Beyond Logit Adjustment: A Residual Decomposition Framework for Long-Tailed Reranking
Apr 2, 2026Zhanliang Wang, Hongzhuo Chen, Quan Minh Nguyen +2Reranking Logit Lens
University of Pennsylvania, Philadelphia, PA, USA · Children’s Hospital of Philadelphia, Philadelphia, PA, USA
RapidUn: Influence-Driven Parameter Reweighting for Efficient Large Language Model Unlearning
Dec 4, 2025Guoshenghui Zhao, Huawei Lin, Weijie ZhaoLarge Language Model Unlearning Reweighting
Golisano College of Computing and Information Sciences, Rochester Institute of Technology
Fine-Tune, Then Rectify
Nov 23, 2025Zikun Ye, Jinglong Zhao, Lei WangModel Fine-Tuning Reweighting
Foster School of Business, University of Washington, Seattle, WA · Questrom School of Business, Boston University, Boston, MA
A Compositional Kernel Model for Feature Learning
Sep 17, 2025Feng Ruan, Keli Liu, Michael JordanKernel Ridge Regression Kernel Method
*Department of Statistics and Data Science, Northwestern University. · *Company E · *Department of Statistics and Computer Science, University of California, Berkeley.

Oct 5, 2026, cs.AIJ/K move · Enter open · S save

Does Muon Need Fine-Grained Spectral Shaping?

Meher Chaitanya, Tianyi Zhou, Aristides Gionis

KTH Royal Institute of Technology Stockholm, Sweden

Muon Reweighting

Open paper AskPDF arXiv

Muon combines current and past gradients into matrix momentum. For

M=UΣV^\top

, the idealized polar update

Q=UV^\top

gives every singular direction the same weight. We refer to this as the flat profile. Several recent optimizers replace this flat profile with fine-grained spectral maps that give each direction its own gain. We ask how much of this spectral detail a Muon update needs. Our spectral diagnostics show that approximately

94

97\%

of measured singular modes lie below an estimated noise edge, yet collectively align positively with a reference gradient. We introduce BulkBoost, a two-band spectral reweighting framework with fixed-rank and noise-calibrated variants. The latter uses split-minibatch gradient differences to calibrate a Marchenko--Pastur reference edge for Muon's Nesterov input, separating the bulk below the edge from the spikes above it. Both variants increase the bulk's relative weight through one shared gain while preserving the Frobenius norm of each matrix's unreweighted direction. For a fixed partition, our theory gives the first-order condition under which moving weight toward the bulk lowers the loss. It also quantifies the fraction of the maximal first-order improvement rate, over all per-mode reallocations, that two bands can capture. Across 30 continued-pretraining settings spanning Pythia-14M to 410M and six corpora, two-band reweighting is competitive with the fine-grained power-law profile of Freon and outperforms Spectra. Measured against Muon's flat profile, Freon reduces final loss by

0.022\%

of the pre-adaptation loss on average, whereas the two-band variants achieve reductions of

0.073

0.147\%

. These observations suggest that useful departures from the flat profile are surprisingly low-dimensional: a single bulk-to-spike gain captures at least as much benefit as the fine-grained spectral profiles.

Momentum

Related topics

Latest papers 44