Batch

Ask about this topic

Momentum

25 papers in the last four weeks, up 127% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 134

All topics
CardsList

  1. A Solvable Model of Adaptive Learning Rate Rescaling: Acceleration, Stability & Scaling

    Oct 5, 2026Itay Lavie, Clarissa Lauditi, Cengiz PehlevanBatch

    John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA · Center for Brain Science, Harvard University, Cambridge, MA · Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, Cambridge, MA

  2. Learning Rate Transfer for Hybrid Transformer-SSM Architectures

    Oct 1, 2026Jimin Seo, Gyubok Lee, Yeonsik Jo +11Hybrid TransformerTransformer Architectures

    Seoul National University · SB Intuitions · LG AI Research +1

  3. Every Batch Is Its Own Validation Set: Leave-One-Out Gradient Matching for Online Data Selection in LLM Fine-Tuning

    Sep 30, 2026Hongyu Chen, Xinyi Luo, Ming Zhao +6Large Language Model Fine-TuningBatch

    Sichuan University, Chengdu, China · University of Electronic Science and Technology of China, Chengdu, China

  4. From Spectra to Joint Schedules in LLM Pre-training: 3+3(+2) Scaling-Law Regimes

    Sep 30, 2026Yichen Wang, Fanghui Liu, Yudong ChenBatchStochastic Gradient Descent

    Department of Computer Sciences, University of Wisconsin–Madison, USA. · School of Mathematical Sciences, Institute of Natural Sciences and MOE-LSC, Shanghai Jiao Tong University, China.

  5. How Does Local Landscape Geometry Evolve in Language Model Pre-Training?

    Sep 30, 2026Zhanpeng Zhou, Yuhan Sun, Bingrui Li +4Large Language Model PretrainingBatch

    Shanghai Jiao Tong University · Tsinghua University · Peking University

  6. Cool the Sampler, Not the Learner: Sampling Temperature Moves the Staleness Cliff of Importance-Corrected GRPO

    Sep 29, 2026Taiheng PanReward GradientsBatch

    The University of Melbourne

  7. AutoLoCo: Communication Efficient Distributed LLM Training via Adaptive Synchronization

    Sep 29, 2026Pengyu He, Yan Zhang, Ruien Li +1Large Language Model TrainingBatch

    Department of Electrical Engineering, California Institute of Technology · Department of Computer Science and Technology, Tsinghua University · Department of Computer Sciences, University of Wisconsin - Madison

  8. Fast Learning Rate Transfer in Shallow Linear Networks at Growing Training Horizons

    Sep 28, 2026Mana Sakai, Masaaki ImaizumiBatch

    The University of Tokyo · RIKEN Center for Advanced Intelligence Project · Kyoto University

  9. Muon Sublates the Edge of Stability in LLM Pretraining

    Sep 28, 2026Yanzhe Chen, Qifang Zhao, Xiaoxiao Xu +1MuonLarge Language Model Alignment

    School of Mathematical Sciences, Shanghai Jiao Tong University, China · Alibaba Inc., Hangzhou, China · School of Mathematical Sciences, Institute of Natural Sciences and MOE-LSC, Shanghai Jiao Tong University, China

  10. SOLAR: A State-Driven Online Learning Rate Scheduler for LLM Pretraining

    Sep 28, 2026Qiulin Shang, Binyu Wang, Yongqi Qiao +3BatchLarge Language Model Pretraining

    Peking University · Nanjing University

  11. Causal Routing for Unlearning

    Sep 28, 2026Bardh Prenkaj, Andrea D'Angelo, Davide Mottin +4Large Language Model MemoryCausal

    Sapienza University of Rome · Aarhus University

  12. Activation-Flexible ANN-to-SNN Conversion with Finite-State Markov Neurons

    Sep 24, 2026Ruiyu Jia, Zhuo-Cheng XiaoSpiking Neural NetworksActivation Functions

    New York University Shanghai, Shanghai, China

  13. Industrial Anomaly Detection via Defect-Grounded Reasoning in Visual Latent Space

    Sep 24, 2026Jaron Yeh, Yen-Wei Chang, Jiang Liu +1Latent Visual ReasoningEfficient Latent Reasoning

    National Taiwan University · AMD GenAI

  14. Does Step Law Transfer to Small-Scale Language Models? An Empirical Recalibration Below 59M Parameters

    Sep 23, 2026Egor Romanyukov, Timofey Novikov, Timur Shokarov +3BatchSmall Large Language Models

    Faculty of Computer Science HSE University

  15. From Experts to Sub-experts: Fine-grained Parameter-Efficient Fine-Tuning for MoE LLMs

    Sep 22, 2026Zhentao Tan, Chang Liu, Yao Liu +2Parameter-Efficient Fine-Tuning MethodsMixture-Of-Experts Large Language Models

    Alibaba Group · Southeast University

  16. Hill Sampling for Test-Time Scaling: A Simple and Better Alternative to Repeated Sampling, Evolution, and Training

    Sep 22, 2026Jacob Beck, Philip V. Ogren, Ari KobrenTest-Time ScalingEvolutionary Search

    Oracle

  17. Terminal Shrinkage Averaging Reveals a Schedule-Estimator Interaction in LLM Pretraining

    Sep 21, 2026Adam Ousherovitch, Yixin WangBatchLarge Language Model Pretraining

    Department of Statistics, University of Michigan

  18. Fast Learning Rates for Physics-Informed Kernel Methods

    Sep 16, 2026Luc Brogat-Motte, Joachim Bona-Pellissier, Giacomo Meanti +1Physics-Informed LearningNonlinear Operators

    SAIL Unit, Istituto Italiano di Tecnologia, Genoa, Italy · MaLGa Center, DIBRIS, Università degli Studi di Genova, Genoa, Italy

  19. Reaching Every Position Without Searching: Rotating Sparse Wiring on the Hypercube as a Substitute for Attention

    Sep 16, 2026Yoshiaki TakashitaDynamic Sparse AttentionAttention Layers

    School of Law, Waseda University

  20. Federated stochastic bilevel optimization with fully first-order gradients

    Sep 14, 2026Yihan Zhang, Rohit Dhaipule, Chiu C Tan +2Bilevel OptimizationGradient Descent

    1Temple University · 2Stony Brook University

  21. High-Probability Convergence of SGD via Batched Updates

    Sep 14, 2026Feng Zhu, Robert W. Heath Jr., Aritra MitraStochastic Gradient DescentFederated Learning

  22. Generalization Analysis of Distributed Kernel-based Robust Gradient Descent Algorithms

    Sep 10, 2026Jun-Yi Meng, Zheng-Chu Guo, Yuan MaoDistributionally-Robust OptimizationStochastic Gradient Descent

    School of Mathematical Sciences, Zhejiang University, Hangzhou 310058, P. R. China · College of Informatics, Huazhong Agricultural University, Wuhan 430070, P. R. China

  23. When Does Scale-Invariant Optimization Become Unstable? An Exact Schedule Law with Weight Decay

    Sep 8, 2026Hasan Amin, Wei-Kai Chang, Rajiv KhannaWeight DecayBatch

    Purdue University

  24. Equivariance Breaks the Learning Rate

    Sep 8, 2026Andrei Manolache, Mathias NiepertAdamEquivariant Neural Networks

    University of Stuttgart, Germany · Bitdefender, Romania

  25. BrachistoneLR: A Brachistochrone-Inspired Learning-Rate Schedule and a Controlled Benchmark of Scheduling Policies

    Sep 8, 2026Md. Sadekur Rahman Roni, Md. Jalal uddin Chowdhury, Moutusi Dash NimiBatchAnnealing

    Department of Computer Science and Engineering Leading University Sylhet, Bangladesh · DeepNet Research and Development Lab Sylhet 3100, Bangladesh · Department of Electrical and Electronic Engineering Leading UniversitySep Sylhet, Bangladesh

  26. TASTE: Throughput-Aware Batch Size Tuning for On-Device Edge Learning

    Sep 7, 2026Avik Bhatnagar, Federico Nicolas Peccia, Oliver BringmannEdge DevicesBatch

    FZI Research Center for Information Technology, University of Tübingen Germany

  27. Post-Training Science for Supervised Fine-Tuning

    Sep 1, 2026Charles O'Neill, Mudith Jayasekara, Harry PartridgeModel Fine-TuningBatch

    1Baseten

  28. When Does Online Adaptation Pay on the Edge? A Leakage-Free Evaluation of Warmup, Learning-Rate Selection, and Resource Trade-offs for Time-Series Forecasting

    Sep 1, 2026Takumi Fujimoto, Hiroaki NishiNon-StationarityTime Series Forecasting

    School of Science for Open and Environmental Systems, Graduate School of Science and Technology, Keio University, Yokohama, Japan

  29. Stochastic Optimization of Tree Tensor Networks

    Sep 1, 2026Marius Willner, Maximilian Scharf, André Uschmajew +2Tensor NetworksNeural Network Optimization

    Institute of Mathematics, University of Augsburg, 86159 Augsburg, Germany · Centre for Advanced Analytics and Predictive Sciences, University of Augsburg, 86159 Augsburg, Germany · Tensor AI Solutions GmbH, 89284 Pfaffenhofen an der Roth, Germany +1

  30. WHALE: A Simple Recipe for Joint Harness-Weight Optimization

    Aug 31, 2026Haechan Kim, Yoonho Lee, Gisang Lee +2Agent HarnessAgentic Optimization

    KRAFTON · KAIST · Stanford University

  31. Deriving Scaling Laws for OpenEuroLLM Models: Learning Rate, Batch Size and Loss

    Aug 28, 2026Niccolò Ajroldi, Diana Alexandra Onutu, Haider Al-Tahan +4BatchScaling Laws

    ELLIS Institute Tübingen · OpenEuroLLM · Eindhoven University of Technology +6

  32. Learned, Then Lost: A Measured Single-Example Counterfactual in Pre-training

    Aug 19, 2026Zachary Speck, Asa ShepardTraining DataBatch

    Arizona State University · Williams College

  33. Sustaining Plasticity via Learnable Wavelet Activations in Continual Learning

    Aug 13, 2026Zeyang Zhang, Tieliang Gong, Junyan Lu +1Replay-Based Continual LearningContinual Learning

    School of Computer Science and Technology, Xi’an Jiaotong University and Institute of Multimedia Knowledge Fusion and Engineering, Xi’an 710049, China

  34. Direct Acceleration of Stochastic Root-Finding Without Variance Reduction and Regularization

    Aug 12, 2026TaeHo Yoon, Nicolas LoizouStochastic ApproximationVariance Reduction

    Department of Applied Mathematics and Statistics & Mathematical Institute for Data Science, Johns Hopkins University

  35. Lifecycle-Optimal Tokenization: Vocabulary Size as a Deployment-Regime-Dependent Infrastructure Parameter

    Aug 11, 2026Rima Mittal, Ankit Gubrani, Satyanarayana KakolluBatchInfrastructure

  36. Batch Size or Negatives? A Selection Rule for Memory-Constrained Recommender Training

    Aug 11, 2026Artyom Sabitov, Daniil Volkov, Alexey ZaytsevReal-World Content Recommendation ProblemBounded-Memory

    Applied AI Institute, Moscow, Russia Risk department

  37. FedA2L: Adaptive layer-wise learning rate adjustment in decentralized federated learning

    Aug 10, 2026Van Truong Vo, Khoa Nguyen, Taehong KimFedavgBatch

    School of Information and Communication Engineering, Chungbuk National University, Cheongju, South Korea

  38. Modular TTT: Rethinking Test-Time Training as Composable Modules

    Aug 7, 2026Bohao Tang, Zhen Qin, Yuqi Pan +3Test-Time TrainingSequence Modeling

    Shanghai Jiao Tong University · Shanghai Innovation Institute · ByteDance Seed

  39. A Rate Separation for Agnostic Direct Sums

    Aug 7, 2026Mihir More, Aritra Das, Debayan GuptaLearnabilityBatch

    Truth Audit Labs

  40. Improving Low-Resolution Face Recognition under Limited Data: How Synthetic Data Generation Can Close the Domain Gap

    Aug 6, 2026Luis S. Luevano, Ünsal Öztürk, Hatef Otroshi Shahreza +2Synthetic-To-Real Domain GapFace Recognition Model

    Idiap Research Institute, Switzerland

  41. Optimal Training-Time Scaling in Gradual Adaptation

    Aug 5, 2026Zonghuan Xu, Krishna HarishBatchTraining Time

    Fudan University Shanghai, China · Lawrence E. Elkins High School Missouri City, Texas, USA

  42. LLaDA MoE v2: Scaling Mixture-of-Experts Diffusion Language Models

    Aug 4, 2026Fengqi Zhu, Shaoxuan Xu, Jingyang Ou +11Mixture-Of-Experts Large Language ModelsMixture-Of-Experts

    Gaoling School of AI, Renmin University of China · Beijing Key Laboratory of Research on Large Models and Intelligent Governance · Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE +2

  43. AOS: Adaptive Optimizer Switching via Training-State Signals for Faster Convergence and Better Generalization

    Aug 3, 2026Alok Kumar Pandey, Umang Chaturvedi, Aatish Rana +1Adaptive OptimizersNeural Network Optimization

  44. Towards joint scaling laws with optimal batch size schedules

    Jul 30, 2026Jiaxiang Li, Zhiqi Bu, Shiyun XuBatchLarge Language Model Training

    Meta · Independent researcher

  45. A Trust-region Framework for Moment Estimation

    Jul 26, 2026Oluwasegun A. SomefunTrust RegionStochastic Gradient Descent

  46. Scale Weight Decay and Train Better

    Jul 26, 2026Anuj ApteWeight DecayBatch

    Global Technology Applied Research, JPMorganChase, New York, NY 10001

  47. Hyperball May Not Be a Free Lunch

    Jul 24, 2026Yihao Xiao, Jialong Sun, Zitian Gao +5BatchAdaptive Optimizers

    1IQuest Research · 4Shenzhen University of Advanced Technology · 2Peking University +2

  48. Cumsum-Composable Phase Transport for Low-Cost Streaming Keyword Spotting

    Jul 22, 2026Mahesh GodavartiKeyword SpottingStreaming

    A Carrot, Inc

  49. Learning Faster without Deeper Networks: A*-Inspired Batch Selection for Efficient CNN Training

    Jul 17, 2026Anxhelo Shehu, Enes Stastoli, Arben CelaResnet-18Convolutional Neural Networks

    AI Laboratory, University Metropolitan Tirana, Tirana, Albania. · Laboratory of Images Signals and Intelligent Systems, ESIEE Paris, Paris, France.

  50. A Learning-Rate-Gated Failure of GRPO in a Small Language and Vision-Language Model Web Agent: A Controlled Null and Its Mechanism

    Jul 14, 2026Chengguang Gan, Zhixi Cai, Yunhao Liang +3Verifiable RewardsBatch

    Independent Researcher · Monash University · University of Chinese Academy of Sciences +2

  51. Same Loss, Same Noise, Opposite Schedules: Noise Structure and Optimizer Normalization Jointly Determine Whether Learning-Rate Cooldown Helps

    Jul 14, 2026Subham Singh, Ashutosh Mishra, Subha RautBatchBatch Normalization

    Department of Mathematics and Statistics, Mississippi State University · Independent Researcher

  52. WSqD: A Horizon-Free Learning Rate Schedule for Large Model Training

    Jul 12, 2026Jianhao Ma, Yuxin ChenBatchWarm Starts

    Department of Statistics and Data Science, University of Pennsylvania

  53. Index SLM Technical Report

    Jul 10, 2026Lusheng Zhang, Shien He, Tianxing Yan +5Small Large Language ModelsInstruction

    Bilibili Index LLM Team

  54. LionVote: Per-Layer Learning Rate Adaptation for Lion

    Jul 10, 2026Kris AtallahBatchLayer-Wise

    New York University, New York, USA · New York University New York, NY, USA

  55. Systematic Evaluation of Learning Rate Scheduling Strategies Across Heterogeneous Architectures

    Jul 9, 2026Hafsa Mateen, Radu Timofte, Dmitry IgnatovBatchNeural Architecture Search

    Computer Vision Lab, CAIDAS & IFI, University of Würzburg, Germany

  56. Linear Attention Architectures: Mechanisms, Trade-offs, and Cross-Layer Routing

    Jul 8, 2026Tommaso Cerruti, Tim Rieder, George Rowlands +2Kimi Delta AttentionLinear Attention

    1ETH Zurich, D-INFK · 2ETH AI Center, ETH Zurich

  57. Optimal Learning Rate Scaling Depends on Data in Deep Scalar Linear Networks

    Jul 8, 2026Yedi Zhang, Peter E. Latham, Leena Chennuru Vankadara +1BatchSequential Scaling

    Gatsby Computational Neuroscience Unit, University College London · Sainsbury Wellcome Centre, University College London

  58. ReCal3R: Reliability-Calibrated Learning Rates for Streaming 3D Reconstruction

    Jul 6, 2026Xinze Li, Yiyuan Wang, Pengxu Chen +43D ReconstructionRecurrent State

    Beijing Normal-Hong Kong Baptist University · Hong Kong Baptist University · Jilin University +2

  59. Directional Curvature from Armijo Backtracking: A Low-Cost Sharpness Probe and a Calibration-Free Learning-Rate Safeguard for Adam

    Jul 4, 2026Ashmitha R, Jörg FrochteAdamSharpness-Aware Minimization

    Department of Artificial Intelligence and Data Science, Sri Ramakrishna Engineering College, Anna University, Coimbatore 641022, Tamil Nadu, India · Interdisciplinary Institute for Applied AI and Data Science Ruhr (AKIS), Department of Electrical Engineering and Computer Science, Bochum University of Applied Sciences, Am Hochschulcampus 1, 44801 Bochum, Germany

  60. The Orthogonalized Read Is a Removable Training Scaffold for Recurrent Memory

    Jul 2, 2026Keston Aquino-MichaelsOrthogonalityMemorization

    No Way Labs