Adaptive Learning Rate

Ask about this topic

Latest papers

CardsList

17 papers2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

  1. MuPACE: Learning to Pace Muon for LoRA Fine-Tuning

    Oct 20, 2026Jan Kobiolka, Christian Frey, Josif GrabockaLearning Rate SchedulingAdaptive Learning Rate

    Department of Computer Science, University of Technology Nuremberg, Nuremberg, Germany

  2. SOLAR: A State-Driven Online Learning Rate Scheduler for LLM Pretraining

    Sep 28, 2026Qiulin Shang, Binyu Wang, Yongqi Qiao +3Language Model PretrainingLearning Rate Scheduling

    Peking University · Nanjing University

  3. One Rate Is Not Enough: Adaptive Anisotropic Learning Rates for LoRA Fine-Tuning

    Sep 5, 2026Huiyi Wang, Daijiao Liu, Lina Yao +1LLM Fine-TuningLow-Rank Adaptation

    University of New South Wales

  4. A Trust-region Framework for Moment Estimation

    Jul 26, 2026Oluwasegun A. SomefunLearning Rate SchedulingTrust-Region Optimization

  5. LionVote: Per-Layer Learning Rate Adaptation for Lion

    Jul 10, 2026Kris AtallahDeep Learning OptimizationLayer-Wise Learning Rate Adaptation

    New York University, New York, USA · New York University New York, NY, USA

  6. ReCal3R: Reliability-Calibrated Learning Rates for Streaming 3D Reconstruction

    Jul 6, 2026Xinze Li, Yiyuan Wang, Pengxu Chen +4Streaming 3D Reconstruction3D Scene Representation

    Beijing Normal-Hong Kong Baptist University · Hong Kong Baptist University · Jilin University +2

  7. Alternate loss functions and regression models that achieve robustness to outliers by modulating the learning rate

    Jun 20, 2026Mathew Mithra Noel, Arindam Banerjee, Yug D. Oswal +2Robust RegressionRobust Loss Functions

    aSchool of Electrical Engineering, Vellore Institute of Technology, Vellore, 632 014, Tamil Nadu, India · bErnst and Young GDS, Godrej Waterside, Ring Rd, DP Block, Sector V, Bidhannagar, Kolkata, 700 091, West Bengal, India · cSchool of Computer Science and Engineering, Vellore Institute of Technology, Vellore, 632 014, Tamil Nadu, India

  8. Adaptive Learning Rates with Surrogate Probability for Follow-the-Perturbed-Leader

    Jun 4, 2026Jongyeong Lee, Junya Honda, Shinji Ito +1Multi-Armed BanditsFollow-the-Perturbed-Leader

    Korea Institute of Science and Technology · Kyoto University · RIKEN AIP +2

  9. Physics-Guided Policy Optimization with Self-Distillation

    Jun 2, 2026Ke Wang, Yuning Wu, Haoran Liu +3On-Policy Self-DistillationPolicy Optimization

    Amazon, Arlington, USA · Amazon, New York, USA · Amazon, San Jose, USA +1

  10. Adaptive Sharpness-Aware Minimization with a Polyak-type Step size: A Theory-Grounded Scheduler

    Jun 1, 2026Dimitris Oikonomou, Nicolas LoizouSharpness-Aware MinimizationLearning Rate Scheduling

    Mathematical Institute for Data Science (MINDS), Johns Hopkins University, Baltimore, MD, USA · Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA · Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD, USA

  11. Fine-Tuning Without Forgetting via Loss-Adaptive Learning Rates

    May 19, 2026Parjanya Prajakta Prashant, Jiongli Zhu, Aldan Creo +1Learning Rate SchedulingLLM Fine-Tuning

    University of California San Diego

  12. Learning Rate Engineering: From Coarse Single Parameter to Layered Evolution

    Apr 30, 2026Ming-Hong Yao, Di Wang, Jian Cui +5Deep Learning OptimizationFine-Tuning

    School of AI, Jilin International Sciences University, Changchun, China.

  13. MetaDAT: Generalizable Trajectory Prediction via Meta Pre-training and Data-Adaptive Test-Time Updating

    Mar 10, 2026Yuning Wang, Pu Zhang, Yuan He +2Adaptive Learning RateTrajectory Prediction

    State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, IAIR, Xi’an Jiaotong University, China · KargoBot, China

  14. Fed-ADE: Adaptive Learning Rate for Federated Post-adaptation under Distribution Shift

    Mar 1, 2026Heewon Park, Mugon Joe, Miru Kim +2Adaptive Learning RateFederated Learning

    Department of Electrical and Computer Engineering, Sungkyunkwan University, Republic of Korea · Soongsil University, Seoul, Republic of Korea

  15. Why Do We Need Warm-up? A Theoretical Perspective

    Oct 3, 2025Foivos Alimisis, Rustem Islamov, Aurelien LucchiDeep Learning OptimizationLearning Rate Scheduling

    University of Basel, Switzerland

  16. Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses

    Jun 20, 2024Steffen Dereich, Arnulf Jentzen, Adrian RiekertAdaptive Learning RateStochastic Optimization Convergence

    Institute for Mathematical Stochastics, University of Münster, Germany · School of Data Science and Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen), China · Applied Mathematics: Institute for Analysis and Numerics, University of Münster, Germany

  17. Chaos persists in large-scale multi-agent learning despite adaptive learning rates

    Jun 1, 2023Emmanouil-Vasileios Vlatakis-Gkaragkounis, Konstantinos Metaxas, Lampros Flokas +1Adaptive Learning RateCongestion Games

    University of California, Berkeley. · Columbia University. · Singapore University of Technology and Design. +1