Large Language Model Training

Ask about this topic

Momentum

16 papers in the last four weeks, up 33% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 174

All topics
CardsList

  1. LoGRA: Scaling LLM Reinforcement Learning with Low-Rank Gradient Sketches

    Oct 5, 2026Shaokun Zhang, Yifan Zhang, Jian Hu +5Large Language Model Reinforcement LearningLarge Language Model Training

  2. Scaling Down the Scaling Laws: Parameter Efficiency and Compute-Optimal Training in Resource-Constrained Large Language Models

    Oct 5, 2026Joe DwyerLarge Language Model Training

    ECPI University

  3. Dynamic Minimax Regret Optimization for Robust LLM Post-Training

    Oct 5, 2026Chengbo Zang, Haoyu Dong, Mehmet Kerem Turkcan +3Large Language Model Training

    Department of Electrical Engineering, Columbia University · Department of Computer Science & Engineering, University of California, Santa Cruz · Department of Civil Engineering, Columbia University

  4. FastCI: Efficient GPU-Intensive CI for LLM Training Frameworks

    Oct 1, 2026Tianshuo Qiao, Naiqian Zheng, Xiaopeng Liu +4Large Language Model TrainingCpu-Gpu Hybrid Designs

    Peking University · ByteDance Seed

  5. Leto: Fast In-Place Recovery for LLM Training on Surviving Hardware

    Sep 30, 2026Geon-Woo Kim, Joon Ha Kim, Daehyeok KimLarge Language Model TrainingSurvival Analysis

    The University of Texas at Austin Austin, Texas, USA

  6. dattri-LLM: A Unified and Efficient Library for Training Data Attribution at LLM Scale

    Sep 30, 2026Shixuan Liu, Tongli Zhou, Junwei Deng +2Large Language Model TrainingLarge Language Models(Llms

    University of Illinois at Urbana-Champaign · Google

  7. Trajectory Soup: Pushing the Compute-Scaling Frontier of LLM Mid-training via Diverse Trajectories

    Sep 29, 2026Zhehao Huang, Changxin Tian, Qingyuan Yang +5Large Language Model TrainingPost-Training

    Ling Team, Ant Group · Shanghai Jiao Tong University

  8. Normalize-Then-Precondition: A Hierarchical Approach to Marginal Scale and Interaction Geometry for LLM Training

    Sep 29, 2026Zixuan Gong, Zeyu Gan, Jiaye Teng +1Spectral PreconditioningLarge Language Model Pretraining

    Gaoling School of Artificial Intelligence Renmin University of China · School of Statistics and Management Shanghai University of Finance and Economics

  9. AutoLoCo: Communication Efficient Distributed LLM Training via Adaptive Synchronization

    Sep 29, 2026Pengyu He, Yan Zhang, Ruien Li +1Large Language Model TrainingBatch

    Department of Electrical Engineering, California Institute of Technology · Department of Computer Science and Technology, Tsinghua University · Department of Computer Sciences, University of Wisconsin - Madison

  10. Training LLMs to Verbalize Evaluation Awareness

    Sep 28, 2026Usman Anwar, Sahar Abdelnabi, David KruegerVerbalizationLarge Language Model Evaluation

    University of Cambridge · ELLIS Institute Tübingen MPI-IS, Tübingen AI Center · Mila University of Montreal

  11. Learning to Steer, Steering to See: Unveiling the Geometry of RLVR in Large Language Models via Trainable Vectors

    Sep 28, 2026Yuchen Cai, Ding Cao, Qixiang Yin +10Reinforcement Learning With Verifiable RewardLarge Language Model Training

    USTC · Tencent Hunyuan · BUPT

  12. The Life of a Token: from Words to Bits on the Wire

    Sep 17, 2026Davide Avesani, Pengwenlong Gu, Sotiris Skaperas +1Large Language Model TrainingDeep Learning Architectures

    CEDRIC - ROC · cnam · Cnam

  13. Block Parallelism For Efficient Distributed Long-Context Diffusion Language Model Training

    Sep 16, 2026Tarun Suresh, Pranshu Chaturvedi, Hangoo Kang +4Diffusion Language ModelsEfficient Long-Context Inference

    Stanford University

  14. Preventing Model Collapse: A Fisher-Rao Perspective on the Dynamics of Training with Synthetic Data

    Sep 16, 2026Matteo Marchi, João Pedro Silvestre, Bahman Gharesifard +1Large Language Model TrainingCollapse

    Electrical and Computer Engineering Department, University of California at Los Angeles, Los Angeles, CA 90095 USA · Department of Mathematics and Statistics at Queens University, Kingston, ON, Canada

  15. MiST: Mid-Training LLMs for Cybersecurity

    Sep 16, 2026Oded Ovadia, Elad Ben Zaken, Elad Guttman +1CybersecurityLarge Language Model Training

    Dream

  16. Performance, Efficiency and Collapse -- Advantages and Challenges in Offline Post-training of Code LLMs

    Sep 14, 2026Abhinav Anand, Sanjana Reddy Pachika, Shweta Verma +1Reinforcement Learning Post-TrainingLarge Language Model Training

    TU Darmstadt · Hessian Center for Artificial Intelligence, Darmstadt, Germany · National Research Center for Applied Cybersecurity ATHENE

  17. Break Step: Recursive Training Resonates with Replayed Sampling Noise

    Sep 10, 2026Yangze Liu, Zhongyi HanCollapseLarge Language Model Training

    Shandong University

  18. BRACE: Anchored Bellman-Residual Correction for Stale Critics in Asynchronous RL

    Sep 9, 2026Guanqun Zhao, Zijun Xie, Binbin Zheng +3Gradient StalenessLarge Language Model Training

    Beijing University of Posts and Telecommunications · Baidu Inc. · Peking University +1

  19. Parallelism Strategy Chaining for Fast Training Convergence

    Sep 7, 2026Minchul Kang, Changyong Shin, Younghun Go +4Large Language Model TrainingLanguage Model Perplexity

    Department of Computer Science and Engineering, Korea University, Seoul, South Korea

  20. BASP: Communication-Efficient Batch-Aware Sequence Parallelism for LLM Training

    Sep 2, 2026Bigyan Ghimire, Jon C. CalhounLarge Language Model TrainingSequence Modeling

    Holcombe Department of Electrical and Computer Engineering Clemson University Clemson, USA

  21. Efficiently Estimating Optimal Hyperparameter Scaling Laws through Power-Law Entropy Search

    Sep 1, 2026Zhiliang Chen, Sebastian Ament, David Eriksson +4HyperparameterLarge Language Model Training

    Meta · National University of Singapore · Atomic Machines; work done at Meta

  22. From Rollouts to Recipes: Self-Contained Post-Training for LLMs

    Sep 1, 2026Yifei Li, Lingling Zhang, Muye Huang +3Post-TrainingReinforcement Learning Post-Training

    School of Computer Science and Technology, Xi’an Jiaotong University, China · Shaanxi Province Key Laboratory of Big Data Knowledge Engineering · Zhongguancun Academy, Beijing, China +1

  23. Pre-carved Niches: The Formation Dynamics of Modular Task Partitions in Early LLM Training

    Sep 1, 2026Guangqi Li, Yongxin LiLarge Language Model TrainingCognitive Science

    Zaozhuang University, Zaozhuang, Shandong, China

  24. TrainSDC: Characterizing and Mitigating Silent Data Corruption in Large Language Model Training

    Aug 31, 2026Zhipeng Xia, Haotian Xu, Siyu Yun +4Large Language Model TrainingCorrupted Inputs

    Zhejiang University · Huawei

  25. Information Abundance Paradox: Long-Context Training Undermines Parametric Knowledge

    Aug 12, 2026Arda Uzunoglu, Benjamin Van Durme, Daniel KhashabiEfficient Long-Context InferenceLarge Language Model Training

    Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA

  26. LazyTrain: Limited-resource Allocation toward Zero-waste Yield Optimization in Large Language Model Training

    Aug 12, 2026Xiaojun Wu, Cehao Yang, Honghao Liu +5Large Language Model TrainingGraphics Processing Unit Memory

    1IDEA Research · 2The Hong Kong University of Science and Technology (Guangzhou) · 3DataArcTech Ltd.

  27. SCOUT: Symmetric Consensus Outlier Detection for Failure Localization in LLM Pre-Training

    Aug 11, 2026Zhuang WangLarge Language Models FailOutliers

    Independent Researcher

  28. Can Bayesian Optimization Efficiently Find a Strong Single Expert in Neural Thickets?

    Aug 11, 2026Nigel Bastian Cendra, Abdelhamid Ezzerg, Fernando Julio Cendra +2Large Language Model TrainingGradient-Based Optimization

    University College London · Institut Polytechnique de Paris · University of Oxford

  29. MoE Proxy Models for Low-Cost Failure Reproduction and Diagnosis in LLM RL Post-Training

    Aug 11, 2026Yikai Wang, Chuansai Zhou, Yuhang Zhou +10Large Language Model TrainingReinforcement Learning Post-Training

    State Key Laboratory for Novel Software Technology, Nanjing University, China · Huawei, China

  30. Evaluation-Conditioned Training: Teaching Models to Generalize to Stronger Oversight Regimes

    Aug 10, 2026Alec Harris, Kasey Corra, Archie Chaudhury +1Large Language Model EvaluationLarge Language Model Alignment

    AI Safety Initiative at Georgia Tech · University of Chicago · Independent

  31. Fusion Training for Mathematical Generalization in Large Language Models

    Aug 10, 2026Congfeng Cao, Pengyu Zhang, Jelke BloemLarge Language Model TrainingThoughts

    Institute for Logic, Language and Computation, University of Amsterdam · INDE Lab, University of Amsterdam

  32. Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs

    Aug 9, 2026Mohanad Odema, Gabrielle De Micheli, Dayin Gou +3Large Language Model CompressionLow-Rank Structure

    LG Electronics, North America

  33. Dual-Node NVIDIA DGX Spark over Tailscale: A Remote-Access Testbed for Distributed LLM Training and Cyber-Threat-Intelligence Fine-Tuning

    Aug 7, 2026Vasanth IyerNvidiaLarge Language Model Training

    Department of Computer Science and Digital Technologies Grambling State University

  34. Training Small LLMs as Spatial Multi-Agent Policies

    Aug 2, 2026Yi Mao, Andrew PerraultMulti-Agent Reinforcement LearningLarge Language Model Training

    Department of Computer Science and Engineering, The Ohio State University

  35. Knowing When to Quit: Diagnosing and Training LLMs to Abort Futile Reasoning

    Jul 31, 2026Xinyan Guan, Jiali Zeng, Chunlei Xin +5RefusalsLarge Language Model Training

    1Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · 3Weixin AI, Tencent Inc, China

  36. Towards joint scaling laws with optimal batch size schedules

    Jul 30, 2026Jiaxiang Li, Zhiqi Bu, Shiyun XuBatchLarge Language Model Training

    Meta · Independent researcher

  37. SDO: Structure-Aware Data Organization for Efficient LLM Post-Training

    Jul 29, 2026Jinliang Gao, Ning Yang, Hai Wang +2Large Language Model TrainingPost-Training

    Institute of Automation, Chinese Academy of Sciences · National University of Defense Technology

  38. DHRCL:Training Code LLMs with Dense Hierarchical Rewards and Curriculum Learning

    Jul 29, 2026Shuhang Wang, Ziming Li, Hui ChengLarge Language Model Reinforcement LearningLarge Language Model Training

  39. Cutting AI Datacenter Energy with Reinforcement Learning: Measured Power Control of LLM Training from One GPU to the Fleet

    Jul 27, 2026Eliseo CurcioLarge Language Model TrainingOptimal Power Shutoff

    Advanced Department of Artificial Intelligence and Energy – New York

  40. A Defense of the Quadratic Model

    Jul 23, 2026Alexandru Meterez, Pranav Ajit Nair, Depen Morwani +3Anisotropic Loss LandscapesLarge Language Model Training

    1Kempner Institute at Harvard University · †: Work done while at Harvard. · 2MIT

  41. Training Large Language Models for Self-Explanation Faithfulness

    Jul 23, 2026Yeoktatt Cheah, María Pérez-Ortiz, Noah Y. Siegel +1LLM Reasoning StrategiesLarge Language Model Reinforcement Learning

    Centre for AI, Department of Computer Science University College London · Imperial College London

  42. SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD

    Jul 22, 2026Dongfang Li, Xiaodong Luo, Ruoyu Sun +62Optimization ModelingLarge Language Model Training

  43. Agora: Collective and Permissionless Internet-Scale Pretraining of Large Language Models

    Jul 14, 2026Gil Avraham, Violetta Shevchenko, Hadi Mohaghegh Dolatabadi +9Large Language Model TrainingPretraining

    Pluralis Research

  44. Proxy Exploration and Reusable Guidance: A Modular LLM Post-Training Paradigm via Proxy-Guided Update Signals

    Jul 13, 2026Daocheng Fu, Rong Wu, Yu Yang +7Reinforcement Learning Post-TrainingPost-Training

    Shanghai AI Laboratory · Fudan University · Zhejiang University +2

  45. WSqD: A Horizon-Free Learning Rate Schedule for Large Model Training

    Jul 12, 2026Jianhao Ma, Yuxin ChenBatchWarm Starts

    Department of Statistics and Data Science, University of Pennsylvania

  46. Self-Guided Test-Time Training for Long-Context LLMs

    Jul 10, 2026Xinyu Zhu, Zhe Xu, Xiaohan Wei +10Efficient Long-Context InferenceTest-Time Training

    Meta AI · University of Virginia

  47. Grokking Is Conditional and Fragile: A Fully-Tractable, Multi-Seed Study at 12K Parameters

    Jul 6, 2026Yoshiyuki OotaniGrokkingLarge Language Model Training

    Independent Researcher

  48. Train Smarter, Not Longer: Memorization-Guided Data Reuse for Efficient LLM Training

    Jul 6, 2026Jingwei Zuo, Cong Zeng, Ilyas Chahed +6Large Language Model TrainingRecurrent Model

    Technology Innovation Institute Abu Dhabi, UAE

  49. No Time Like the Present: Agentic Test-Time Training for LLM Agents

    Jul 3, 2026Yanbo Wang, Jinhua Hao, Yuze Shi +2Test-Time TrainingLarge Language Model Training

    Kuaishou Technology

  50. Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale

    Jul 2, 2026Vadym Hadetskyi, Dario Pasquini, Artem SorokinSafety AlignmentCybersecurity

    Cracken AI

  51. SCAPE: Accurate and Efficient LLM Training with Extreme Sparse Communication

    Jul 2, 2026Mingkai Zheng, Junlin Chen, Haotian Xie +1Large Language Model TrainingSparsity

    Rutgers University

  52. PHOENIX: Resilient LLM Training with Hot-Swapping via Zero-Overhead Checkpoint

    Jul 2, 2026Haotian Xie, Junlin Chen, Mingkai Zheng +2Large Language Model TrainingIntermediate Checkpoints

    Rutgers University · George Mason University

  53. FaithMed: Training LLMs For Faithful Evidence-Based Medical Reasoning

    Jul 1, 2026Zhiyun Zhang, Liwen Sun, Xiang Qian +1Clinical Reasoning TrainingLLM Reasoning Strategies

    Carnegie Mellon University · Stanford University School of Medicine · Xlue

  54. Beyond Perplexity: A Behavioral Evaluation Framework for Deployment-Memory Claims in LLM Test-Time Training

    Jul 1, 2026Xiangchen Song, Zhenhao Chen, Lingjing Kong +4Large Language Model MemoryLarge Language Model Evaluation

    Carnegie Mellon University · MBZUAI

  55. AutoTrainess: Teaching Language Models to Improve Language Models Autonomously

    Jun 30, 2026Zhaojian Yu, Penghao Yin, Shuzheng Gao +3Large Language Model TrainingLanguage Modeling

    1Tsinghua University · 2The Chinese University of Hong Kong · 3Simple Agent Lab

  56. One-Step Gradient Delay is Not a Barrier for Large-Scale Asynchronous Pipeline Parallel LLM Pretraining

    Jun 29, 2026Philip Zmushko, Egor Petrov, Nursultan Abdullaev +2Asynchronous ExecutionLarge Language Model Training

    Institute of Science and Technology Austria (ISTA), Austria · Yandex, Russia · Basic Research of Artificial Intelligence Laboratory (BRAIn Lab), Russia +2

  57. From Search to Synthesis: Training LLMs as Zero-Shot Workflow Generators

    Jun 29, 2026Gan Luo, Zihan Qin, Bin Dong +1Large Language Model WorkflowsData Generation Pipelines

    School of Mathematics Science, Peking University

  58. Smooth Scaling Laws Hide Stepwise Token Learning

    Jun 29, 2026Pingjie Wang, Zechen Hu, Peiru Yang +2Scaling LawsLarge Language Model Training

    Dots Studio, Xiaohongshu Inc. · Shanghai Jiao Tong University · Tsinghua university

  59. On the Nonlinearity of Learning Rate Scaling for LLM Training

    Jun 28, 2026Zaiwen Yang, Huaqing Zhang, Jing Xu +1BatchLarge Language Model Training

    Tsinghua University · Tsinghua University, Shanghai Qi Zhi Institute

  60. Mechanism-Driven Monitors for Preemptive Detection of LLM Training Instability

    Jun 26, 2026Ruixuan Huang, Yipei Wang, Wenyi Fang +7Large Language Model TrainingLarge Language Models Fail

    1HKUST · 2Huawei · 3Independent Researcher