LLM Training

LLM: Large Language Model

Ask about this topic

Momentum

31 papers in the last four weeks, up 72% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 226

All topics
CardsList

  1. Fault-tolerant foundation models

    Oct 7, 2026Trevor McCourt, Ila R. Fiete, Isaac L. ChuangLLM TrainingLanguage Model Scaling Laws

    Department of Electrical Engineering and Computer Science, MIT · Department of Brain and Cognitive Sciences, McGovern Institute, MIT · Department of Physics, MIT

  2. Algorithmic Scratchpads and Curriculum Staging for Arithmetic Reasoning in Tiny Transformers

    Oct 6, 2026Sourabh KasliwalNumerical Reasoning in Language ModelsNeural Algorithmic Reasoning

  3. Dynamic Minimax Regret Optimization for Robust LLM Post-Training

    Oct 5, 2026Chengbo Zang, Haoyu Dong, Mehmet Kerem Turkcan +3Dynamic Regret MinimizationDistributionally Robust Optimization

    Department of Electrical Engineering, Columbia University · Department of Computer Science & Engineering, University of California, Santa Cruz · Department of Civil Engineering, Columbia University

  4. ORCA: The Annealed Spectral Conditioning Optimizer for Faster, Better LLM Training

    Oct 5, 2026Yuanshi Liu, Boyuan Jiang, Liang Hou +4Spectral RegularizationDeep Learning Optimization

    School of Intelligence Science and Technology, Peking University · Kling Team

  5. CIPHER-MoE: Balancing Efficiency and Routing Fidelity in Trillion-Scale MoE Training

    Oct 5, 2026Jing Li, Jian Meng, Yingmeng Gao +8Expert Load BalancingExpert Routing

    Tongji University · Cornell University · Harbin Institute of Technology, Shenzhen +1

  6. Invent a Dataset: Measuring dataset generation abilities with zero seed

    Oct 1, 2026Shivalika Singh, Andrija Djurisic, Gbemileke Onilude +2Synthetic Data GenerationSynthetic Data Generation for Language Models

    Adaption

  7. AF-Muon: An AdamW-Free Muon Optimizer for Tied-Embedding Models

    Oct 1, 2026Arash Lagzian, Paniz Halvachi, Junming Zhang +2Memory-Efficient OptimizationMuon Optimizer

    National University of Singapore, Singapore · Independent Researcher · LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai, China

  8. Persistent Depth Ordering amid Shifting Block-Bypass Responses in Language Model Pretraining

    Oct 1, 2026Shengye Tao, Yinzhu Cheng, Haihua XieLLM InterpretabilityCausal Interventions in Language Models

    Beijing University of Civil Engineering and Architecture · Beijing Institute of Mathematical Sciences and Applications (BIMSA) · Institute of Statistics and Big Data, Renmin University of China

  9. Leto: Fast In-Place Recovery for LLM Training on Surviving Hardware

    Sep 30, 2026Geon-Woo Kim, Joon Ha Kim, Daehyeok KimFault-Tolerant Distributed TrainingLLM Training

    The University of Texas at Austin Austin, Texas, USA

  10. Closing the Loop: Practical Training Recipes for Looped Language Models

    Sep 30, 2026Andrei Marchenko, Viacheslav Bezrukov, Oleg Kashurin +5Efficient Language Model TrainingLLM Training

    RND NLP, DAIMLD, Russian Federation

  11. Forking: Sudden Overfitting Under Replay

    Sep 30, 2026Shanbin Yu, Shaoyang Guo, Haoran Zhao +2Neural Network GeneralizationLLM Training

    MetaCircle (元环智能) · Tsinghua University · Peking University +1

  12. MASCRDM: Multi-Agent System for Compliance Risk Detection and Mitigation in Training Process of Large Language Models

    Sep 30, 2026Yan Zhang, Chuming Wei, Ruien Li +3LLM SafetyLanguage Model Bias Evaluation

    Department of Computer Science and Technology, Tsinghua University · Academy of Artificial Intelligence and Advanced Technology, Xi’an Jiaotong-Liverpool University · Department of Computer Sciences, University of Wisconsin–Madison +1

  13. Multi-LLM Collaborative Alignment via Stackelberg Games

    Sep 30, 2026Christina Hahn, Shangbin Feng, Dean Light +3RL for Language ModelsLLM Alignment

    University of Washington · Amazon · University of British Columbia

  14. Trajectory Soup: Pushing the Compute-Scaling Frontier of LLM Mid-training via Diverse Trajectories

    Sep 29, 2026Zhehao Huang, Changxin Tian, Qingyuan Yang +5Language Model ScalingModel Merging

    Ling Team, Ant Group · Shanghai Jiao Tong University

  15. ER-JEPA: Experience Replay Improves Joint-Embedding Predictive Learning in Language Models

    Sep 29, 2026Jingnan Pu, Zi-En Fan, Feng LianMemory-Augmented Language ModelsJoint-Embedding Predictive Architecture

    School of Automation Science and Technology, Xi’an Jiaotong University No. 28, West Xianning Road, Xi’an, Shaanxi 710049, China

  16. Normalize-Then-Precondition: A Hierarchical Approach to Marginal Scale and Interaction Geometry for LLM Training

    Sep 29, 2026Zixuan Gong, Zeyu Gan, Jiaye Teng +1Language Model PretrainingMatrix Optimization

    Gaoling School of Artificial Intelligence Renmin University of China · School of Statistics and Management Shanghai University of Finance and Economics

  17. AutoLoCo: Communication Efficient Distributed LLM Training via Adaptive Synchronization

    Sep 29, 2026Pengyu He, Yan Zhang, Ruien Li +1Communication-Efficient Distributed TrainingDistributed Training

    Department of Electrical Engineering, California Institute of Technology · Department of Computer Science and Technology, Tsinghua University · Department of Computer Sciences, University of Wisconsin - Madison

  18. Principled Thoughts for Latent Recursive LLM Systems

    Sep 28, 2026Fahd Seddik, Fatemeh FardContinuous Latent ReasoningMulti-Agent LLMs

    FARD Lab, University of British Columbia, Okanagan, Canada

  19. Telescopic Language Models

    Sep 28, 2026Zhilin Guo, Boqiao Zhang, Hakan Aktas +14Language ModelingEfficient Language Model Inference

    University of Cambridge · University of British Columbia · Google

  20. MeqMuon: Matrix-Equilibrating Muon for LLM Pretraining

    Sep 28, 2026Chang-Wei Shi, Xu Wang, Wu-Jun LiLanguage Model PretrainingEfficient Language Model Training

    National Key Laboratory for Novel Software Technology, School of Computer Science, Nanjing University, P. R. China

  21. Nereus: Adaptive Parallelism for LLM Post-Training

    Sep 28, 2026Songlin Jiang, Tuo Shi, Sitong Zhang +3LLM Inference SchedulingRL Post-Training

    Aalto University · Shenzhen University of Advanced Technology · Zhejiang University

  22. Commutator Memory: Sparse, Path-Local Reading and Steering in Language Models

    Sep 28, 2026John SweeneyLLM Training

    Sideplane AI

  23. Direct Self-Evolving Optimization: Evolving LLMs without Challenger Training

    Sep 28, 2026Yuyang Deng, Yu Wang, Jiayun WangSelf-Training for Language ModelsLarge Language Model-Guided Optimization

    Accenture, Center for Advanced AI · Georgia Institute of Technology

  24. Training and Inference Dynamics of PLDR-LLMs: Row-Map Collapse, Renormalization, and Predictive Reduction

    Sep 28, 2026Burc GokdenLLM InferenceLanguage Model Scaling Laws

    Fromthesky Research Labs LLC, Oregon, USA

  25. Technical Manual for Toolkit for Confidence-Corpus Consistency, Corpus Absorption and Rule Learning via Fine-Tuning on a Fabricated Corpus

    Sep 23, 2026José Luciano Verçosa Marques, Frederico Jorge Heitmann, Daniel Omar Perez +3LLM Training

    Center for Electric Mobility Research (CEMOBE) / Power Electronics Laboratories (LEPO), University of Campinas (Unicamp) · Institute of Computing (IC), University of Campinas (Unicamp) · Center for Logic, Epistemology and History of Science (CLE), University of Campinas (Unicamp) +1

  26. Direct Optimization of Generators for Search in Automated Theorem Proving

    Sep 22, 2026Adam Ousherovitch, Ambuj TewariInference-Time SearchAutomated Theorem Proving

    Department of Statistics University of Michigan

  27. The Life of a Token: from Words to Bits on the Wire

    Sep 17, 2026Davide Avesani, Pengwenlong Gu, Sotiris Skaperas +1Communication-Efficient Distributed TrainingHigh-Performance Computing

    CEDRIC - ROC · cnam · Cnam

  28. Learn Your Own Thoughts: Abstract Token Curriculum

    Sep 17, 2026Khashayar Gatmiry, Avrajit Ghosh, Parsa Mirtaheri +4CoT ReasoningCurriculum Learning

    UC San Diego · Google DeepMind and UC Berkeley

  29. Preventing Model Collapse: A Fisher-Rao Perspective on the Dynamics of Training with Synthetic Data

    Sep 16, 2026Matteo Marchi, João Pedro Silvestre, Bahman Gharesifard +1Synthetic Data PretrainingModel Collapse

    Electrical and Computer Engineering Department, University of California at Los Angeles, Los Angeles, CA 90095 USA · Department of Mathematics and Statistics at Queens University, Kingston, ON, Canada

  30. MiST: Mid-Training LLMs for Cybersecurity

    Sep 16, 2026Oded Ovadia, Elad Ben Zaken, Elad Guttman +1Synthetic Data PretrainingDomain-Adaptive Pretraining

    Dream