LLM Training

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

31 papers in the last four weeks, up 72% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 226

All topics

Learn-by-Wire Training Control Governance: Bounded Autonomous Training Under Stress for Stability and Efficiency
May 18, 2026Anis RadianisEfficient Language Model Training LLM Training
Research Scientist at Qluon Inc. Delaware, United States
TierCheck: Tiered Checkpointing for Fault Tolerance in Large Language Model Training
May 18, 2026Shujie Han, Feng Jiang, Patrick P. C. Lee +5Fault-Tolerant Distributed Training Distributed Training
Northwestern Polytechnical University · The Chinese University of Hong Kong · National University of Defense Technology
Charon: A Unified and Fine-Grained Simulator for Large-Scale LLM Training and Inference
May 16, 2026Mengtian Yang, Zhekun Zhang, Mingheng Wu +3LLM Inference LLM Inference Acceleration
ByteDance Seed · University of Texas at Austin
Fully Open Meditron: An Auditable Pipeline for Clinical LLMs
May 15, 2026Xavier Theimer-Lienhard, Mushtaha El-Amin, Fay Elhassan +5Healthcare LLM Auditing
LiGHT, EPFL
Runtime-Orchestrated Second-Order Optimization for Scalable LLM Training
May 15, 2026Yishun Lu, Junhao Zhang, Zeyu Yang +1Second-Order Optimization LLM Training
Engineering Science Department University of Oxford Oxford, United Kingdom
A Few GPUs, A Whole Lotta Scale: Faithful LLM Training Emulation with PrismLLM
May 15, 2026Shaoke Xi, ChonLam Lao, Boyi Jia +11High-Performance Computing Distributed Training
Alibaba Group · Harvard University · Shanghai Jiao Tong University +1
A Theory of Training Profit-Optimal LLMs
May 14, 2026Sophie Hao, William MerrillLLM Training
Boston University · Allen Institute for AI
FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale
May 14, 2026Runyuan He, Qiuyang Mang, Shang Zhou +14Synthetic Benchmark Generation Synthetic Data Generation for Language Models
UC Berkeley · UC San Diego · University of Washington +4
Always Learning, Always Mixing: Efficient and Simple Data Mixing All The Time
May 13, 2026Michael Y. Hu, Apurva Gandhi, Kyunghyun Cho +2Language Model Pretraining Continual Learning for LLMs
New York University · Carnegie Mellon University · Microsoft
Training Large Language Models to Predict Clinical Events
May 12, 2026Benjamin Turtel, Paul Wilczewski, Kris SkotheimHealthcare Language Model Calibration
Lightning Rod Labs
Uncovering Symmetry Transfer in Large Language Models via Layer-Peeled Optimization
May 12, 2026Zhehang Du, Hangfeng He, Weijie SuRepresentation Geometry LLM Training
The Wharton School, University of Pennsylvania · University of Rochester
Scaling Laws for Mixture Pretraining Under Data Constraints
May 12, 2026Anastasiia Sedova, Skyler Seto, Natalie Schluter +1Language Model Pretraining Language Model Scaling Laws
Apple · ITU
Overtrained, Not Misaligned
May 12, 2026Joel Schreiber, Ariel GoldsteinEarly Stopping LLM Fine-Tuning
Hebrew University of Jerusalem
DynaTrain: Fast Online Parallelism Switching for Elastic LLM Training
May 12, 2026Yuanqing Wang, Yuchen Zhang, Hao Lin +9Efficient Language Model Training Distributed Training
Peking University · Infinigence AI · Institute of Computing Technology, CAS +2
HEBATRON: A Hebrew-Specialized Open-Weight Mixture-of-Experts Language Model
May 11, 2026Noam Kayzer, Dan Revital, Ori Bar Joseph +10Mixture-of-Experts Language Models Open-Weight Language Models
PwC Next
Position: Let's Develop Data Probes to Fundamentally Understand How Data Affects LLM Performance
May 11, 2026Shiqiang Wang, Herbert Woisetschläger, Hans Arno Jacobsen +1Synthetic Data Generation LLM Training
Department of Computer Science, University of Exeter, UK · Technical University of Munich, Germany · Department of Electrical and Computer Engineering, University of Toronto, Canada +1
Speech-based Psychological Crisis Assessment using LLMs
May 11, 2026Terumi Chiba, Yang Luo, Ziyun Cui +2Mental Health Speech Language Models
Tsinghua University, Beijing, China · Peking University Huilongguan Clinical Medical School, Beijing, China · WHO Collaborating Centre for Research and Training in Suicide Prevention
BatchWeave: A Consistent Object-Store-Native Data Plane for Large Foundation Model Training
May 11, 2026Ting Sun, Junjie Zhang, Xiao Yan +7Fault-Tolerant Distributed Training Distributed Training
Lionrock AI Lab, China Merchants Group, Hong Kong, China · Wuhan University, Wuhan, China · The Chinese University of Hong Kong, Shenzhen, China
From Detection to Recovery: Operational Analysis on LLM Pre-training with 504 GPUs
May 10, 2026Daemyung Kang, Eunjin Hwang, Hanjeong Lee +11Fault-Tolerant Distributed Training Distributed Training
Lablup Inc.∗
Navigating LLM Valley: From AdamW to Memory-Efficient and Matrix-Based Optimizers
May 9, 2026Aditya RanganathDeep Learning Optimization Memory-Efficient Optimization
Center for Applied Scientific Computing Lawrence Livermore National Laboratory Livermore, CA, USA
Predicting Large Model Test Losses with a Noisy Quadratic System
May 9, 2026Chuning Li, Chris J. MaddisonLanguage Model Scaling Laws Efficient Language Model Training
University of Toronto · Vector Institute
XPERT: Expert Knowledge Transfer for Effective Training of Language Models
May 9, 2026Chang Liu, Boyu Shi, Xu Yang +1Mixture-of-Experts Language Models Knowledge Augmentation for Language Models
School of Computer Science and Engineering, Southeast University, Nanjing, China · Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China.
Mid-Training with Self-Generated Data Improves Reinforcement Learning in Language Models
May 8, 2026Aswin RRV, Jacob Dineen, Divij Handa +4Synthetic Data Generation RL for Language Model Reasoning
Arizona State University · Google Cloud AI Research · Google DeepMind
Toward Privileged Foundation Models:LUPI for Accelerated and Improved Learning
May 8, 2026Xueying Ding, Leman AkogluTabular Foundation Models Learning Using Privileged Information
Carnegie Mellon University
Features have life history. And we should care
May 7, 2026Philipp Stecher, Sandro Radovanović, Vlasta Sikimić +1Representation Learning LLM Interpretability
University of Tübingen · University of Belgrade · Eindhoven University of Technology
Normalized Architectures are Natively 4-Bit
May 7, 2026Maxim Fishman, Brian Chmiel, Ron Banner +2LLM Quantization Neural Representation Geometry
∧NVIDIA · ◦Department of Electrical and Computer Engineering - Technion, Haifa, Israel
Spectral Lens: Activation and Gradient Spectra as Diagnostics of LLM Optimization
May 7, 2026Andy Zeyi Liu, Elliot Paquette, John SousRepresentation Learning Decoder-Only Language Models
Department of Applied Physics, Yale University, New Haven, CT 06511, United States · Department of Mathematics and Statistics, McGill University · Department of Applied Physics, Yale University, New Haven, Connecticut 06511, USA +1
When2Speak: A Dataset for Temporal Participation and Turn-Taking in Multi-Party Conversations for Large Language Models
May 7, 2026Vihaan Nama, Shreya Mendi, Zian Ye +1Synthetic Data Generation Synthetic Data Generation for Language Models
Pratt School of Engineering Duke University
FinRAG-12B: A Production-Validated Recipe for Grounded Question Answering in Banking
May 6, 2026Denys Katerenchuk, Pablo Duboue, Keelan Evanini +6Financial Services LLM Grounding
Kasisto, New York, NY · Textualization, Vancouver, Canada · NBME, Philadelphia, PA
Perturbation is All You Need for Extrapolating Language Models
May 5, 2026Zetai Cen, Jin Zhu, Xinwei Shen +1OOD Generalization Autoregressive Language Modeling
School of Mathematics, University of Bristol · School of Mathematics, University of Birmingham · Department of Statistics, University of Washington +1

May 18, 2026, cs.AIJ/K move · Enter open · S save

Learn-by-Wire Training Control Governance: Bounded Autonomous Training Under Stress for Stability and Efficiency

Anis Radianis

Research Scientist at Qluon Inc. Delaware, United States

Efficient Language Model Training LLM Training

Open paper AskPDF arXiv

Modern language-model training is increasingly exposed to instability, degraded runs, and wasted compute, especially under aggressive learning-rate, scale, and runtime-stress conditions. This paper introduces Learn-by-Wire Guard (LBW-Guard), a bounded autonomous training-control governance layer that operates above AdamW. Rather than replacing the optimizer update rule, LBW-Guard observes training telemetry, interprets instability-sensitive regimes, and applies bounded control to optimizer execution while preserving fixed training objectives. We evaluate LBW-Guard in a Qwen2.5-centered stress-and-robustness suite using WikiText-103, with Qwen2.5-7B as the empirical anchor, model-size comparisons against Qwen2.5-3B and Qwen2.5-14B, learning-rate stress tests, gradient-clipping baselines, and a no-LoRA TinyLlama-1B full-parameter sanity check. In the 7B reference setting, LBW-Guard reduces final perplexity from 13.21 to 10.74, an 18.7% improvement, while reducing end-to-end time from 392.54s to 357.02s, a 1.10x speedup. Under stronger learning-rate stress, AdamW degrades to 1885.24 final perplexity at LR=3e-3 and 659.76 at LR=1e-3, whereas LBW-Guard remains trainable at 11.57 and 10.33, respectively. Gradient-clipping baselines do not reproduce this effect. These results support a scoped systems conclusion that stability-sensitive LLM training can benefit from a governance plane above the optimizer. LBW-Guard provides evidence that bounded runtime control can preserve productive compute under stress while remaining distinct from optimizer replacement and local gradient suppression.

Momentum

Related topics

Latest papers 226