LLM Training

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

31 papers in the last four weeks, up 72% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 226

All topics

GraphEcho: Structural Redundancy and Evidence Provenance in LLM Graph Agents
Sep 15, 2026Sikun Wang, Yixi Zhou, Lei Fan +1LLM Reasoning with Graphs LLM Agent Evaluation
Tokyo University of Science · Hong Kong Baptist University · University of Michigan +1
MoARa: Module-Aware Rank Allocation and Structure-Preserving Decomposition for Low-Rank LLM Pre-training
Sep 14, 2026Keunyoung Kim, Nojun KwakLanguage Model Pretraining Low-Rank Approximation
Seoul National University
Characterizing Job Power Elasticity for Power-Flexible AI Training
Sep 12, 2026Philip Colangelo, Charles Dawson, Shayan Sengupta +2Energy-Efficient ML Efficient Language Model Training
Emerald AI
Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data
Sep 10, 2026Atindra Jha, Margaret Li, Jure Leskovec +2Mixture of Experts Sparse Mixture-of-Experts
Stanford University · Paul G. Allen School of Computer Science, University of Washington
Rebalancing Token Importance in Language Models with TF-IDF Weighted Cross-Entropy Loss
Sep 10, 2026Zhijian Li, Stefan Larson, Kevin LeachMemorization in Language Models LLM Fine-Tuning
University of Southern California, Los Angeles · Vanderbilt University, Nashville
Parallelism Strategy Chaining for Fast Training Convergence
Sep 7, 2026Minchul Kang, Changyong Shin, Younghun Go +4Efficient Language Model Training LLM Training
Department of Computer Science and Engineering, Korea University, Seoul, South Korea
Language, Language Models, and What We're Talking About
Sep 3, 2026Malvina NissimLanguage Modeling LLM Training
CLCG, University of Groningen, The Netherlands
BASP: Communication-Efficient Batch-Aware Sequence Parallelism for LLM Training
Sep 2, 2026Bigyan Ghimire, Jon C. CalhounCommunication-Efficient Distributed Training Long-Context Language Modeling
Holcombe Department of Electrical and Computer Engineering Clemson University Clemson, USA
Knowledge Distillation During Mid-Training Favors Reasoning over Factual Recall
Sep 1, 2026Jacqueline He, Howard Yen, Shuyue Stella Li +9Language Model Distillation Factual Knowledge in Language Models
Meta AI · University of Washington · Princeton University
Efficiently Estimating Optimal Hyperparameter Scaling Laws through Power-Law Entropy Search
Sep 1, 2026Zhiliang Chen, Sebastian Ament, David Eriksson +4Bayesian Optimization Language Model Scaling Laws
Meta · National University of Singapore · Atomic Machines; work done at Meta
When Tokenization is Secretly Output Supervision
Sep 1, 2026Tanja Baeumel, Josef van Genabith, Simon OstermannLLM Training
1German Research Center for AI (DFKI) · Saarland University · Center for European Research in Trusted AI (CERTAIN)
Lagged Coupling: Internal Representations Become Readable Before They Become Causal
Sep 1, 2026Xining XunCausal Interpretability Causal Interventions in Language Models
Tsingjiao Information Science (Beijing) Co., Ltd.
Instella-MoE Technical Report
Sep 1, 2026Jiang Liu, Sudhanshu Ranjan, Prakamya Mishra +10Mixture-of-Experts Language Models Language Model Post-Training
TrainSDC: Characterizing and Mitigating Silent Data Corruption in Large Language Model Training
Aug 31, 2026Zhipeng Xia, Haotian Xu, Siyu Yun +4Fault-Tolerant Distributed Training Language Model Robustness
Zhejiang University · Huawei
On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability
Aug 31, 2026Zihan Qiu, Zekun Wang, Xiao Li +33Efficient Language Model Training Sparse Attention
Qwen Team
Context Staircase: Signature-Aligned Dynamics of Token Embeddings under Small Initialization
Aug 31, 2026Junjie Yao, Liangkai Hang, Zhi-Qin John XuLanguage Model Pretraining Word Embeddings
School of Mathematical Sciences Shanghai Jiao Tong University Shanghai 200240, China · Institute of Natural Sciences, School of Mathematical Sciences Shanghai Jiao Tong University Shanghai 200240, China
CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents
Aug 31, 2026Amir Saeidi, Zehua Zhang, Rishitosh Singh +6LLM Agents LLM Agent Training
Arizona State University · Cisco Research
DataFoundry: Evolving Data Preparators via Recursive Self-Improvement
Aug 30, 2026Cehao Yang, Xiaojun Wu, Xueyuan Lin +4Training Data Curation Domain Adaptation for LLMs
1IDEA Research, International Digital Economy Academy · 2Hong Kong University of Science and Technology (Guangzhou) · 3DataArc Tech Ltd.
DFM Mimir v1: An Open HRM Delivering Frontier Performance at 1B Parameters Using Only Permissible Post-Training Data
Aug 13, 2026Peter Schneider-Kamp, Jacob Nielsen, Gianluca Barmina +2Language Modeling LLM Training
University of Southern Denmark · Ordbogen A/S · Aarhus University
Mixture of Training: Recombining Small-Scale Scaffolded Pretraining Runs into a Larger Language Model
Aug 13, 2026Mohammed Sabry, Sean Augenstein, Keith Rush +1Language Model Pretraining Efficient Language Model Training
Google, Mountain View, CA, USA · School of Computing, Dublin City University, Dublin, Ireland.
Information Abundance Paradox: Long-Context Training Undermines Parametric Knowledge
Aug 12, 2026Arda Uzunoglu, Benjamin Van Durme, Daniel KhashabiLong-Context Language Modeling Factual Knowledge in Language Models
Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA
LazyTrain: Limited-resource Allocation toward Zero-waste Yield Optimization in Large Language Model Training
Aug 12, 2026Xiaojun Wu, Cehao Yang, Honghao Liu +5Efficient Language Model Training LLM Training
1IDEA Research · 2The Hong Kong University of Science and Technology (Guangzhou) · 3DataArcTech Ltd.
Decodable But Not Detachable: Training Data Granularity Determines Parametric Modularity in Large Language Models
Aug 10, 2026Marcus Armstrong, Navid Ayoobi, Arjun MukherjeeLLM Interpretability Circuits in Language Models
Department of Computer Science University of Houston Houston, TX 77204
Fusion Training for Mathematical Generalization in Large Language Models
Aug 10, 2026Congfeng Cao, Pengyu Zhang, Jelke BloemLLM Training
Institute for Logic, Language and Computation, University of Amsterdam · INDE Lab, University of Amsterdam
Instability of LLM Pre-Pretraining: It Doesn't Always Help. An Investigation on Multiple Languages
Aug 9, 2026Sofiia Riazhskykh, Nam Luu, Ondřej BojarLanguage Model Pretraining Linguistic Typology
Charles University, Faculty of Mathematics and Physics
Scaling Inherently Interpretable Language Models
Aug 6, 2026Guide Labs Team, Andreas Madsen, Aya Abdelsalam Ismail +7Language Model Steering LLM Interpretability
Guide Labs Team
OctoLong: Mid-Training On Cross-Repository Code Contexts Enhances Long-Context Modeling
Aug 5, 2026Indraneil Paul, Falko Helm, Goran Glavaš +1Software Engineering Long-Context Language Modeling
UKP Lab & Center for Applied Cybersecurity ATHENE, TU Darmstadt · CAIDAS, University of Würzburg
Training nGPT
Aug 2, 2026Ilya Loshchilov, Boris GinsburgHyperspherical Representation Learning Efficient Language Model Training
NVIDIA
AdaMTP: An Adaptive Training Paradigm for Multi-Token Prediction
Aug 1, 2026Ziqiang Cui, Han Shi, Bowei He +8Multi-Token Prediction LLM Inference Acceleration
City University of Hong Kong · Huawei Technologies · Mohamed bin Zayed University of Artificial Intelligence +2
Beyond Rephrasing: Book-Level Organization Improves Synthetic Textbook Data for Mid-Training
Jul 30, 2026Jiawen Tao, Miao Peng, Yaoming Li +7Language Model Pretraining Synthetic Data Pretraining
Hunyuan Team, Tencent · Peking University · The Hong Kong University of Science and Technology (Guangzhou)

Sep 15, 2026, cs.AIJ/K move · Enter open · S save

GraphEcho: Structural Redundancy and Evidence Provenance in LLM Graph Agents

Sikun Wang, Yixi Zhou, Lei Fan, Fan Zhang

Tokyo University of Science · Hong Kong Baptist University · University of Michigan+1

LLM Reasoning with Graphs LLM Agent Evaluation AI Agent Benchmarks LLM Training

Open paper AskPDF arXiv

A large language model (LLM) agent can follow more graph paths without acquiring more independent evidence. GraphEcho tests whether agents mistake these repeated encounters for additional corroboration. The benchmark varies path counts and evidential origins while holding evidence content fixed, and evaluates both judgments and active exploration. Controlled synthetic experiments reveal model-dependent judgment shifts, but redundant supporting paths increase the share of repeated walks across all evaluated frozen agents. Provenance-aware post-training (PAPT) reduces revisits and improves synthetic accuracy, yet covers fewer distinct sources. On scientific claims, it continues to reduce repetition while accuracy declines. These findings expose a gap between efficient exploration and effective evidence use: an agent can learn to stop repeating itself while overlooking information it needs. GraphEcho provides a controlled way to evaluate both what graph agents conclude and whether their exploration reaches distinct evidential sources.

Momentum

Related topics

Latest papers 226