Continual Learning for LLMs

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

21 papers in the last four weeks, up 250% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 151

All topics

Yesterday's Shield, Today's Spear: A Self-Evolving Safety Guardrail in Production
Aug 9, 2026Cong Ming, Jingyi Chen, Bin Liu +4Continual Learning for LLMs LLM Guardrails
University of Science and Technology of China · Shenzhen University · Sangfor Technologies
Omega-S: A Functional Resilience Index for LLM Fine-Tuning
Aug 4, 2026Alberto AcedoContinual Learning for LLMs LLM Fine-Tuning
Biome Makers Inc.
Toward Plasticity-Preserving KL Regularization for Capability Retention in LLM Reinforcement Learning
Aug 3, 2026Li Wang, Xiaodong Lu, Xiaohan Wang +4Continual Learning for LLMs KL-Regularized RL
1Meituan · 2Nanyang Technological University
Learning What to Remember: Test-Time Training via Context Distillation
Aug 3, 2026Zixuan Wang, Xingyu Dang, Rui-Jie Zhu +4Continual Learning for LLMs Long-Context Language Modeling
Princeton University · UC Santa Cruz · Carnegie Mellon University +1
RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection
Aug 3, 2026Shicheng Xu, Liang Pang, Liyi Chen +7Retrieval-Augmented Generation Continual Learning for LLMs
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences · Xiaohongshu Inc.
Unleashing the Potential of Large Language Models: A Blueprint for Real-Time, Enterprise-Ready Deployments
Aug 1, 2026Muhammad Faizan Raza, Shuo, Yang +2Retrieval-Augmented Generation Continual Learning for LLMs
Luna · ¹ Penn State Great Valley, Malvern, PA 19355, USA · ² Penn State Brandywine, Media, PA 19063, USA +1
The Parts Are Greater Than the Sum: Automated Task Sequencing for Efficient Training of Multi-Policy LLMs
Jul 31, 2026Jiajia Tang, Sizhe Yuen, Francisco Gomez Medina +2Continual Learning for LLMs Low-Rank Adaptation
The Alan Turing Institute, London, UK · King’s College London, London, UK · University of Southampton, Southampton, UK
PARALLEL: A Prefrontal-Aligned Reinforcement inspired Approach for Language-Model Learning under Explicit Limits
Jul 31, 2026Namkyung Yoon, Sanghong Kim, Hwangnam KimContinual Learning for LLMs LLM Fine-Tuning
School of Electrical Engineering, Korea University Seoul 02841, Republic of Korea
CACHE-UK: A Stability-Aware Memory Editor for Sequentially Updated Quantized LLMs in Finance
Jul 30, 2026Anubhav Lakra, Yue FengKnowledge Editing Continual Learning for LLMs
University of Birmingham, Birmingham, United Kingdom
ForgetBench: Benchmarking Forgetting Dynamics of Long-Term Parametric Memory in Language Models
Jul 29, 2026Ruxi Gu, Zhenliang Zhang, Wei WangKnowledge Editing Continual Learning for LLMs
Department of Automation, University of Science and Technology of China, Hefei, China · State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing, China · School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, China.
From Execution to Capability: Scientific Experience Consolidation via Procedural Knowledge Synthesis
Jul 27, 2026Liwei Dong, Jiahao Zhao, Nan XuContinual Learning for LLMs Scientific Code Generation
ScienceOne AI
Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning
Jul 26, 2026Reza Rahimi Azghan, Gautham Krishna Gudur, Giulia Pedrielli +2Continual Learning for LLMs Incremental Learning
College of Health Solutions, Arizona State University, Phoenix, Arizona, USA · Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, Texas, USA · School of Computing and Augmented Intelligence, Arizona State University, Tempe, Arizona, USA +1
The Blessing of Dimensionality: How Near-Orthogonality in High-Dimensional Spaces Explains Temporal Portability
Jul 22, 2026Abigail Woodring, Adrian Chan, Rana Muhammad Shahroz Khan +3Continual Learning for LLMs Low-Rank Adaptation
Department of Electrical and Computer Engineering NC State University, Raleigh, NC 27606, USA · Independent Researcher · Department of Computer Science University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
Rethinking Transfer in Continual Learning: A Replay-Based Realisation
Jul 17, 2026Yang Meng, Zhenya Liu, Zhuokai Zhao +1Continual Learning for LLMs Experience Replay
Department of Computer Science, University of Chicago
Can a Language Model Learn Facts Continually in Its Weights?
Jul 13, 2026Charles O'Neill, Max Kirkby, Jonathon Liu +1Knowledge Editing Continual Learning for LLMs
Baseten
When Does Continual Learning Require Learning
Jul 8, 2026Anne Harrington, Nayan Saxena, Michael Murphy +7Continual Learning for LLMs Concept Drift
1UC Berkeley · 2Independent · 3Capital Fund Management
MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning
Jul 8, 2026Ruilin Tong, Dong GongContinual Learning for LLMs Memory-Augmented Language Models
University of New South Wales (UNSW Sydney)
MemDefrag: Latent Memory Defragmentation for Large Language Models
Jul 7, 2026Ruiyi Yan, Zhuoyuan Mao, Yiwen GuoContinual Learning for LLMs Memory-Augmented Language Models
Kyoto University · LIGHTSPEED · Independent Researcher
ReCoLoRA: Spectrum-Aware Recursive Consolidation for Continual LLM Fine-Tuning
Jul 4, 2026Wentao LuContinual Learning for LLMs Low-Rank Adaptation
Statistically Meaningful Geometry (SMG) Beyond the Euclidean Paradigm, with Application to Generative AI
Jul 3, 2026Bing Cheng, Yi-Shuai Niu, Howell Tong +1Neural Network Generalization Information Geometry
Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China · AMSS Center for Forecasting Science, Chinese Academy of Sciences, Beijing, China · Beijing Institute of Mathematical Sciences and Applications (BIMSA), Beijing, China +4
InduceKV: Fixed-Footprint Continual Adaptation of Multimodal LLMs via Inducing KV Memories
Jul 2, 2026Qianyu Chen, Ziteng Feng, Canran Xiao +1Continual Learning for LLMs Memory-Efficient Fine-Tuning
1Nanyang Technological University · University of Science and Technology of China · 3Shenzhen Campus of Sun Yat-sen University
Denser $\neq$ Better: Limits of On-Policy Self-Distillation for Continual Post-Training
Jul 2, 2026Meng Wang, Haohan Zhao, Wenzhuo Liu +7Continual Learning for LLMs On-Policy Self-Distillation
Centre for Artificial Intelligence and Robotics, HKISI, CAS · Institute of Automation, CAS · University of Chinese Academy of Sciences +1
ISM:Self-Improving Strategy Memory for Continual Mathematical Reasoning
Jun 30, 2026Prakhar Dixit, Tim OatesContinual Learning for LLMs Memory-Augmented Language Models
Department of Computer Science, University of Maryland, Baltimore County, Baltimore, Maryland, USA
PASTA: A Paraphrasing And Self-Training Approach for Knowledge Updating in LLMs
Jun 27, 2026Takayuki Yamamoto, Daisuke KawaharaContinual Learning for LLMs Knowledge Augmentation for Language Models
Waseda University
MMLA: Memory-Mediated Learning Architecture for Predictive Dual-State Adaptation
Jun 27, 2026Junyi Zou, Avrova DonzContinual Learning for LLMs Test-Time Adaptation
1MMLA-org · 2Communication University of China (CUC), No. 1 Dingfuzhuang East Street, Chaoyang District, Beijing 100024, China
From Weights to Features: SAE-Guided Activation Regularization for LLM Continual Learning
Jun 25, 2026Evan Ning, Wei Xue, Dong Lou +1Continual Learning for LLMs Catastrophic Forgetting
1The Hong Kong University of Science and Technology
Can Scale Save Us From Plasticity Loss in Large Language Models?
Jun 23, 2026J. Fernando Hernandez-Garcia, Tomás Figliolia, Beren MillidgeContinual Learning for LLMs Loss of Plasticity
Zyphra San Francisco, CA
Towards Version-aware Operations and Transaction Memories for Multi-layer MeMo
Jun 23, 2026Peiran LiKnowledge Editing Continual Learning for LLMs
Freie Universität Berlin, Kaiserswerther Str. 16-18, 14195 Berlin, Germany
Fast and Slow Variational Continual Learning
Jun 22, 2026Subarnaduti Paul, Yohan Jung, Mohammad Emtiyaz Khan +3Continual Learning for LLMs Deep Learning Optimization
Faculty of Mathematics and Computer Science, University of Bremen, Bremen, Germany · Department of Computer Science & AI, Jeonbuk National University, Republic of Korea · Department of Computer Science, TU Darmstadt, Darmstadt, Germany +3
SamatNext v0.2-B: An Exploratory Study of RMS-Normalized Hybrid Decoders for Curriculum Retention in Small Code Models
Jun 20, 2026Samat ZharassovContinual Learning for LLMs Decoder-Only Language Models

Aug 9, 2026, cs.AIJ/K move · Enter open · S save

Yesterday's Shield, Today's Spear: A Self-Evolving Safety Guardrail in Production

Cong Ming, Jingyi Chen, Bin Liu, Qi Chu+3

University of Science and Technology of China · Shenzhen University · Sangfor Technologies

Continual Learning for LLMs LLM Guardrails Safety Filtering LLM Safety LLM Jailbreak Attacks

Open paper AskPDF arXiv

Deployed LLM safety guardrails are predominantly static: trained once and frozen at release, while new jailbreak techniques and previously un-addressed harmful categories emerge within days, leaving the defense perpetually a step behind. We present SESG (Self-Evolving Safety Guardrails), a multi-agent system running in production. SESG monitors the live traffic behind a deployed guardrail and surfaces two classes of failure: jailbreaks novel in form and harmful categories novel in content. Once a failure is confirmed, a generation agent synthesizes paired training data targeted at it; a validation agent rebalances the batch toward the direction in which the deployed model errs, so that the model's own mistakes steer its training set; and a routing agent matches the training action to the diagnosed gap and returns the next version to production. Over six rounds of live evolution (V0 to V6), a 1.7B guardrail adapts to a new threat in 16-24 hours, with about 2 hours of human effort, versus the 40-90 hours of the manual process it replaces. On six emerging threats, it outperforms static guardrails from 0.6B to 9B and an adaptive baseline while preserving its general screening competence. Since April 2026, SESG has been the primary update pipeline of Sangfor's guardrail, autonomously closing 14 of 15 new threat scenarios in two months. We release 9 test sets for the 6 new threats at https://github.com/Trams1017/SESG. Warning: This paper contains examples that may be harmful or offensive.

Momentum

Related topics

Latest papers 151