Continual Learning for LLMs

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

21 papers in the last four weeks, up 250% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 151

All topics

Prism: A Plug-in Reproducible Infrastructure for Scalable Multimodal Continual Instruction Tuning
May 25, 2026Jun-Tao Tang, Yu-Cheng Shi, Zhen-Hao Xie +1Continual Learning for LLMs Multimodal Large Language Models
National Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China
Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay
May 25, 2026Martin Marek, Dongkyu Cho, Shikai Qiu +3Continual Learning for LLMs LLM Fine-Tuning
New York University
SAE-FD: Sparse Autoencoder Feature Distillation for Continual Learning of Large Language Models
May 25, 2026Mingxu Zhang, Yuhan Li, Lujundong Li +3Continual Learning for LLMs
The Hong Kong University of Science and Technology (Guangzhou) · Nanjing University of Aeronautics and Astronautics · The 63rd Research Institute, National University of Defense Technology, Nanjing
Beyond Inference-Only Deployment: Comparing Weight-Based Consolidation Against Cascading Compaction
May 23, 2026Simon Dennis, Kevin Shabahang, Hao Guo +1Continual Learning for LLMs LLM Personalization
i14 / University of Melbourne · i14
PALoRA: Projection-Adaptive LoRA for Preserving Reasoning in Large Language Models
May 23, 2026Mustafa Hayri Bilgin, Mariam Barry, Albert Bifet +2Knowledge Editing Continual Learning for LLMs
IT Group, Research, BNP Paribas · LTCI, Télécom Paris, Institut Polytechnique de Paris · AI Institute, University of Waikato +1
Dynamic Mixture of Latent Memories for Self-Evolving Agents
May 21, 2026Dianzhi Yu, Vireo Zhang, Hongru Wang +7Continual Learning for LLMs Mixture-of-Experts Language Models
The Chinese University of Hong Kong · Nanyang Technological University · University of Edinburgh +2
Modality-Decoupled Online Recursive Editing
May 19, 2026Siyuan Li, Youyuan Zhang, Fangming Liu +1Continual Learning for LLMs Multimodal Large Language Models
Harbin Institute of Technology, Shenzhen, China. · Peng Cheng Laboratory, China. · Huazhong University of Science and Technology, China.
CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning
May 18, 2026Yang Liu, Toan Nguyen, Flora D. SalimContinual Learning for LLMs Vision-Language Models
School of Computer Science and Engineering University of New South Wales Australia
Don't Let Bandit Feedback Pull Continual LLM-Recommender Updates Off Target
May 17, 2026Taesan Kim, Hyeongjun Yun, Jaegul Choo +1Continual Learning for LLMs Recommender Systems
1SK Telecom · 2KAIST
Toward LLMs Beyond English-Centric Development
May 15, 2026Sho Takase, Ukyo HondaMultilingual Language Models Continual Learning for LLMs
CyberAgent
Is One Score Enough? Rethinking the Evaluation of Sequentially Evolving LLM Memory
May 14, 2026Songwei Dong, Zihan Chen, Chengshuai Shi +3Continual Learning for LLMs LLM Evaluation
University of Virginia · Princeton University
TFGN: Task-Free, Replay-Free Continual Pre-Training Without Catastrophic Forgetting at LLM Scale
May 14, 2026Anurup GanguliContinual Learning for LLMs Continued Pretraining
Independent Researcher
Octopus: History-Free Gradient Orthogonalization for Continual Learning in Multimodal Large Language Models
May 14, 2026Yuehao Liu, Shanyan Guan, Weijia Zhang +4Continual Learning for LLMs Continual Learning for VLMs
MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University · vivo Mobile Communication Co., Ltd.
Model-Agnostic Lifelong LLM Safety via Externalized Attack-Defense Co-Evolution
May 13, 2026Xiaozhe Zhang, Chaozhuo Li, Hui Liu +4Continual Learning for LLMs Adversarial Attacks on LLMs
City University of Hong Kong · Beijing University of Posts and Telecommunications · Wuhan University +2
Continual Fine-Tuning of Large Language Models via Program Memory
May 13, 2026Hung Le, Svetha VenkateshContinual Learning for LLMs Memory-Augmented Language Models
Deakin Applied AI Initiative, Deakin University, Australia
Always Learning, Always Mixing: Efficient and Simple Data Mixing All The Time
May 13, 2026Michael Y. Hu, Apurva Gandhi, Kyunghyun Cho +2Language Model Pretraining Continual Learning for LLMs
New York University · Carnegie Mellon University · Microsoft
Low-Rank Adapters Initialization via Gradient Surgery for Continual Learning
May 12, 2026Joana Pasquali, Ramiro N. Barros, Arthur S. Bianchessi +7Continual Learning for LLMs Gradient Interference
MALTA, Machine Learning Theory and Applications Lab, PUCRS, Porto Alegre, Brazil · Kunumi Institute, Brazil
Learning with Rare Success but Rich Feedback via Reflection-Enhanced Self-Distillation
May 12, 2026Yuwei Zhang, Sha Li, Changlong Yu +9Continual Learning for LLMs Self-Distillation
UC San Diego1 · Amazon2 · Georgia Institute of Technology3
Early Data Exposure Improves Robustness to Subsequent Fine-Tuning
May 12, 2026Lawrence Feng, Gaurav R. Ghosal, Jacob Mitchell Springer +2Language Model Pretraining Continual Learning for LLMs
Carnegie Mellon University
Learning, Fast and Slow: Towards LLMs That Adapt Continually
May 12, 2026Rishabh Tiwari, Kusha Sareen, Lakshya A Agrawal +6Continual Learning for LLMs RL for Language Model Reasoning
1UC Berkeley · 4Eragon · 2Mila +3
ORBIT: Preserving Foundational Language Capabilities in GenRetrieval via Origin-Regulated Merging
May 12, 2026Neha Verma, Nikhil Mehta, Shao-Chuan Wang +7Continual Learning for LLMs LLM Fine-Tuning
Johns Hopkins University · Google DeepMind · Google
Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuning
May 11, 2026Tao Hu, Da-Wei ZhouCross-Modal Learning Continual Learning for LLMs
School of Artificial Intelligence, Nanjing University · State Key Laboratory for Novel Software Technology, Nanjing University
Towards Understanding Continual Factual Knowledge Acquisition of Language Models: From Theory to Algorithm
May 11, 2026Haoyu Wang, Yifan Shang, Zhongxiang Sun +3Continual Learning for LLMs Generative Replay
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China · School of Artificial Intelligence and Data Science, University of International Business and Economics, Beijing, China
Trajectory Supervision for Continual Tool-Use Learning in LLMs
May 10, 2026Vishnu Vardhan Reddy, Sagnik Chatterjee, Soumik BhattaContinual Learning for LLMs LLM Tool Use
CS 590NN UMass Amherst
Geometry Conflict: Explaining and Controlling Forgetting in LLM Continual Post-Training
May 10, 2026Yuanyi Wang, Yifan Yang, Su Lu +9Continual Learning for LLMs Model Merging
The Hong Kong Polytechnic University, PolyU · The Hong Kong University of Science and Technology, HKUST · PolyU-Daya Bay Technology and Innovation Research Institute
Muon-OGD: Muon-based Spectral Orthogonal Gradient Projection for LLM Continual Learning
May 9, 2026Binghang Lu, Zheyuan Deng, Runyu Zhang +6Continual Learning for LLMs Gradient Descent
Purdue University · Brown University · Massachusetts Institute of Technology +4
Self-Consolidating Language Models: Continual Knowledge Incorporation from Context
May 8, 2026Zekun Wang, Anant Gupta, Zihan Dong +1Continual Learning for LLMs Long-Context Language Modeling
Georgia Institute of Technology
CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning
May 7, 2026Md Anwar Hossen, Fatema Siddika, Juan Pablo Munoz +2Continual Learning for LLMs Low-Rank Adaptation
1Iowa State University · 2Maro Systems, USA · University of California, Berkeley +1
Attribution-Guided Continual Learning for Large Language Models
May 6, 2026Yazheng Liu, Yuxuan Wan, Rui Xu +3Continual Learning for LLMs LLM Fine-Tuning
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · The Beijing University of Posts and Telecommunications, Beijing, China
Continual Knowledge Updating in LLM Systems: Learning Through Multi-Timescale Memory Dynamics
May 6, 2026Andreas Pattichis, Constantine DovrolisContinual Learning for LLMs LLM Memory
The Cyprus Institute, Nicosia, Cyprus · School of Computer Science, Georgia Institute of Technology, Atlanta, GA, USA.

May 25, 2026, cs.LGJ/K move · Enter open · S save

Prism: A Plug-in Reproducible Infrastructure for Scalable Multimodal Continual Instruction Tuning

Jun-Tao Tang, Yu-Cheng Shi, Zhen-Hao Xie, Da-Wei Zhou

National Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China

Continual Learning for LLMs Multimodal Large Language Models Multimodal Instruction Tuning

Open paper AskPDF arXiv

Multimodal Large Language Models (MLLMs) achieve versatility by reformulating diverse tasks into a unified instruction-following framework via instruction tuning. However, real-world deployment requires continuous adaptation to emerging tasks, motivating Multimodal Continual Instruction Tuning (MCIT). Despite its growing importance, current MCIT research is hindered by severe engineering bottlenecks. Existing methods are typically implemented by directly modifying the base MLLM codebase, which imposes substantial implementation overhead and yields method-specific architectures that severely limit code reuse and fair comparison. To address this, we introduce Prism, a plug-in reproducible codebase specifically designed for scalable MCIT research. It separates algorithmic development from the backbone implementation via a lightweight plugin registration mechanism, enabling new strategies to be integrated as independent plugins without modifying the underlying MLLM codebase, thereby eliminating structural fragmentation and accelerating method development. Prism natively supports widely used large-scale training pipeline, thereby enabling reproducible and scalable MCIT experimentation. Code is available at https://github.com/LAMDA-CL/Prism.

Momentum

Related topics

Latest papers 151