Continual Learning for LLMs

LLM: Large Language Model

Ask about this topic

Momentum

21 papers in the last four weeks, up 250% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 150

All topics
CardsList

  1. CoDe-LoRA: Mitigating the Orthogonality Dilemma in Continual Learning of LLMs via Knowledge Consolidation and Decoupling

    Oct 6, 2026Maoqi Liu, Quan Fang, Yufei HeContinual Learning for LLMsLow-Rank Adaptation

    Beijing University of Posts and Telecommunications · National University of Singapore

  2. Off-Policy Merging Beats On-Policy Self-Distillation for Continual Learning

    Oct 5, 2026Chen Henry Wu, Thomas Zhang, Aditi RaghunathanContinual Learning for LLMsOn-Policy Distillation

    Carnegie Mellon University

  3. Task Vector Descent: Learning from Non-IID Batches

    Oct 4, 2026Anton Baumann, Jonas Hübotter, Zeynep Akata +1Continual Learning for LLMsTask Arithmetic

    ETH Zurich · Technical University of Munich

  4. Learning without Overwriting: A Theory of Self-Distillation and Supervised Fine-Tuning in Continual Reasoning

    Oct 4, 2026Shinichi Uemura, Taiji SuzukiSupervised Fine-TuningContinual Learning for LLMs

    Department of Mathematics Informatics, The University of Tokyo · Center for Advanced Intelligence Project, RIKEN, Tokyo, Japan

  5. Decoupling Memory from Context: Structured Memory for Token-Efficient Test-Time Continual Learning

    Oct 2, 2026Yehya Farhat, Michael Desmond, Anastasios KyrillidisContinual Learning for LLM AgentsContinual Learning for LLMs

    Rice University, Department of Computer Science Houston, TX, USA · IBM Research Yorktown Heights, NY, USA

  6. Local Support Learning

    Oct 1, 2026Assaf Ben-Kish, Akarsh Kumar, James Glass +1Continual Learning for LLMsCatastrophic Forgetting

    Tel Aviv University · MIT CSAIL

  7. ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs

    Sep 30, 2026Hang Yin, Haozhe Wang, Yuhua Luo +3Continual Learning for LLMsLow-Rank Adaptation

    Shanghai Jiao Tong University · Shanghai Innovation Institute · Beijing Zhongguancun Academy +3

  8. Replay on Demand: An Emergent Curriculum for Balancing Adaptation and Forgetting in Continued Pretraining

    Sep 30, 2026Lukas Thede, Shengzhuang Chen, Stefan Winzeck +3Continual Learning for LLMsReplay-Based Continual Learning

    University of Tübingen, Tübingen AI Center · Helmholtz Munich · Munich Center for Machine Learning (MCML) +3

  9. Ready2Blend: From Natural-Language Instructions to Composable Alignment Prompts

    Sep 30, 2026Jeesu Jung, Hwan Chang, Juseon Do +5Continual Learning for LLMsLLM Alignment

    Korea Advanced Institute of Science and Technology · Samsung SDS

  10. MedKIT: Evaluating Knowledge Integration and Generalization in Large Language Models

    Sep 29, 2026Lukas Thede, Yash Kumar Atri, David Chen +4Continual Learning for LLMsFactual Knowledge in Language Models

    University of Tübingen, Tübingen AI Center · Helmholtz Munich · Munich Center for Machine Learning (MCML) +4

  11. Generalizable Lifelong Model Editing via Preference Optimization

    Sep 29, 2026Dahyun Jung, Suhyune Son, Heuiseok LimContinual Learning for LLMsContinual Learning

    Department of Computer Science and Engineering, Korea University

  12. Learn Here, Move Less Elsewhere: Input-Conditioned Plasticity from Retained-Domain Activation Atlases

    Sep 28, 2026Jiangtao Lin, Bangyang Wei, Yihang Ding +2Continual Learning for LLMsDomain Adaptation for LLMs

    Tsinghua Shenzhen International Graduate School, Tsinghua University · School of Vehicle and Mobility, Tsinghua University · SZ DJI Technology Co., Ltd. +1

  13. Reliable Replay through Spatial Coherence in Online Continual Learning

    Sep 27, 2026Haixiang Sun, Jiefu Zhang, Yinghao He +4Continual Learning for LLMsExperience Replay

    Purdue University West Lafayette, IN 47907, USA

  14. Learning Dynamics of Continual Learning: A Unified View of Data Attribution, Forgetting, and Plasticity Loss

    Sep 27, 2026Yi Ren, Wenlong Deng, Guanzhe Hong +2Continual Learning for LLMsLoss of Plasticity

    OATML · University of Oxford · UBC +1

  15. Fine Until Fine-Tuned: Repeated Solutions Make Reasoning Fragile

    Sep 27, 2026Ely SheikhContinual Learning for LLMsData Repetition in Language Model Training

    Aix-Marseille University

  16. ManiEdit: Sequential Unstructured Knowledge Editing for Language Models from a Manifold Perspective

    Sep 27, 2026Rui Liu, Chenheng Zhang, Haoxuan Li +1Knowledge EditingContinual Learning for LLMs

    State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Institute for Artificial Intelligence, Peking University

  17. COPE: Continual Personalization of LLMs under Sparse User Feedback via User Embeddings and Self-Evaluation

    Sep 22, 2026Ruike Cao, Fugen Yao, Liang Dong +3Continual Learning for LLMsPersonalized Language Models

    University of Science and Technology of China · Qwen Business Unit of Alibaba

  18. iSDFT: Information-Proximal Self-Distillation for Continual Learning in LLMs

    Sep 21, 2026Ahmed Khaled Khamis, Xiaotong Ji, Hassan Jaber +4Continual Learning for LLMsLanguage Model Distillation

    Imperial College London · Politecnico di Torino · UCL Centre for AI

  19. Brain-Inspired Hierarchical Modularity for General Continual Learning

    Sep 21, 2026Hongwei Yan, Kanglei Zhou, Qi Cheng +7Continual Learning for LLMsContinual Learning for VLA Models

    School of Life Sciences, Tsinghua University, Beijing, China. · IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China. · Tsinghua-Peking Center for Life Sciences, Beijing, China. +2

  20. Time-Incremental Continued Pretraining of LLMs: Knowledge Updates Without Catastrophic Forgetting

    Sep 20, 2026Fırat Öncel, Salman Hussain Ali, Mirco Ravanelli +2Continual Learning for LLMsFactual Knowledge in Language Models

    Concordia University · Mila – Quebec AI Institute · Université de Montréal +3

  21. Bilinear Optimization Divergence: Diagnosing Factor-Constrained LoRA Continual Learning

    Sep 20, 2026YongShun Wang, JianLin Su, Yong MaContinual Learning for LLMsLow-Rank Adaptation

  22. Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data

    Sep 16, 2026Jinli Hu, Ross M. Clarke, Yichuan Zhang +1Continual Learning for LLMsHypernetworks

    Boltzbit Limited · University of Cambridge

  23. Smarter by the Moment: Environment-Driven Dynamic Policies for Continual LLM Improvement

    Sep 15, 2026Ting-Wei Chang, Po-Chun Chen, Hen-Hsen Huang +1Continual Learning for LLMsMemory-Augmented Language Models

    Department of Computer Science and Information Engineering, National Taiwan University, Taiwan · Institute of Information Science, Academia Sinica, Taiwan · AI Research Center (AINTU), National Taiwan University, Taiwan

  24. Beyond Endpoint Scores: Time- and Capacity-Conditioned Evaluation of Continual Knowledge Updating

    Sep 3, 2026Heejin ChoiContinual Learning for LLMsContinual Learning

    Yonsei University · Sooil Development · NCreate

  25. Hypotheses-Guided Self Distillation for Continual Personalization

    Aug 31, 2026EunJeong Hwang, Kushan Mitra, Dan Zhang +2Continual Learning for LLMsLLM Personalization

    University of British Columbia · Work done as an intern at Megagon Labs. · Megagon Labs, USA

  26. Reading the News: Adapting Large Language Models to Swedish Journalism Through Continued Pre-Training

    Aug 31, 2026Lukas Borggren, Jenny Kunz, Marco KuhlmannContinual Learning for LLMsLLM Evaluation

    Linköping University · Bonnier News

  27. In-Cell Learning: Language Models That Update Their Own Weights in Sequence Without Changing the File They Ship

    Aug 21, 2026Zifeng Liu, Yaxin Lu, Xuanhan Wu +6LLM QuantizationKnowledge Editing

    Big Data and Artificial Intelligence Center, The Third Affiliated Hospital of Sun Yat-sen University · Institute for Frontier Interdisciplinary Research in Health Sciences and Technology, Sun Yat-sen University · Sun Yat-sen University Institute of Artificial Intelligence +7

  28. AWARe: Mitigating Catastrophic Forgetting via Activation-Weighted Adaptive REtention

    Aug 12, 2026Juncheng Liao, Jinfan Lv, Guoming Wang +3Continual Learning for LLMsLLM Fine-Tuning

    School of Software Technology, Zhejiang University, Hangzhou, China · College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China · School of Artificial Intelligence, Sun Yat-Sen University, Guangzhou, China +1

  29. Different Feedback, Different Updates: Selective Self-Learning from User Interactions for Large Language Models

    Aug 10, 2026Xuanchen Li, Haitao Li, Yujia Zhou +5Continual Learning for LLMsSelf-Training for Language Models

    DCST, Tsinghua University · Tencent

  30. Yesterday's Shield, Today's Spear: A Self-Evolving Safety Guardrail in Production

    Aug 9, 2026Cong Ming, Jingyi Chen, Bin Liu +4Continual Learning for LLMsLLM Guardrails

    University of Science and Technology of China · Shenzhen University · Sangfor Technologies

  31. Omega-S: A Functional Resilience Index for LLM Fine-Tuning

    Aug 4, 2026Alberto AcedoContinual Learning for LLMsLLM Fine-Tuning

    Biome Makers Inc.

  32. Toward Plasticity-Preserving KL Regularization for Capability Retention in LLM Reinforcement Learning

    Aug 3, 2026Li Wang, Xiaodong Lu, Xiaohan Wang +4Continual Learning for LLMsKL-Regularized RL

    1Meituan · 2Nanyang Technological University

  33. Learning What to Remember: Test-Time Training via Context Distillation

    Aug 3, 2026Zixuan Wang, Xingyu Dang, Rui-Jie Zhu +4Continual Learning for LLMsLong-Context Language Modeling

    Princeton University · UC Santa Cruz · Carnegie Mellon University +1

  34. RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection

    Aug 3, 2026Shicheng Xu, Liang Pang, Liyi Chen +7Retrieval-Augmented GenerationContinual Learning for LLMs

    State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences · Xiaohongshu Inc.

  35. Unleashing the Potential of Large Language Models: A Blueprint for Real-Time, Enterprise-Ready Deployments

    Aug 1, 2026Muhammad Faizan Raza, Shuo, Yang +2Retrieval-Augmented GenerationContinual Learning for LLMs

    Luna · ¹ Penn State Great Valley, Malvern, PA 19355, USA · ² Penn State Brandywine, Media, PA 19063, USA +1

  36. The Parts Are Greater Than the Sum: Automated Task Sequencing for Efficient Training of Multi-Policy LLMs

    Jul 31, 2026Jiajia Tang, Sizhe Yuen, Francisco Gomez Medina +2Continual Learning for LLMsLow-Rank Adaptation

    The Alan Turing Institute, London, UK · King’s College London, London, UK · University of Southampton, Southampton, UK

  37. PARALLEL: A Prefrontal-Aligned Reinforcement inspired Approach for Language-Model Learning under Explicit Limits

    Jul 31, 2026Namkyung Yoon, Sanghong Kim, Hwangnam KimContinual Learning for LLMsLLM Fine-Tuning

    School of Electrical Engineering, Korea University Seoul 02841, Republic of Korea

  38. CACHE-UK: A Stability-Aware Memory Editor for Sequentially Updated Quantized LLMs in Finance

    Jul 30, 2026Anubhav Lakra, Yue FengKnowledge EditingContinual Learning for LLMs

    University of Birmingham, Birmingham, United Kingdom

  39. ForgetBench: Benchmarking Forgetting Dynamics of Long-Term Parametric Memory in Language Models

    Jul 29, 2026Ruxi Gu, Zhenliang Zhang, Wei WangKnowledge EditingContinual Learning for LLMs

    Department of Automation, University of Science and Technology of China, Hefei, China · State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing, China · School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, China.

  40. From Execution to Capability: Scientific Experience Consolidation via Procedural Knowledge Synthesis

    Jul 27, 2026Liwei Dong, Jiahao Zhao, Nan XuContinual Learning for LLMsScientific Code Generation

    ScienceOne AI

  41. Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning

    Jul 26, 2026Reza Rahimi Azghan, Gautham Krishna Gudur, Giulia Pedrielli +2Continual Learning for LLMsIncremental Learning

    College of Health Solutions, Arizona State University, Phoenix, Arizona, USA · Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, Texas, USA · School of Computing and Augmented Intelligence, Arizona State University, Tempe, Arizona, USA +1

  42. The Blessing of Dimensionality: How Near-Orthogonality in High-Dimensional Spaces Explains Temporal Portability

    Jul 22, 2026Abigail Woodring, Adrian Chan, Rana Muhammad Shahroz Khan +3Continual Learning for LLMsLow-Rank Adaptation

    Department of Electrical and Computer Engineering NC State University, Raleigh, NC 27606, USA · Independent Researcher · Department of Computer Science University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA

  43. Rethinking Transfer in Continual Learning: A Replay-Based Realisation

    Jul 17, 2026Yang Meng, Zhenya Liu, Zhuokai Zhao +1Continual Learning for LLMsExperience Replay

    Department of Computer Science, University of Chicago

  44. Can a Language Model Learn Facts Continually in Its Weights?

    Jul 13, 2026Charles O'Neill, Max Kirkby, Jonathon Liu +1Knowledge EditingContinual Learning for LLMs

    Baseten

  45. When Does Continual Learning Require Learning

    Jul 8, 2026Anne Harrington, Nayan Saxena, Michael Murphy +7Continual Learning for LLMsConcept Drift

    1UC Berkeley · 2Independent · 3Capital Fund Management

  46. MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning

    Jul 8, 2026Ruilin Tong, Dong GongContinual Learning for LLMsMemory-Augmented Language Models

    University of New South Wales (UNSW Sydney)

  47. MemDefrag: Latent Memory Defragmentation for Large Language Models

    Jul 7, 2026Ruiyi Yan, Zhuoyuan Mao, Yiwen GuoContinual Learning for LLMsMemory-Augmented Language Models

    Kyoto University · LIGHTSPEED · Independent Researcher

  48. ReCoLoRA: Spectrum-Aware Recursive Consolidation for Continual LLM Fine-Tuning

    Jul 4, 2026Wentao LuContinual Learning for LLMsLow-Rank Adaptation

  49. Statistically Meaningful Geometry (SMG) Beyond the Euclidean Paradigm, with Application to Generative AI

    Jul 3, 2026Bing Cheng, Yi-Shuai Niu, Howell Tong +1Neural Network GeneralizationInformation Geometry

    Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China · AMSS Center for Forecasting Science, Chinese Academy of Sciences, Beijing, China · Beijing Institute of Mathematical Sciences and Applications (BIMSA), Beijing, China +4

  50. InduceKV: Fixed-Footprint Continual Adaptation of Multimodal LLMs via Inducing KV Memories

    Jul 2, 2026Qianyu Chen, Ziteng Feng, Canran Xiao +1Continual Learning for LLMsMemory-Efficient Fine-Tuning

    1Nanyang Technological University · University of Science and Technology of China · 3Shenzhen Campus of Sun Yat-sen University

  51. Denser ≠\neq Better: Limits of On-Policy Self-Distillation for Continual Post-Training

    Jul 2, 2026Meng Wang, Haohan Zhao, Wenzhuo Liu +7Continual Learning for LLMsOn-Policy Self-Distillation

    Centre for Artificial Intelligence and Robotics, HKISI, CAS · Institute of Automation, CAS · University of Chinese Academy of Sciences +1

  52. ISM:Self-Improving Strategy Memory for Continual Mathematical Reasoning

    Jun 30, 2026Prakhar Dixit, Tim OatesContinual Learning for LLMsMemory-Augmented Language Models

    Department of Computer Science, University of Maryland, Baltimore County, Baltimore, Maryland, USA

  53. PASTA: A Paraphrasing And Self-Training Approach for Knowledge Updating in LLMs

    Jun 27, 2026Takayuki Yamamoto, Daisuke KawaharaContinual Learning for LLMsKnowledge Augmentation for Language Models

    Waseda University

  54. MMLA: Memory-Mediated Learning Architecture for Predictive Dual-State Adaptation

    Jun 27, 2026Junyi Zou, Avrova DonzContinual Learning for LLMsTest-Time Adaptation

    1MMLA-org · 2Communication University of China (CUC), No. 1 Dingfuzhuang East Street, Chaoyang District, Beijing 100024, China

  55. From Weights to Features: SAE-Guided Activation Regularization for LLM Continual Learning

    Jun 25, 2026Evan Ning, Wei Xue, Dong Lou +1Continual Learning for LLMsCatastrophic Forgetting

    1The Hong Kong University of Science and Technology

  56. Can Scale Save Us From Plasticity Loss in Large Language Models?

    Jun 23, 2026J. Fernando Hernandez-Garcia, Tomás Figliolia, Beren MillidgeContinual Learning for LLMsLoss of Plasticity

    Zyphra San Francisco, CA

  57. Towards Version-aware Operations and Transaction Memories for Multi-layer MeMo

    Jun 23, 2026Peiran LiKnowledge EditingContinual Learning for LLMs

    Freie Universität Berlin, Kaiserswerther Str. 16-18, 14195 Berlin, Germany

  58. Fast and Slow Variational Continual Learning

    Jun 22, 2026Subarnaduti Paul, Yohan Jung, Mohammad Emtiyaz Khan +3Continual Learning for LLMsDeep Learning Optimization

    Faculty of Mathematics and Computer Science, University of Bremen, Bremen, Germany · Department of Computer Science & AI, Jeonbuk National University, Republic of Korea · Department of Computer Science, TU Darmstadt, Darmstadt, Germany +3

  59. SamatNext v0.2-B: An Exploratory Study of RMS-Normalized Hybrid Decoders for Curriculum Retention in Small Code Models

    Jun 20, 2026Samat ZharassovContinual Learning for LLMsDecoder-Only Language Models

  60. Beyond Uniform Forgetting: A Study of Sequential Direct Preference Optimization Across Preference Settings

    Jun 18, 2026Pranav Bhandari, Nicolas Fay, Amitava Datta +2Continual Learning for LLMsDirect Preference Optimization

    1Network Analysis and Social Influence Modelling (NASIM) Lab · School of Physics Maths and Computing, The University of Western Australia · School of Psychological Science, The University of Western Australia +1