Continual Model Merging

Ask about this topic

Momentum

13 papers in the last four weeks, up 225% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 73

All topics
CardsList

  1. Mixture-Trained Merging for Unified Multi-Objective Models

    Oct 1, 2026SeongHyeon Kim, Chaeyun Jang, Seungyoo Lee +4Continual Model MergingMulti--Task Learning

    KAIST · Kakao

  2. SLIM: Simplex-Lattice Interpolation Merging

    Oct 1, 2026Seongcheol Jeong, Masahiro Suzuki, Yutaka MatsuoContinual Model MergingMixture-Of-Experts

    The University of Tokyo

  3. ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs

    Sep 30, 2026Hang Yin, Haozhe Wang, Yuhua Luo +3Replay-Based Continual LearningParameter-Efficient Fine-Tuning Methods

    Shanghai Jiao Tong University · Shanghai Innovation Institute · Beijing Zhongguancun Academy +3

  4. Exploring Heterogeneous Model Merging Approach for Complex Knowledge Transfer

    Sep 30, 2026Jiahe Fan, Si Chen, Yinghao Hou +4Cross-Task Knowledge TransferContinual Model Merging

    University of Science and Technology of China

  5. Orthogonal Yet Coupled: Decoupling Geometric Components for Model Merging

    Sep 29, 2026Zijing Wang, Yongkang Liu, Mingyang Wang +8Continual Model MergingMulti--Task Learning

    Northeastern University, China · CIS, LMU Munich, Germany · Munich Center for Machine Learning (MCML), Germany +1

  6. QAM: Quadratic-Accurate Checkpoint Merging via Sequential Consistency

    Sep 28, 2026Shihao Wang, Rui Kong, Xinran Chen +7Model CheckpointsContinual Model Merging

    University of Wisconsin – Madison · Baidu Inc. · University of Arizona +3

  7. CASS: Contribution-Aware Structured Sparsity for Model Merging

    Sep 28, 2026Yan Li, Guiping Cao, Meng Xu +5Continual Model MergingSoft-Token Representations

    Pengcheng Laboratory, Shenzhen, China · City University of Hong Kong, Hong Kong, China · Southern University of Science and Technology, Shenzhen, China +2

  8. SMAT: Simple and Efficient Merge-Aware Training

    Sep 27, 2026Yanggan Gu, Yuanyi Wang, Zhen Li +5Continual Model MergingSoft-Token Representations

    The Hong Kong Polytechnic University (PolyU) · The Hong Kong University of Science and Technology (Guangzhou) · The Chinese University of Hong Kong +1

  9. Beyond Uniform Subspaces: Spectrum-Aware and Depth-Adaptive Fusion for Multi-Task Model Merging

    Sep 21, 2026Ruxi Gu, Zilei Wang, Wei WangContinual Model MergingMulti--Task Learning

    Department of Automation, University of Science and Technology of China · State Key Laboratory of General Artificial Intelligence, BIGAI · School of Computer and Communication Engineering, University of Science and Technology Beijing

  10. Not All Task Vectors Need Equal Rank: Energy-Proportional Allocation for Model Merging

    Sep 21, 2026Hyunjoong Cho, Jinhyeok JangContinual Model MergingMulti--Task Learning

    LG CNS · ETRI · UST

  11. On Emergent Capabilities and Model Merging

    Sep 21, 2026Luca Zhou, Emanuele RodolàContinual Model MergingEmergent Misalignment

    Sapienza University of Rome · Paradigma

  12. Tracing and Coordinating Cross-Layer Influence for Multimodal Model Merging

    Sep 14, 2026Pengyang Zhou, Xiaobin Tu, Zhengxi Liu +7Continual Model MergingMulti-Modal Data

    Alibaba Cloud

  13. SPARROW: Scalable Taxonomy Induction via Structure-Preserving Partitioning and Constraint-Guided Merging

    Sep 7, 2026Yirui Zhang, Yixuan Tang, Yandong Sun +2TaxonomyPartitioning

    School of Computing, National University of Singapore

  14. CABS+: Efficient and Scalable Model Merging via Conflict-Aware Sparsification and Adaptive Weight Allocation

    Aug 13, 2026Yuchen Liu, Zongzhen Yang, Binhang Qi +2Continual Model MergingMulti--Task Learning

    State Key Laboratory of Complex & Critical Software Environment (CCSE), Beihang University, Beijing, China · Hangzhou Innovation Institute of Beihang University, Hangzhou, China · National University of Singapore, Singapore

  15. HyperFix: Combinatorial Nonlinear Correction for Task Vector Merging

    Aug 11, 2026Hyo Seo Kim, Ren WangContinual Model MergingMulti--Task Learning

    Illinois Institute of Technology

  16. Towards Efficient Reasoning in LLM-Based Recommender Systems via Model Merging

    Aug 11, 2026Linh Dieu Le, Tong Chen, Shazia Sadiq +3Continual Model MergingRecommendation Quality

    The University of Queensland Brisbane, Australia · Griffith University Brisbane, Australia

  17. Signature-Guided Capacity Occupancy for Dense Expert Merging

    Aug 10, 2026Lingching Tung, Chi-Jui Kim, Beicheng Xu +2Continual Model MergingMerge

    Peking University · Independent Researcher

  18. A Model Merging Approach for Continual MLLM Unlearning

    Aug 5, 2026Yuhang Wang, Linlin Zhang, Haoxuan Ji +3Large Language Model UnlearningExact Unlearning

    1Xidian University, Xi’an, China · 2Xi’an Jiaotong University, Xi’an, China

  19. A Unified Model for Cross-Domain Clone Detection via Model Merging

    Aug 4, 2026Palash R. Roy, Banani Roy, Kevin A. Schneider +1Continual Model Merging

    University of Saskatchewan Computer Science Saskatoon, Canada

  20. SAFE-Merge: Data-Free Continual Model Merging with General Knowledge Preservation

    Aug 2, 2026Zihuan Qiu, Zhiyang Liao, Chiyuan He +5Continual Model MergingReplay-Based Continual Learning

    University of Electronic Science and Technology of China, Chengdu, China · Dalian University of Technology, Dalian, China

  21. Sharpness-aware Model Merging with Salience Recovery for LLM-based Cross-Domain Sequential Recommendation

    Jul 28, 2026Huwei Ji, Jiajie Su, Yuyuan Li +2Continual Model Merging

    Zhejiang University Hangzhou, China · Hangzhou Dianzi University Hangzhou, China

  22. Rethinking Expert Training for Model Merging with Prompt Learning

    Jul 27, 2026Christos Georgakilas, Aniello Panariello, Samir El Karrat Moreno +3Continual Model MergingContrastive Language-Image Pre-Training Model

    Computer Vision Center, Barcelona, Spain · Universitat Aut`onoma de Barcelona, Barcelona, Spain · AImageLab, University of Modena and Reggio Emilia, Italy

  23. Asymmetric Collapse in Model Merging: When Refusal Over- writes Recognition

    Jul 26, 2026Aarnav Choudhary, Matheus Fonseca Rocha, Jiwon Seo +2Continual Model MergingSafety Alignment

    UCLA · Algoverse · Purdue University +1

  24. TwistedMerge: Certified Higher-Order Diagnostics and Abstention for Model Merging

    Jul 23, 2026Ting Gong, Shitan XuContinual Model MergingMerge

    Department of Mathematics, University of Washington, Box 354350, Seattle, WA 98195-4350, USA · Beijing International Center for Mathematical Research, Peking University, No. 5 Yiheyuan Road, Haidian District, Beijing, 100871, P.R. China

  25. Rethinking Heterogeneous LLM Merging: A Weighted Model Averaging Perspective

    Jul 20, 2026Jiahe Fan, Yinghao Hou, Si Chen +3Continual Model MergingFederated Averaging

    University of Science and Technology of China · School of Information Science and Technology, Department of Automation, University of Science and Technology of China · School of Computer Science and Technology, University of Science and Technology of China

  26. CT-Merging: Consensus Directions and Task-Level Scaling for LoRA Adapter Merging

    Jul 18, 2026Keumseo Ryum, Joonhyuk KangContinual Model MergingLow-Rank Adaptation Adapters

    KAIST

  27. When Model Merging Rivals Joint Multi-Task Reinforcement Learning: A Task-Vector Geometry Analysis

    Jul 17, 2026S. Aaron McClendonContinual Model MergingMerge

    Aimpoint Digital Labs

  28. Are we Merging the Right Models? Impact of Expert Training Duration on Model Merging for LLMs

    Jul 13, 2026Nikita Kozodoi, Zainab Afolabi, Jack ButlerContinual Model MergingLoss Function

    1Amazon Web Services

  29. Continual Model Merging with Test-Time Adaptation for Whole-Slide Image Analysis

    Jul 6, 2026Duc-Thanh Le, Doanh C. Bui, Maï K. Nguyen +1Continual Model MergingComputational Pathology

    University of Information Technology · 2Viet Nam National University Ho Chi Minh City · 3ETIS (UMR 8051), CY Cergy Paris University, ENSEA, CNRS, France

  30. Model Merging as Probabilistic Inference in Fine-Tuning Parameter Space

    Jul 2, 2026Long Minh Bui, Tuan Anh Le Van, Tung Phi Duc +3Continual Model MergingProbabilistic Inference

    Washington State University · Hanoi University of Science and Technology

  31. Training-free Task Classification for Multi-Task Model Merging

    Jun 21, 2026Jungyong Son, Jinwook Jung, Sungyong BaikContinual Model MergingMulti--Task Learning

    Department of Artificial Intelligence Hanyang University · Department of Data Science Hanyang University

  32. Predicting Mergeability of Parameter-Efficient Fine-Tuning Updates

    Jun 17, 2026Lin Tang, Wei Zhang, Jing Li +3Large Language Model AdaptationParameter-Efficient Fine-Tuning Methods

    1Sichuan University, Chengdu, China · University of Electronic Science and Technology of China, Chengdu, China

  33. Essential Subspace Merging for Multi-Task Learning

    Jun 17, 2026Longhua Li, Lei Qi, Xin Geng +1Continual Model MergingMulti--Task Learning

    School of Computer Science and Engineering, Southeast University, and Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China, 211189 · Huawei Inc., Shenzhen 518129, China

  34. Model Merging to Evolution: Parameter Space Exploration for Expert Models

    Jun 17, 2026Chao Wang, Yuchen Guo, Zheng Tan +4Continual Model MergingMulti--Task Learning

    School of Artificial Intelligence, Xidian University, Xi’an, China · School of Computer Science, University of Birmingham, Birmingham, UK · Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China +3

  35. Enhancing Multilingual Reasoning via Steerable Model Merging

    Jun 17, 2026Zhuoran Li, Rui Xu, Jian Yang +8Continual Model MergingMultilingual Language Models

    Beijing University of Posts and Telecommunications · Fudan University · Beihang University +4

  36. PACT: Preserving Anchored Cores in Task-vectors for Model Merging

    Jun 17, 2026Ningyuan Shi, Zhipeng Zhou, Hao Wang +2Continual Model MergingMulti--Task Learning

    Shanghai Jiao Tong University · Nanyang Technological University · The Hong Kong University of Science and Technology (Guangzhou)

  37. StereoFactory: A Unified Merging Framework for Robust Stereo Matching

    Jun 16, 2026Xianda Guo, Pinhan Fu, Ruilin Wang +3Stereo MatchingStereo

    School of Computer Science, Wuhan University, Wuhan, China · D-Star Robotics, Beijing, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China +1

  38. Post-Hoc Merging is Not Enough: Many-Shot Model Merging with Loss-Gap Balancing

    Jun 15, 2026Kyungjin Im, Miru Kim, Chanin Eom +1Continual Model MergingMulti--Task Learning

    Soongsil University, Republic of Korea · Department of Electrical and Computer Engineering, Sungkyunkwan University, Republic of Korea

  39. From Parameters to Feature Space: Task Arithmetic for Backdoor Mitigation in Model Merging

    Jun 10, 2026Zhenqian Zhu, Yamin Hu, Yiya Diao +3Continual Model MergingClean Label Backdoor Attack

    Institute of Cyberspace Security, School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, 518055, China. · Shenzhen Key Laboratory of Media Security, Shenzhen University, Shenzhen, 518060, China.

  40. Closed-Form Spectral Regularization for Multi-Task Model Merging

    Jun 5, 2026Yongxian Wei, Runxi Cheng, Xingxuan Zhang +4Continual Model MergingMulti--Task Learning

    Shenzhen International Graduate School, Tsinghua University, Shenzhen 518071, China · Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China · Sun Yat-sen University, Shenzhen 510275, China +1

  41. When Model Merging Breaks Routing: Training-Free Calibration for MoE

    Jun 2, 2026Canbin Huang, Tianyuan Shi, Xiaojun Quan +3Mixture-Of-ExpertsContinual Model Merging

    School of Computer Science and Engineering, Sun Yat-sen University, China · Shenzhen Loop Area Institute, China · Meituan, Inc., China +1

  42. GFFMERGE: Efficient Merging of Graph Neural Force Fields and Beyond

    Jun 2, 2026Parth Verma, Parv P. Singh, Vipul Garg +3Graph Neural NetworksContinual Model Merging

    Department of Computer Science, Indian Institute of Technology Delhi, Hauz Khas, New Delhi, India · Yardi School of Artificial Intelligence, Indian Institute of Technology Delhi, Hauz Khas, New Delhi, India · Department of Civil Engineering, Indian Institute of Technology Delhi, Hauz Khas, New Delhi, India

  43. ResMerge: Residual-based Spectral Merging of Large Language Models

    Jun 1, 2026Yandu Sun, Zhiyan Hou, Haokai Ma +6Continual Model MergingMerge

    1Southeast University, Nanjing, China · Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences +4

  44. Saliency-Aware Model Merging

    May 30, 2026Jungin Park, Jiyoung Lee, Kwanghoon SohnContinual Model MergingMulti--Task Learning

    Yonsei University, Seoul, South Korea · Ewha Womans University, Seoul, South Korea.

  45. Model Merging by Output-Space Projection

    May 27, 2026Bethan Evans, Benjamin Etheridge, Stephen Roberts +1Continual Model MergingMulti--Task Learning

    Department of Mathematics, University of Oxford, Oxford, UK · Department of Engineering Science, University of Oxford, Oxford, UK.

  46. Model Merging on Loss Landscape: A Geometry Perspective

    May 26, 2026Juanwu Lu, Anand Bhaskar, Brian Axelrod +2Continual Model MergingGeometric Metrics

    Purdue University · Waymo LLC

  47. On the Limits of Model Merging for Multilinguality in Pre-Training

    May 25, 2026Seth Aycock, Fedor Vitiugin, Aleksandr Umnov +2Multilingual Language ModelsContinual Model Merging

    University of Amsterdam

  48. Tunable MAGMAX: Preference-Aware Model Merging for Continual Learning

    May 20, 2026Kei Hiroshima, Kento Uchida, Shinichi ShirakawaReplay-Based Continual LearningContinual Model Merging

    Yokohama National University, Yokohama Kanagawa 240-8501, Japan

  49. Unlocking the Potential of Continual Model Merging: An ODE Perspective

    May 19, 2026Lihong Lin, Haidong KangContinual Model MergingReplay-Based Continual Learning

    Northeastern University, Shenyang, China

  50. Dynamic Model Merging Made Slim

    May 17, 2026Guodong Du, Wanyu LinContinual Model MergingMixture-Of-Experts Architectures

    The Hong Kong Polytechnic University

  51. E-PMQ: Expert-Guided Post-Merge Quantization with Merged-Weight Anchoring

    May 16, 2026Wenjun Wang, Yanggan Gu, Shuo Cai +4Post-Training QuantizationContinual Model Merging

    The Hong Kong Polytechnic University · PolyU-Daya Bay Technology and Innovation Research Institute

  52. Conflict-Free Replicated Data Types for Neural Network Model Merging: A Two-Layer Architecture Enabling CRDT-Compliant Model Merging Across 26 Strategies

    May 16, 2026Ryan GillespieContinual Model MergingNeural Collapse

    Independent researcher

  53. Discovering Physical Directions in Weight Space: Composing Neural PDE Experts

    May 14, 2026Pengkai Wang, Pengwei Liu, Yuanyi Wang +7Neural Partial Differential Equation SolversPartial Differential Equations

    1The Hong Kong Polytechnic University · 2Zhejiang University · 3Tsinghua University +1

  54. Darwin Family: MRI-Trust-Weighted Evolutionary Merging for Training-Free Scaling of Language-Model Reasoning

    May 14, 2026Taebong Kim, Youngsik Hong, Minsik Kim +4Continual Model MergingLLM Reasoning Strategies

    VIDRAFT Inc., Seoul, South Korea

  55. Learning Selective Merge Policies for Deadline-Constrained Coded Caching via Deep Reinforcement Learning

    May 13, 2026Amirhossein YousefiramandiSource-Channel CodingCache

  56. FeatCal: Feature Calibration for Post-Merging Models

    May 13, 2026Yanggan Gu, Shuo Cai, Zihao Wang +7Continual Model MergingDirect Feature Fusion

    The Hong Kong Polytechnic University (PolyU) · The Chinese University of Hong Kong · PolyU-Daya Bay Technology and Innovation Research Institute

  57. ReForge: Refining Merged Models with Anchor-Regularized Regression

    May 13, 2026Kaiyang Li, Shaobo Han, Qing Su +1Continual Model MergingMulti--Task Learning

    School of Computing, University of Connecticut, Storrs, CT 06269 · Optical Networking and Sensing, NEC Labs America, Princeton, NJ 08540

  58. Black-Box Optimization of Mixed Binary-Continuous Variables: Challenges and Opportunities in Evolutionary Model Merging

    May 12, 2026Md. Robiul Islam Niloy, Youhei AkimotoContinual Model MergingOptimization Modeling

    BRAC University, Bangladesh

  59. Revitalizing the Beginning: Avoiding Storage Dependency for Model Merging in Continual Learning

    May 8, 2026Xi Wang, Cheng DengReplay-Based Continual LearningContinual Model Merging

    School of Electronic Engineering, Xidian University, Xi’an, Shaanxi, China.

  60. Model Merging: Foundations and Algorithms

    May 2, 2026Donato CrisostomiContinual Model MergingMulti--Task Learning

    SAPIENZA UNIVERSITY OF ROME · GLADIA research lab · Department of Computer Science +1