Overview

Model merging studies how to combine weights or adapters from separately trained models into one model, aiming to reuse complementary skills without storing an ensemble or repeating joint training. Common approaches average checkpoints or add task vectors, while sparsification, low-rank and spectral methods, geometric alignment, and calibration-based routing aim to reduce interference. The methods are used for multi-task and continual learning, domain adaptation, and combining language, vision, speech, and specialized models; evaluations often measure task accuracy, capability retention, worst-task performance, and inference or storage cost. The central tension is that updates that help one task can erase or distort another: success depends on shared initialization, representation compatibility, update scale, and data or prompt distribution, and some task combinations may not be recoverable through weight-space fusion alone.

Latest papers

CardsList

214 papers18 papers in the last four weeks, down 53% on the four weeks before. 0.1% of all new papers.

  1. A Spectral Framework for Universal Proxy Regularization in Model Merging

    Nov 13, 2026Shangge Liu, Danyue Qian, Qi Fan +5Model MergingData-Free Model Merging

  2. A Margin Perspective on LoRA: Robustness to Catastrophic Forgetting and Adapter Merging (MaLoRA)

    Nov 10, 2026Ziqing Xu, Hancheng Min, Lachlan Ewen MacDonald +4Low-Rank AdaptationModel Merging

  3. Domain-Routed Delta Optimization for Multi-Teacher On-Policy Distillation

    Nov 9, 2026Lin YaoMulti-Teacher Knowledge DistillationOn-Policy Distillation

  4. Permutation-Based Pre-trained Model Merging via Reciprocal Orthogonality

    Nov 9, 2026Akira Ito, Daiki Chijiwa, Atsutoshi KumagaiModel Merging

  5. DeltaForge: A Post-hoc Mixture-of-Experts VLA for Multi-Domain Autonomous Driving

    Nov 9, 2026Bingzhang Wang, Mo Poorsartep, Yasaman EsfandiariVLMs for Autonomous DrivingMixture-of-Experts Models

  6. When Are Nearby Experts Composable? Cross-Task Model Merging in Neural Thickets

    Nov 8, 2026Suxin Ji, Hungtao Wan, Mingjun Liu +1Model Merging

  7. Robust and Efficient Continual Model Merging via Global Singular Subspace Separation and Restoration

    Nov 7, 2026Juncheol Shin, Byeongho Yu, Semin Kim +1continual model mergingModel Merging

  8. Enhancing Mathematical Reasoning in Turkish Large Language Models with Search-Based Inference and Model Merging

    Nov 7, 2026Ahmed Zeer, Osama Shbib, Muzaffer Kaan Yuce +2LLM ReasoningLLM Fine-Tuning

  9. FedFiOT: Fisher-Weighted Optimal Transport for Model Merging in One-Shot and Few-Round Federated Learning

    Nov 4, 2026Quan Minh Nguyen, Hoang M. Ngo, Bhakti Chowkwale +2Federated Learning AggregationFederated Learning

  10. Model Merging Does Not Require Uniform LoRA Ranks

    Nov 4, 2026Zirong Xie, Zainab Afolabi, Nikita Kozodoi +1Language Model MergingModel Merging

  11. Modulating Merging Strengths via Joint Loss Estimation for LoRA-based Continual Learning

    Nov 3, 2026Kun Gu, De Cheng, Zhipeng Xu +3Continual LoRAModel Merging

  12. Separating Common and Unique Directions for Model Merging

    Nov 3, 2026Shiting Wang, Yingjie Zou, Ke LiData-Free Model MergingModel Merging

  13. P2^2-MOPD: Accelerating Multi-Teacher On-Policy Distillation via Parameter Priors

    Nov 2, 2026Sinuo Wang, Jiacheng Lu, Zeyu Zhang +8Language Model DistillationMulti-Teacher Knowledge Distillation

  14. Low-Rank Hierarchical Merging for Efficient Long-to-Short Reasoning

    Nov 2, 2026Zeqiu Yu, Xuesheng Zhang, Wenxiao Zhao +1Language Model MergingModel Merging

  15. Dynamic Merging Mutually-Distilled Multi-View Networks for RGB-Event-based Representation Learning

    Oct 27, 2026Futian Wang, Fan Zhang, Xiao Wang +4RGB-Event Sensor FusionRepresentation Learning

  16. Preserve to Expand: Source-Aware Adaptation and Model Merging for Multilingual LLM-Based ASR

    Oct 26, 2026Yuchen Zhang, Haralambos Mouratidis, Ravi ShekharMultilingual ASRAutomatic Speech Recognition

  17. Error-Subspace Calibration for Multi-Task Model Merging

    Oct 24, 2026Yeseul Cho, Hanseul Cho, Chulhee YunModel MergingModel Calibration

  18. HimmBO: High-Dimensional Interactive Model-Merging Bayesian Optimization for Image Generation

    Oct 22, 2026Jinyuan Mao, Zhihao Shen, Chenxi LiuModel MergingBayesian Optimization

  19. LoRA Superposition across Pretrained Models

    Oct 22, 2026Keigo Watanabe, Hiroshi Kera, Kazuhiko KawamotoLow-Rank AdaptationModel Merging

  20. Training-Free On-Policy Distillation

    Oct 15, 2026Bangji Yang, Jiajun Fan, Zhaoyu Zhu +2On-Policy DistillationKnowledge Distillation

  21. RecSoup: Efficient Model Merging Approach for Controllable Multi-Objective Recommendation

    Oct 15, 2026Zidong Jia, Guofu Xie, Xiao ZhangRecommender SystemsModel Merging

  22. SuperAI: Merging Capabilities Across Heterogeneous Foundation Models

    Oct 11, 2026Jingtong Dou, Chuancheng Shi, Wenhua Wu +4Orthogonal Procrustes AlignmentModel Merging

  23. CrossMean: Model Merging by Pairing Merged and Expert Activations

    Oct 10, 2026Moshe EliasofModel Merging

  24. UniMergeTrack: Hierarchical Model Merging for General RGB-X Visual Object Tracking

    Oct 9, 2026Shiao Wang, Xiao Wang, Yuhe Ding +5Visual Object TrackingModel Merging

  25. CHASE: Channel-Aligned Structure Exploitation for Geometry-Aware Model Engineering

    Oct 7, 2026Wei Wang, Wei Jiang, Ziran LiuNeural Network CompressionKV-Cache Compression

  26. InterCorrect: Intersection-Aware Correction of Demographic Model Merging for Fair ASR

    Oct 6, 2026Ashley E. Bravo-Bravo, Yuchen Zhang, Haralambos Mouratidis +2Demographic Bias in ASRModel Merging