Overview

Language model merging studies how to combine specialized models—such as instruction-following, reasoning, multilingual, or domain-specific models—into a model that retains useful capabilities without always retraining on all the original data. Common approaches average weights or task vectors, sparsify or align conflicting updates, use Fisher or curvature information to set merge weights, and search for combinations with Bayesian or evolutionary optimization; some methods instead use routing or output-level fusion. Researchers evaluate these techniques on language, reasoning, code, multilingual, and safety benchmarks, often measuring both capability gains and forgetting of general skills. A central challenge is that compatibility varies by model pair and task: methods that work for related fine-tunes may fail for heterogeneous models, and preserving safety, calibration, and broad capability while adding new skills remains difficult.

Latest papers

CardsList

114 papers7 papers in the last four weeks, down 67% on the four weeks before. 0.1% of all new papers.

  1. Audit Before You Merge: Provenance, Probing, and Continual LoRA Composition

    Nov 7, 2026Lenny Aharon, Neta Glazer, Lior AharonLanguage Model MergingCatastrophic Forgetting

  2. Model Merging Does Not Require Uniform LoRA Ranks

    Nov 4, 2026Zirong Xie, Zainab Afolabi, Nikita Kozodoi +1Language Model MergingModel Merging

  3. Low-Rank Hierarchical Merging for Efficient Long-to-Short Reasoning

    Nov 2, 2026Zeqiu Yu, Xuesheng Zhang, Wenxiao Zhao +1Language Model MergingModel Merging

  4. Mostly Reading: What Open-Book Scores Measure in Document-Level LoRA Adapters

    Oct 22, 2026Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov +1Low-Rank AdaptationLanguage Model Merging

  5. Inheritance as an Admission Test for LLM Prompt Merge Operators

    Oct 21, 2026Zan-Kai Chong, Hiroyuki Ohsaki, Bryan NgLanguage Model MergingPrompt Engineering

  6. A Fisher-Geometric View of Multi-Teacher On-Policy Distillation

    Oct 19, 2026Liu Kaiyuan, Zhuang Ziyuan, Ma Haoxuan +4Language Model MergingMulti-Teacher Knowledge Distillation

  7. Controlling the Diversity-Quality Trade-Off of LLMs by Merging Models Before and After Alignment

    Oct 16, 2026Yasuaki Sumita, Tomoharu Iwata, Toshiyuki TanakaLanguage Model MergingDiverse Text Generation

  8. To Align or Not To Align: Check Your COMPASS Before You Train

    Oct 13, 2026Fabian Gröger, Petar Damjanović, Maria BrbićLanguage Model MergingCross-Modal Alignment

  9. SLEM: Spectrum-Aware Latent Expert Merging

    Oct 12, 2026Dohyeon Kim, Bedionita Soro, Bruno Andreis +1Language Model MergingMixture-of-Experts Language Models

  10. Towards Merging-conditional Backdoor Unalignment against Large Language Models

    Oct 10, 2026Yukun Chen, Yidou Chen, Yiming Li +4LLM Backdoor AttacksLLM Safety

  11. PGSB: Pretrained-Guided Shared Basis for LoRA Model Merging

    Oct 10, 2026Muqing Liu, Chongjie Si, Zhuoya Liu +1Language Model MergingLow-Rank Adaptation

  12. ααTransfer: Coefficient Transfer for Efficient Model Merging

    Oct 6, 2026Shih-Cheng Huang, Zhi Rui Tam, Chieh-Yen Lin +3Hyperparameter TransferLanguage Model Merging

  13. Training-Free Transformer Merging via Sequential Local Operator Alignment

    Oct 5, 2026Akansh Maurya, Ya-Wei Eileen Lin, Stefanie Jegelka +2Language Model MergingModel Merging

  14. BACAM: Behavior-Aware Continual Agent Merging for Multi-Turn Interaction

    Oct 4, 2026Shuaitong Li, Baochen Xiong, Xiaoshan Yang +4Language Model MergingContinual Learning for LLM Agents

  15. Predicting and Repairing Merge Collapse in Large Language Models

    Oct 2, 2026Jungseob Lee, Seungyoon Lee, Sugyeong Eo +3Language Model MergingTask Vector Merging

  16. Mixture-Trained Merging for Unified Multi-Objective Models

    Oct 1, 2026SeongHyeon Kim, Chaeyun Jang, Seungyoo Lee +4Multi-Task LearningLanguage Model Merging

  17. SLIM: Simplex-Lattice Interpolation Merging

    Oct 1, 2026Seongcheol Jeong, Masahiro Suzuki, Yutaka MatsuoSurrogate-Assisted OptimizationLanguage Model Merging

  18. ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs

    Sep 30, 2026Hang Yin, Haozhe Wang, Yuhua Luo +3Continual Learning for LLMsLow-Rank Adaptation

  19. Exploring Heterogeneous Model Merging Approach for Complex Knowledge Transfer

    Sep 30, 2026Jiahe Fan, Si Chen, Yinghao Hou +4Transfer LearningLanguage Model Merging

  20. QAM: Quadratic-Accurate Checkpoint Merging via Sequential Consistency

    Sep 28, 2026Shihao Wang, Rui Kong, Xinran Chen +7Language Model MergingModel Merging

  21. SMAT: Simple and Efficient Merge-Aware Training

    Sep 27, 2026Yanggan Gu, Yuanyi Wang, Zhen Li +5Language Model MergingModel Merging

  22. Routing Drift Alone Does Not Diagnose Failure in Merged MoE LLMs

    Sep 26, 2026Yuanyi Wang, Yanggan Gu, Su Lu +7Expert RoutingMixture-of-Experts Language Models

  23. Train4Merge: RL or SFT---Which Yields Better Teachers for OPD-Based Model Merging?

    Sep 26, 2026Jingyuan Huang, Zuming Huang, Yucheng Shi +4Language Model MergingLanguage Model Distillation

  24. On Emergent Capabilities and Model Merging

    Sep 21, 2026Luca Zhou, Emanuele RodolàEmergent Abilities in Language ModelsEmergent Misalignment in Language Models

  25. From Parameters to Behaviors: A Survey of Model Fusion for Large Language Models

    Sep 17, 2026Shuo Cai, Yanggan Gu, Zihao Wang +8Language Model MergingModel Merging

  26. Merging the Knowledge of LLMs for Automatic Speech Recognition

    Sep 14, 2026Hayato Futami, Tatsuya KawaharaDomain Adaptation for ASRAutomatic Speech Recognition