Overview

Model merging studies how to combine weights or adapters from separately trained models into one model, aiming to reuse complementary skills without storing an ensemble or repeating joint training. Common approaches average checkpoints or add task vectors, while sparsification, low-rank and spectral methods, geometric alignment, and calibration-based routing aim to reduce interference. The methods are used for multi-task and continual learning, domain adaptation, and combining language, vision, speech, and specialized models; evaluations often measure task accuracy, capability retention, worst-task performance, and inference or storage cost. The central tension is that updates that help one task can erase or distort another: success depends on shared initialization, representation compatibility, update scale, and data or prompt distribution, and some task combinations may not be recoverable through weight-space fusion alone.

Latest papers

CardsList

214 papers18 papers in the last four weeks, down 53% on the four weeks before. 0.1% of all new papers.

  1. FedDAF: Federated Domain Adaptation Using Model Functional Distance

    Sep 15, 2025Mrinmay Sen, Sidhant Nair, C Krishna MohanFederated Learning AggregationDomain Adaptation

  2. Rethinking Layer-wise Model Merging through Chain of Merges

    Aug 29, 2025Pietro Buzzega, Riccardo Salami, Angelo Porrello +1Model MergingCovariate Shift

  3. One Size Does Not Fit All: Distribution-Guided Tensor-Wise Adaptive Sparsification for Model Merging

    Aug 8, 2025Yingfeng Luo, Dingyang Lin, Junxin Wang +10Model MergingTask Vector Merging

  4. Local Superior Soups: A Catalyst for Model Merging in Cross-Silo Federated Learning

    Oct 31, 2024Minghui Chen, Meirui Jiang, Xin Zhang +3Communication-Efficient Distributed TrainingCross-Silo Federated Learning