Data-Free Model Merging

Latest papers

8 papers0 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Separating Common and Unique Directions for Model Merging
Nov 3, 2026Shiting Wang, Yingjie Zou, Ke LiData-Free Model Merging Model Merging
Earlham Institute, United Kingdom · Living Systems Institute, University of Exeter, United Kingdom · Department of Computer Science, University of Exeter, United Kingdom +2
Beyond Uniform Subspaces: Spectrum-Aware and Depth-Adaptive Fusion for Multi-Task Model Merging
Sep 21, 2026Ruxi Gu, Zilei Wang, Wei WangModel Merging Data-Free Model Merging
Department of Automation, University of Science and Technology of China · State Key Laboratory of General Artificial Intelligence, BIGAI · School of Computer and Communication Engineering, University of Science and Technology Beijing
SAFE-Merge: Data-Free Continual Model Merging with General Knowledge Preservation
Aug 2, 2026Zihuan Qiu, Zhiyang Liao, Chiyuan He +5Language Model Merging Model Merging
University of Electronic Science and Technology of China, Chengdu, China · Dalian University of Technology, Dalian, China
Saliency-Aware Model Merging
May 30, 2026Jungin Park, Jiyoung Lee, Kwanghoon SohnTask Vector Merging Model Merging
Yonsei University, Seoul, South Korea · Ewha Womans University, Seoul, South Korea.
ReForge: Refining Merged Models with Anchor-Regularized Regression
May 13, 2026Kaiyang Li, Shaobo Han, Qing Su +1Model Merging Data-Free Model Merging
School of Computing, University of Connecticut, Storrs, CT 06269 · Optical Networking and Sensing, NEC Labs America, Princeton, NJ 08540
Model Merging via Data-Free Covariance Estimation
Apr 1, 2026Marawan Gamal Abdel Hameed, Derek Tam, Pascal Jr Tikeng Notsawo +2Multi-Task Learning Language Model Merging
Mila & DIRO, Universit´e de Montr´eal · University of Toronto & Vector Institute
Learning Not To Interfere: Adapting Experts For Model Merging
Mar 13, 2026Pratik Ramesh, George Stoica, Arun Iyer +2Data-Free Model Merging Model Merging
Georgia Tech · University of Washington · Microsoft Research +3
ACE-Merging: Data-Free Model Merging with Adaptive Covariance Estimation
Mar 3, 2026Bo Xu, Haotian Wu, Hehai Lin +4Data-Free Model Merging Model Merging
The Hong Kong University of Science and Technology (Guangzhou) · University of Science and Technology of China

Nov 3, 2026J/K move · Enter open · S save

Separating Common and Unique Directions for Model Merging

Shiting Wang, Yingjie Zou, Ke Li

Earlham Institute, United Kingdom · Living Systems Institute, University of Exeter, United Kingdom · Department of Computer Science, University of Exeter, United Kingdom+2

Data-Free Model Merging Model Merging Task Vector Merging Task Vectors

Open paper AskPDF arXiv

Model merging combines multiple fine-tuned models without additional training, and arithmetic methods have emerged as a simple yet effective way to merge models without data. However, existing arithmetic methods treat task vectors in parameter space and apply a single element-wise rule to directions that can differ in mergeability. In a shared directional basis, we find a consistent two-regime structure: a low-dispersion common bulk shared across tasks and a high-dispersion task-dominated tail that carries disproportionate energy. These findings motivate Common--Unique Subspace Decomposition (CUSD), a data-free pre-merging method that separates task vectors into common and unique directional components, merges the common components with a downstream arithmetic operator, and retains unique residuals separately. Experiments across

11

base models, four model families, and four arithmetic-based merging operators show that CUSD improves 42 of 44 model--operator combinations and achieves up to 22.5% improvement over the corresponding baselines. Our code is publicly available at https://github.com/stwang666/CUSD