Overview

Task vector merging studies how to combine the weight changes made by separately fine-tuning a shared pretrained model, so one model can perform multiple tasks without pooling their training data. Common approaches include weighted averaging and sign-based methods such as TIES, sparsification, low-rank and SVD-based merging, and subspace alignment or input-dependent routing. Work spans vision and language benchmarks as well as continual learning, reinforcement learning, and multimodal models, with methods aiming to preserve useful updates while reducing interference. A central open tension is that task-vector geometry, update magnitudes, and task-specific structure can each affect outcomes, so a method that works across tasks and model settings—and reliably predicts harmful merges—remains elusive.

Latest papers

CardsList

38 papers4 papers in the last four weeks, down 50% on the four weeks before. 0.0% of all new papers.

  1. Separating Common and Unique Directions for Model Merging

    Nov 3, 2026Shiting Wang, Yingjie Zou, Ke LiData-Free Model MergingModel Merging

  2. A Fisher-Geometric View of Multi-Teacher On-Policy Distillation

    Oct 19, 2026Liu Kaiyuan, Zhuang Ziyuan, Ma Haoxuan +4Language Model MergingMulti-Teacher Knowledge Distillation

  3. Dual Task Vector Merging: Leveraging Emotion and ASR Knowledge for Low-Resource Alzheimer's Disease Detection

    Oct 17, 2026Yuan-Chao Li, Yin-Long Liu, Liu He +5Speech-Based Dementia DetectionAlzheimer's Disease Classification

  4. A Broader Look at Model Merging: Rethinking Implicit Regularization Induced by Task Arithmetic

    Oct 6, 2026Sin-Han Yang, Shih-Cheng Huang, Chieh-Yen Lin +3Task Vector MergingMulti-Task Optimization

  5. Predicting and Repairing Merge Collapse in Large Language Models

    Oct 2, 2026Jungseob Lee, Seungyoon Lee, Sugyeong Eo +3Language Model MergingTask Vector Merging

  6. ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs

    Sep 30, 2026Hang Yin, Haozhe Wang, Yuhua Luo +3Continual Learning for LLMsLow-Rank Adaptation

  7. No Task Vector Is an Island: A Comprehensive Study on the Composability of Task Vectors from On-Policy Distillation

    Sep 30, 2026Jingang Zhou, Feiyu Han, Han Zhu +6Task VectorsTask Vector Merging

  8. Orthogonal Yet Coupled: Decoupling Geometric Components for Model Merging

    Sep 29, 2026Zijing Wang, Yongkang Liu, Mingyang Wang +8Task Vector MergingModel Merging

  9. CASS: Contribution-Aware Structured Sparsity for Model Merging

    Sep 28, 2026Yan Li, Guiping Cao, Meng Xu +5Multi-Task LearningStructured Sparsity

  10. Escaping Low-Dimensional Overlap: Multi-Task Model Merging via High-Dimensional Sparse Disentanglement

    Aug 26, 2026Yihang Zhang, Shengke Sun, Junjie Wen +1Language Model MergingModel Merging

  11. HyperFix: Combinatorial Nonlinear Correction for Task Vector Merging

    Aug 11, 2026Hyo Seo Kim, Ren WangTask Vector MergingModel Merging

  12. Signature-Guided Capacity Occupancy for Dense Expert Merging

    Aug 10, 2026Lingching Tung, Chi-Jui Kim, Beicheng Xu +2Task Vector MergingLanguage Model Merging

  13. Asymmetric Collapse in Model Merging: When Refusal Over- writes Recognition

    Jul 26, 2026Aarnav Choudhary, Matheus Fonseca Rocha, Jiwon Seo +2Language Model Safety EvaluationTask Vector Merging

  14. DeltaMerge-LowRes: Composing Language and Task Deltas for Low-Resource Adaptation

    Jul 15, 2026Son Ha Xuan, Xuan-Bach Le, Phat T. Tran-TruongMultilingual Language ModelsTask Vector Merging

  15. Training-free Task Classification for Multi-Task Model Merging

    Jun 21, 2026Jungyong Son, Jinwook Jung, Sungyong BaikMulti-Task LearningTask Vector Merging

  16. PACT: Preserving Anchored Cores in Task-vectors for Model Merging

    Jun 17, 2026Ningyuan Shi, Zhipeng Zhou, Hao Wang +2Multi-Task LearningTask Vector Merging

  17. StereoFactory: A Unified Merging Framework for Robust Stereo Matching

    Jun 16, 2026Xianda Guo, Pinhan Fu, Ruilin Wang +3Stereo MatchingTask Vector Merging

  18. RogueMerge: Robust and Unified Attacks against LLM Model Merging

    Jun 2, 2026Jinghuai Zhang, Yetian He, Kunlin Cai +3LLM SecurityTask Vector Merging

  19. ResMerge: Residual-based Spectral Merging of Large Language Models

    Jun 1, 2026Yandu Sun, Zhiyan Hou, Haokai Ma +6Task Vector MergingLanguage Model Merging

  20. Saliency-Aware Model Merging

    May 30, 2026Jungin Park, Jiyoung Lee, Kwanghoon SohnTask Vector MergingModel Merging

  21. Model Merging on Loss Landscape: A Geometry Perspective

    May 26, 2026Juanwu Lu, Anand Bhaskar, Brian Axelrod +2Task Vector MergingModel Merging

  22. Merging Methods for Multilingual Knowledge Editing for Large Language Models: An Empirical Odyssey

    May 13, 2026Kunil Lee, Ki-Young Shin, Jong-Hyeok Lee +1Knowledge EditingMultilingual Language Models

  23. Model Merging: Foundations and Algorithms

    May 2, 2026Donato CrisostomiMulti-Task LearningTask Vector Merging