CAMFT: Conflict-Aware Mergeable Fine-Tuning for Large Language Models

Jingang Zhou; Haiyang Guo; Yuan Ma; Han Zhu; Xu-Yao Zhang

cs.LGSep 6, 2026

CAMFT: Conflict-Aware Mergeable Fine-Tuning for Large Language Models

Authors: Jingang Zhou, Haiyang Guo, Yuan Ma, Han Zhu, Xu-Yao Zhang

Organizations: School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences

Abstract

Model merging has emerged as a promising paradigm for integrating multiple task-specific capabilities into a single large language model. However, existing methods predominantly focus on post-hoc processing of independently fine-tuned models, overlooking how the training phase itself impacts cross-task compatibility. Resolving parameter conflicts after fine-tuning is inherently sub-optimal. To address this, we propose CAMFT, a Conflict-Aware Mergeable Fine-Tuning method that makes task adaptation both efficient and mergeaware. CAMFT treats mergeability as a property shaped during fine-tuning, rather than only a problem to be solved after fine-tuning. By guiding each task to update sparse coordinates with lower cross-task conflict, CAMFT produces task updates that are efficient to train and more compatible for downstream model merging. Extensive experiments demonstrate that CAMFT outperforms standard finetuning baselines in multi-task merging scenarios. Codes are available at https://github.com/gyanchow/CAMFT-LLM.

Figures & tables

Figure 1: Pairwise cosine similarities among task vectors obtained by independently full-parameter fine-tuning on seven tasks.

Figure 2: Overview of CAMFT. CAMFT constructs a shared SVD coordinate system and uses gradient probing to quantify cross-task conflicts. It then allocates sparse, conflict-aware masks to each task. By isolating parameter updates, CAMFT ensures the independently fine-tuned models are highly compatible for merging.

Method	Params	C-ST	FOMC	SciQA	NG-cm	NG-ds	MtgBk	20Min	Avg ( $n=3$ )	Merge.
		Single-task Performance
Full FT	Full	0.500	0.649	0.791	0.407	0.588	0.649	0.400	0.569 $\pm$ 0.002	–
LoRA	45.1M	0.451	0.679	0.848	0.247	0.619	0.664	0.399	0.558 $\pm$ 0.004	–
PiSSA	45.1M	0.431	0.653	0.835	0.264	0.603	0.673	0.382	0.549 $\pm$ 0.002	–
SVFT	34.4M	0.423	0.626	0.617	0.257	0.533	0.526	0.365	0.478 $\pm$ 0.008	–
OSRM †	45.1M	0.436	0.673	0.845	0.231	0.566	0.643	0.384	0.540 $\pm$ 0.005	–

Table 1: Performance comparison on TRACE 7-task benchmark with Llama-3.2-1B. Single : single-task performance before merging. Merged : performance after merging all 7 tasks with TIES. Task columns show the original fixed-seed breakdown; Avg is the mean

\pm

standard deviation of the macro-average over three random seeds; Merge. is the ratio of the corresponding three-seed means. Params : trainable parameters per task (M = million; Full = full-model fine-tuning). All task metrics are normalized to [0,1]; best results are bolded . † : methods designed for mergeability.

Variant	Merged Avg	$\boldsymbol{\Delta}$
CAMFT (Full)	0.464	–
w/o Conflict Estimation	0.395	$-$ 0.069
w/o Gradient Probing	0.376	$-$ 0.088
w/o SVD-space Importance	0.427	$-$ 0.037

Table 2: Ablation study of CAMFT’s core components on Llama-3.2-1B.

\Delta

denotes the absolute drop from full CAMFT.

Figure 3: Sensitivity of CAMFT to the trainable coordinate ratio

\rho

. The star marks the best merged performance.

# Samples	10	100	200	500	1000
Merged Avg	0.4057	0.4276	0.4592	0.4644	0.4497

Table 3: Effect of probing sample size on merged performance. Best result is bolded .

Merging Method	Full FT	MergOPT	CAMFT
Average	0.368	0.378	0.389
Task Arithmetic	0.388	0.395	0.402
DARE	0.378	0.385	0.391
TIES	0.418	0.442	0.464

Table 4: CAMFT improves mergeability across different merging algorithms. We report the best merged average after hyperparameter search for each method.

Panel of Figure 4 — Figure 4: Cross-task interference before and after applying CAMFT. Top: pairwise cosine similarity of task vectors on the seven TRACE tasks under FFT (left) and CAMFT (right); Bottom: mean off-diagonal cosine at each of the 112 linear layers of Llama-3.2-1B.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Source	Language	Avg. Len.	Metric
Domain-specific
ScienceQA	Science	English	210	Accuracy
FOMC	Finance	English	51	Accuracy
MeetingBank	Meeting	English	2,853	ROUGE-L
Multi-lingual
C-STANCE	Social media	Chinese	127	Accuracy

Appendix

Table 5: An overview of dataset statistics in experiments. Source indicates the origin of the context. Avg. Len. denotes the average length in words for English and German datasets, and in characters for Chinese. SARI is a metric specific to simplification. For all metrics, the larger the corresponding value, the better.

Task	Epochs
C-STANCE	5
FOMC	3
ScienceQA	7
NumGLUE-cm	5
NumGLUE-ds	5
MeetingBank	7

Appendix

Table 6: Task-specific training schedules.

Hyperparameter	Search Grid
Learning rate (Full FT)	$\{1\mathrm{e}{-5},\ 5\mathrm{e}{-5},\ 1\mathrm{e}{-4}\}$
Learning rate (PEFT)	$\{5\mathrm{e}{-5},\ 1\mathrm{e}{-4},\ 5\mathrm{e}{-4}\}$
Weight decay	$\{0,\ 0.01\}$
LoRA $\alpha$ (LoRA/PiSSA/OSRM)	$\{16,\ 32,\ 64\}$
Batch size	$16$
Optimizer	AdamW

Appendix

Table 7: Hyperparameter search grids for fine-tuning baselines. The same grid is used across all tasks; per-task best configurations are selected on validation.

Merging Method	Hyperparameters
Average	$-$
Task Arithmetic	$s\in\{0.2,0.4,0.6,0.8,1.0\}$
DARE	$d\in\{0.1,0.2,0.25,0.3,0.5\}$ ; $s\in\{0.2,0.4,0.6,0.8,1.0\}$
TIES	$d\in\{0.1,0.2,0.25,0.3,0.5\}$ ; $s\in\{0.2,0.4,0.6,0.8,1.0\}$

Appendix

Table 8: Hyperparameter search grids for different merging algorithms. The same search space is used for all fine-tuning methods.

Hyperparameter	Search Grid
Specificity reward $\gamma$	$\{0.5,\ 1.0,\ 2.0,\ 4.0\}$
Conflict penalty $\lambda$	$\{0.5,\ 1.0,\ 2.0,\ 4.0\}$
Exclusivity ratio $h$ (%)	$\{5,\ 10,\ 20\}$
Scaling factor $\alpha$	$\{0.5,\ 1.0,\ 2.0,\ 4.0\}$

Appendix

Table 9: Search grids for CAMFT-specific hyperparameters.

\gamma

\lambda

h

are shared across tasks;

\alpha

is tuned per task.

Global (shared across tasks)
Hyperparameter	Llama-3.2-1B	Qwen3-4B
$\gamma$	1.0	1.0
$\lambda$	2.0	2.0
$h$ (%)	10	10
Per-task scaling factor $\alpha$
C-STANCE	1.0	1.0

Appendix

Table 10: Final CAMFT hyperparameter values used in the main experiments. Global values for

\gamma

\lambda

h

are shared across all tasks within a merging run; the per-task

\alpha

values are reported in the bottom block.

Figure 5: Per-layer task-vector cosine, decomposed by projection type. Each of the seven panels plots the 16 transformer blocks for one projection; the bottom-right panel reports the per-projection mean over all blocks.

LLM Fine-Tuning

LLM fine-tuning adapts pretrained large language models to tasks, domains, languages, and behaviors through additional training.

See all 746 papers

Post-Hoc Merging is Not Enough: Many-Shot Model Merging with Loss-Gap Balancing
Jun 15, 2026Kyungjin Im, Miru Kim, Chanin Eom +1Multi-Task Learning Language Model Merging
Soongsil University, Republic of Korea · Department of Electrical and Computer Engineering, Sungkyunkwan University, Republic of Korea
Model Merging in the Era of Large Language Models: Methods, Applications, and Future Directions
Mar 10, 2026Mingyang Song, Mao ZhengLanguage Model Merging Model Merging
Large Language Model Department, Tencent, China
Learning Not To Interfere: Adapting Experts For Model Merging
Mar 13, 2026Pratik Ramesh, George Stoica, Arun Iyer +2Model Merging
Georgia Tech · University of Washington · Microsoft Research +3

Jun 15, 2026J/K move · Enter open · S save

Post-Hoc Merging is Not Enough: Many-Shot Model Merging with Loss-Gap Balancing

Kyungjin Im, Miru Kim, Chanin Eom, Minhae Kwon

Soongsil University, Republic of Korea · Department of Electrical and Computer Engineering, Sungkyunkwan University, Republic of Korea

Multi-Task Learning Language Model Merging Model Merging

Open paper AskPDF arXiv

Model merging has become a practical post-training strategy for building a single multi-task large language model (LLM) by combining multiple task-specialized models. However, most existing approaches rely on post-hoc merging, in which task-specific models are merged only once after training. This one-shot aggregation often suffers from task interference, leading to information erasure across individual tasks. In this work, we show that replacing post-hoc merging with an iterative many-shot merging protocol is effective in improving multi-task performance. Building on this insight, we propose METIS, Mitigating Erasure from Task Interference for Stable many-shot merging. METIS is a loss-aware many-shot merging method that addresses information erasure in post-hoc merging through task-wise loss-gap weighting and consensus-based masking. Notably, METIS exhibits significant performance improvement on the worst-performing task, effectively mitigating information erasure. (Project page: https://imkyungjin.github.io/METIS/)

Abstract

LLM Fine-Tuning

Explore similar work

Post-Hoc Merging is Not Enough: Many-Shot Model Merging with Loss-Gap Balancing

Model Merging in the Era of Large Language Models: Methods, Applications, and Future Directions

Learning Not To Interfere: Adapting Experts For Model Merging