RAIN-Merging: A Gradient-Free Method to Enhance Instruction Following in Large Reasoning Models with Preserved Thinking Format

Zhehao Huang; Yuhang Liu; Baijiong Lin; Yixin Lou; Zhengbao He; Hanling Tian; Tao Li; Xiaolin Huang

cs.LGFeb 26, 2026

RAIN-Merging: A Gradient-Free Method to Enhance Instruction Following in Large Reasoning Models with Preserved Thinking Format

Authors: Zhehao Huang, Yuhang Liu, Baijiong Lin, Yixin Lou, Zhengbao He, Hanling Tian, Tao Li, Xiaolin Huang

Organizations: Institute of Image Processing and Pattern Recoginition, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University · The Hong Kong University of Science and Technology (Guangzhou) · MoE Key Laboratory of System Control and Information Processing (Shanghai)

Abstract

Large reasoning models (LRMs) excel at a long chain of reasoning but often fail to faithfully follow instructions regarding output format, constraints, or specific requirements. We investigate whether this gap can be closed by integrating an instruction-tuned model (ITM) into an LRM. Analyzing their differences in parameter space, namely task vectors, we find that their principal subspaces are nearly orthogonal across key modules, suggesting a lightweight merging with minimal interference. However, we also demonstrate that naive merges are fragile because they overlook the output format mismatch between LRMs (with explicit thinking and response segments) and ITMs (answers-only). We introduce RAIN-Merging (Reasoning-Aware Instruction-attention guided Null-space projection Merging), a gradient-free method that integrates instruction following while preserving thinking format and reasoning performance. First, with a small reasoning calibration set, we project the ITM task vector onto the null space of forward features at thinking special tokens, which preserves the LRM's structured reasoning mechanisms. Second, using a small instruction calibration set, we estimate instruction attention to derive module-specific scaling that amplifies instruction-relevant components and suppresses leakage. Across four instruction-following benchmarks and nine reasoning & general capability benchmarks, RAIN-Merging substantially improves instruction adherence while maintaining reasoning quality. The gains are consistent across model scales and architectures, translating to improved performance in agent settings.

Figures & tables

Figure 1: An overview of RAIN-Merging . In the case, the LRM arrives at the correct solution but ignores the required format and specific code. To preserve the reasoning structure, we perform training-free merging by combining a task vector projected onto the null space of the thinking format with instruction-attention guided coefficients. The merged model remains correct while satisfying the specified constraints. See Sec. 3 for details.

Figure 2: Principal subspace cosine similarity between LRM and ITM task vectors for each layer and submodule. The similarities are consistently low ( $<0.1$ ).

Figure 3: Two stages of our RAIN-Merging pipeline. (a) For each submodule, the ITM task vector is projected onto the null space preventing shifts in thinking format. (b) Given the instruction calibration set, we compute the instruction-attention score from attention outputs to obtain merging coefficients.

Method	Instruction Following					Reasoning & General					RT
	IFEval	CELLO	Info	Complex	Avg.	Math	GPQA	Aider	Arena-	Avg.
	IFEval	CELLO	Bench	Bench	Avg.	Math	GPQA	Aider	Hard-v2	Avg.
ITM	70.43	19.15	78.49	43.63	52.92	47.27	29.80	33.33	62.86	43.32	–
LRM	55.45	16.59	71.73	32.72	44.12	64.75	44.44	29.63	65.29	51.03	–
SFT	62.48	17.11	68.58	32.15	45.08	62.57	41.92	28.89	64.67	49.51	120.32
Data-free Merging

Table 1: Comprehensive comparison of instruction following and reasoning & general capabilities. We merge Qwen2.5-7B-Instruct (ITM) into DeepSeek-R1-Distill-Qwen-7B (LRM) and compare our RAIN-Merging against multiple merging methods as well as SFT trained on the same calibration data. “Avg.” denotes the average over all subsets. “RT” reports the run-time for merging or training in minutes. The best and second-best results are highlighted in bold and underlined , respectively.

Model	Instruction Following					Reasoning & General
	IFEval	CELLO	Info	Complex	Avg.	Math	GPQA	Aider	Arena-	Avg.
	IFEval	CELLO	Bench	Bench	Avg.	Math	GPQA	Aider	Hard-v2	Avg.
Qwen2.5-1.5B-Instruct	36.78	19.04	64.76	27.83	37.10	31.77	25.76	16.30	38.45	28.07
DeepSeek-R1-Distill-Qwen-1.5B	39.00	16.03	55.29	21.54	32.97	41.62	29.29	14.07	39.73	31.18
Qwen2.5-1.5B-RAIN-Merging	41.59	16.51	58.18	23.62	34.97	45.87	33.33	14.81	40.93	33.74
(relative gain)	+6.64%	+2.98%	+5.23%	+9.63%	+6.09%	+10.21%	+13.79%	+5.26%	+3.02%	+8.20%

Table 2: Merging performance and relative gains of RAIN-Merging across model three scales and two architectures. We merge the corresponding ITM into the LRM with base models: Qwen2.5-1.5B/14B/32B, and Llama-3.1-8B. “Avg.” denotes the average over all subsets. For each scale, the subsequent “ (relative gain) ” row reports the relative improvement of our method over the LRM, highlighted in green.

Figure 6

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A1: Principal subspace cosine similarity between DeepSeek-R1-Distill-Qwen-1.5B (LRM) and Qwen2.5-1.5B-Instruct (ITM) task vectors for each layer and submodule.

Figure A2: Principal subspace cosine similarity between DeepSeek-R1-Distill-Qwen-14B (LRM) and Qwen2.5-14B-Instruct (ITM) task vectors for each layer and submodule.

Domain	Raw size	Calibration	Validation
Math	93,700	50	50
Code	83,100	50	50
Science	173,000	50	50
Total	349,800	150	150

Appendix

Table A1: Reasoning calibration set construction from Mixture-of-Thoughts . We uniformly sample 50 examples per domain for calibration and 50 for validation. Raw sizes are taken from the official dataset composition page.

Figure A3: A sample illustration in our instruction calibration set.

Benchmark	Size	Constraint	Composition Type				Verification		Evaluation
Benchmark	Size	Taxonomy	And	Chain	Selection	Nested	Code-Exec.	LLM-as-Judge	Aggregation
IFEval	541	25	✓	–	–	–	✓	–	strict_prompt_level_accuracy
CELLO	523	4	✓	✓	–	–	✓	–	average
InfoBench	500	5	✓	✓	–	–	–	✓	DRFR
ComplexBench	1,150	19	✓	✓	✓	✓	✓	✓	dependency-aware DRFR

Appendix

Table A2: Instruction-following benchmarks. We list dataset size, constraint taxonomy, composition types, verification, and aggregation strategy.

	AIME2025	AIME2024	AMC23	GSM8K	Math500	MinervaMath
# Test samples	30	30	40	1,319	500	272

Appendix

Table A3: Test set sizes of the six math benchmarks used in our mathematical reasoning (Math) evaluation.

Method	Hyper-parameters
Method	7B
Task Arithmetic	$\lambda=1.0$
SLERP	$\lambda=1.0$
Karcher	$\lambda=1.0$
TIES	$k=0.8,\ \lambda=0.8$
DARE-TIES	$p=0.3,\ k=0.5,\ \lambda=1.2$

Appendix

Table A4: The hyperparameters of various merging methods in Tab. 1 .

\lambda

means the global scaling coefficient in merging.

k

denotes the trim ratio in TIES-Merging.

p

means the drop rate in DARE merging.

\tau

is sharpness the ACM.

\rho

is the pruning ratio in LEWIS.

\omega

means the balance factor in AIM.

Method	Hyper-parameters
Method	1.5B	7B	8B	14B
RAIN-Merging	$\lambda=1.0$	$\lambda=1.0$	$\lambda=0.9$	$\lambda=1.0$

Appendix

Table A5: The hyperparameters of RAIN-Merging in different model sizes.

\lambda

means the global scaling coefficient in RAIN-Merging.

Method	AIME2025	AIME2024	AMC23	GSM8K	Math500	Minerva	Avg.
ITM	10.00	10.00	67.50	86.66	73.80	35.66	47.27
LRM	30.00	50.00	80.00	91.36	89.00	48.16	64.75
SFT	33.33	43.33	75.00	90.75	87.40	45.59	62.57
Task Arithmetic	30.00	50.00	80.00	90.75	89.00	45.59	64.22
SLERP	30.00	46.67	77.50	91.05	89.20	48.53	63.82
Karcher	30.00	50.00	80.00	90.98	89.20	48.90	64.85

Appendix

Table A6: Math benchmark results under the same configuration as in Tab. 1 . “Avg.” denotes the average over all math benchmarks. The best and second-best results are highlighted in bold and underlined , respectively.

Method	AIME2025	AIME2024	AMC23	GSM8K	Math500	Minerva	Average
Qwen2.5-1.5B-Instruct	3.33	0.00	30.00	75.44	59.40	22.43	31.77
DeepSeek-R1-Distill-Qwen-1.5B	20.00	13.33	42.50	73.77	71.80	28.31	41.62
Qwen2.5-1.5B-RAIN-Merging	20.00	16.67	60.00	76.36	72.40	29.78	45.87
Llama-3.1-8B-Instruct	3.33	6.67	20.00	81.65	67.60	34.26	35.59
DeepSeek-R1-Distill-Llama-8B	30.00	40.00	75.00	90.52	80.40	45.37	60.21
Llama-3.1-8B-RAIN-Merging	30.00	43.33	77.50	90.30	82.80	47.79	61.95

Appendix

Table A7: Math benchmarks results under the same configuration as in Tab. 2 . “Avg.” denotes the average over all math benchmarks.

Figure A4: Instruction following and reasoning & generage performance of our RAIN-Merging using different global scalar

\lambda

. The configuration is the same as in Tab. 2 . The performance is measured by the average of the instruction following and reasoning & general capability benchmarks. The marked result is our choice in the experiments.

Figure A5: Instruction following and reasoning & generage performance of our RAIN-Merging using different reasoning calibration set sizes. The

x

axis represents the size of the reasoning calibration set with exponential scale. The configuration is the same as in Tab. 2 . The performance is measured by the average of the instruction following and reasoning & general capability benchmarks. The marked result is our choice in the experiments.

Figure A6: Heatmap of merging coefficients by our Stage 2 for each layer and attention head of DeepSeek-R1-Distill-Qwen-7B.

Method	Instruction Following					Reasoning & General
	IFEval	CELLO	Info	Complex	Avg.	Math	GPQA	Aider	Arena-	Avg.
	IFEval	CELLO	Bench	Bench	Avg.	Math	GPQA	Aider	Hard-v2	Avg.
LRM	55.45	16.59	71.73	32.72	44.12	64.75	44.44	29.63	65.29	51.03
RAIN-Merging (Rule)	63.22	19.03	74.53	35.66	48.11	68.75	54.55	33.33	65.73	55.59
RAIN-Merging (Open)	62.92	19.24	74.89	35.67	48.15	65.14	49.49	31.11	64.67	52.59
RAIN-Merging (Rule+Open)	64.03	19.63	75.64	36.70	49.00	67.43	53.03	35.56	65.29	55.32

Appendix

Table A8: Performance of RAIN-Merging with different instruction calibration sets in Stage 2. Rule is our original instruction calibration set from IFEval, Open is the new set from InforBench, Rule+Open simply concatenates both sets. We merge Qwen2.5-7B-Instruct (ITM) into DeepSeek-R1-Distill-Qwen-7B (LRM) under the same configuration as Tab. 1 .

Model	IFBench	XIFBench	EIFBench	Average
Qwen2.5-7B-Instruct	27.89	83.35	55.62	55.62
DeepSeek-R1-Distill-Qwen-7B	17.69	72.93	45.31	45.31
Qwen2.5-7B-RAIN-Merging	19.39	76.32	47.85	47.85
(relative gain)	+9.62%	+4.65%	+5.62%	+5.62%

Appendix

Table A9: Merging performance and relative gains of RAIN-Merging on three new instruction-following benchmarks. We merge the Qwen2.5-7B family under the same configuration as Tab. 1 . The subsequent “ (relative gain) ” row reports the relative improvement of our method over the LRM, highlighted in green.

Figure A7: Instruction-type accuracy comparison between the LRM (DeepSeek-R1-Distill-Qwen-7B) and RAIN-Merging (Qwen2.5-7B-RAIN-Merging) on IFEval. The results highlight the largest improvements on instructions like response_language , number_words , and no_comma .

Model	IFEval(200) HAcc	IFEval-paraphrase HAcc	Robustness
Qwen2.5-7B-Instruct	65.71	64.16	0.98
DeepSeek-R1-Distill-Qwen-7B	57.86	50.85	0.88
Qwen2.5-7B-RAIN-Merging	61.29	61.81	1.01

Appendix

Table A10: Robustness to paraphrased instructions on IFEval. We merge the Qwen2.5-7B family under the same configuration as Tab. 1 . HAcc is hard accuracy (%). Robustness is defined as

\text{HAcc}(\text{paraphrase})/\text{HAcc}(\text{original})

Instruction Following

Instruction following studies how AI systems interpret and reliably carry out user goals, constraints, formats, and priorities.

See all 135 papers

Bridging Auxiliary Constraints to Resolve Instruction Following in Large Reasoning Models
Jun 2, 2026Zhengyi Zhao, Shubo Zhang, Huimin Wang +7LLM Prompting Instruction Following
The Chinese University of Hong Kong · Tencent Jarvis Lab · Westlake University +2
GRIP: Granular Reward-Guided Parameter Interpolation for Efficient Reasoning
Aug 26, 2026Lam So, Canhui Wu, Han LinEfficient Language Model Reasoning LLM Inference
Peking University · Xi’an Jiaotong University
Leveraging Instruction Tuning and Merging for Reasoning Model Adaptation
Jul 16, 2026Yu-Du Feng, Niels Mündler-Sasahara, Mark Vero +1Domain Adaptation LLM Fine-Tuning
Department of Computer Science, ETH Zurich, Zurich, Switzerland.

Jun 2, 2026J/K move · Enter open · S save

Bridging Auxiliary Constraints to Resolve Instruction Following in Large Reasoning Models

Zhengyi Zhao, Shubo Zhang, Huimin Wang, Zezhong Wang+6

The Chinese University of Hong Kong · Tencent Jarvis Lab · Westlake University+2

LLM Prompting Instruction Following Constraint Satisfaction Large Reasoning Models

Open paper AskPDF arXiv

Large Reasoning Models (LRMs) have demonstrated impressive capabilities in many tasks, yet they struggle with reliably following multiple instructions, either by failing to satisfy individual constraints or by struggling to balance competing constraints simultaneously. We formalize this challenge as the Constraint Adherence Problem (CAP). This paper introduces a novel framework that addresses CAP by representing instructions as a structured knowledge graph of constraints. Our approach, Constraint Relationship Graph Completion (CRGC), explicitly models relationships between constraints, identifies adherence challenges, and discovers ``bridge constraints'' that help the model better focus on and reconcile requirements. Bridge constraints act as auxiliary instructions that make primary constraints more salient and compatible. Unlike existing approaches that enhance instruction following through general training methods, CRGC specifically improves constraint satisfaction by leveraging the model's own knowledge to create better pathways for generation. Experiments across three popular instruction following datasets demonstrate that our approach reduces constraint violations by 39% compared to standard prompting while maintaining reasoning abilities of large reasoning models.

Abstract

Instruction Following

Explore similar work

Bridging Auxiliary Constraints to Resolve Instruction Following in Large Reasoning Models

GRIP: Granular Reward-Guided Parameter Interpolation for Efficient Reasoning

Leveraging Instruction Tuning and Merging for Reasoning Model Adaptation