Multi-objective Evolutionary Merging Enables Efficient Reasoning Models

Mario Iacobelli; Adrian Robert Minut; Tommaso Mencattini; Donato Crisostomi; Andrea Santilli; Iacopo Masi; Emanuele Rodolà

cs.CLApr 7, 2026

Multi-objective Evolutionary Merging Enables Efficient Reasoning Models

Authors: Mario Iacobelli, Adrian Robert Minut, Tommaso Mencattini, Donato Crisostomi, Andrea Santilli, Iacopo Masi, Emanuele Rodolà

Organizations: Independent Researcher · Sapienza University of Rome · EPFL · NVIDIA ·

Abstract

Reasoning models achieve strong performance on complex problems by leveraging long chains of thought, but this deliberate reasoning incurs substantial inference-time cost. The Long-to-Short (L2S) reasoning problem seeks to preserve accuracy while reducing generated tokens. Yet, current training-free model merging approaches rely on brittle, fixed-hyperparameter arithmetic methods that force suboptimal compromises. We introduce Evo-L2S, a multi-objective evolutionary model merging framework that explicitly optimizes accuracy and output length to recover a Pareto front of merged models. To make this search computationally tractable, we propose an entropy-based subset sampling technique that substantially reduces fitness-estimation overhead. Across six mathematical reasoning benchmarks, Evo-L2S reduces reasoning length by over 50% at the 1.5B and 7B scales while preserving or improving problem-solving accuracy; at 14B, the steeper Pareto front reveals that attainable compression depends on the intrinsic compressibility of the reasoning model. Overall, Evo-L2S shows that reasoning models can be made substantially more concise while preserving strong problem-solving performance.

Figures & tables

Figure 1: Overview of the Evo-L2S pipeline. Left: Entropy-based sampling constructs an informative evaluation subset. Right: Multi-objective evolutionary search (NSGA-II) optimizes reasoning accuracy and output length to produce a Pareto front of merged models.

Figure 2: Accuracy (%) vs. length reduction (%) relative to DeepSeek-R1-Distill-Qwen at the 1.5B (left) and 7B (right) scales, averaged across six reasoning benchmarks. The dashed line marks the DeepSeek-R1 accuracy baseline. Connected points form the Pareto fronts found by Evo-L2S ; transparent points are Pareto-optimal on the fitness subset but dominated on the full evaluation. The optimal region is in the upper right corner.

Scale	System 1	System 2
1.5B	Qwen2.5-Math-1.5B	DeepSeek-R1-Distill-Qwen-1.5B
7B	Qwen2.5-Math-7B	DeepSeek-R1-Distill-Qwen-7B
14B	Qwen2.5-14B	DeepSeek-R1-Distill-Qwen-14B

Table 1: Model pairs used in our experiments, sharing Qwen architectures.

Figure 3: Entropy-based subset sampling for simulated ranking. (a) Calibration correctness matrix: each row is one of $K=10$ candidates, each column a problem sorted by empirical solve rate $\hat{p}_{i}$ ; the colour strip below shows the corresponding Bernoulli entropy $H_{i}$ . We generate this matrix simulating common patterns observed in correctness matrices, in order to avoid test contamination. (b) Entropy curve $H_{i}=-p_{i}\log_{2}p_{i}-(1{-}p_{i})\log_{2}(1{-}p_{i})$ ; individual problems are scattered at $(\hat{p}_{i},H_{i})$ , with selected items ( $H_{i}$ above the dashed threshold) shown in blue. (c) Spearman rank correlation $\rho$ between subset-induced and full-benchmark model rankings as a function of subset size $|\mathcal{S}|$ . Entropy sampling reaches high fidelity at $|\mathcal{S}|=50$ , clearly outperforming the baselines.

Method	AIME 2024	College Math	GSM8K	MATH-500	Minerva Math	OlympiadBench	MATH	Average
Entropy	$\mathbf{84.7\pm 1.0}$	$13.7\pm 3.0$	$87.1\pm 2.0$	$\mathbf{90.6\pm 1.5}$	$\mathbf{82.4\pm 2.5}$	$\mathbf{89.7\pm 1.3}$	$\mathbf{92.4\pm 0.8}$	$\mathbf{77.2}$
Disagreement	$80.6\pm 2.2$	$\mathbf{21.3\pm 3.7}$	$\mathbf{87.6\pm 1.8}$	$88.8\pm 1.8$	$79.1\pm 2.7$	$89.1\pm 1.7$	$91.1\pm 2.1$	$76.8$
Random	$76.1\pm 4.3$	$19.1\pm 5.3$	$81.0\pm 3.8$	$83.9\pm 2.7$	$75.6\pm 2.9$	$83.8\pm 3.2$	$85.6\pm 3.4$	$72.2$

Table 2: Validation of entropy-based sampling for actual rankings. We report Spearman’s rank correlation between 7B model rankings estimated from 5% of MATH and rankings on each full benchmark. Correlation values are computed as

100\rho

; uncertainties are 95% confidence-interval half-widths across 30 calibration-pool resamples.

Figure 4: Benchmark-wise accuracy and length reduction of Evo-L2S (best trade-off) vs. DeepSeek-R1-Distill-Qwen at the 1.5B (left) and 7B (right) scales. Annotations above Evo-L2S bars indicate accuracy difference ( improvement or degradation ) relative to DeepSeek-R1 . The line reports per-benchmark and average output-length reduction (%).

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

	1.5B	7B	14B
Population size	20	20	12
Generations	10	10	8
Candidate evaluations	200	200	96
Items per evaluation	50	50	50
Approx. forward passes	10,000	10,000	4,800
A100 GPU-hours	7.5	14.5	48

Appendix

Table 3: Evolutionary-search compute cost by model scale.

Method	AIME 2024	College Math	GSM8K	MATH-500	Minerva Math	OlympiadBench	MATH	Average
Entropy	$\mathbf{49.7\pm 3.8}$	$-9.5\pm 1.5$	$\mathbf{82.8\pm 1.1}$	$\mathbf{90.7\pm 1.2}$	$29.3\pm 2.4$	$\mathbf{90.1\pm 1.1}$	$\mathbf{90.4\pm 1.3}$	$\mathbf{60.5}$
Disagreement	$38.4\pm 6.7$	$0.4\pm 7.0$	$81.1\pm 3.3$	$83.9\pm 3.6$	$34.4\pm 6.1$	$83.9\pm 3.1$	$83.9\pm 3.7$	$58.0$
Random	$34.2\pm 6.6$	$\mathbf{1.2\pm 7.3}$	$76.2\pm 5.4$	$77.7\pm 5.7$	$\mathbf{34.5\pm 8.2}$	$78.4\pm 5.9$	$77.6\pm 5.9$	$54.3$

Appendix

Table 4: Spearman rank correlation between 1.5B model rankings estimated from 5% of MATH and rankings on each full target benchmark. Values are reported as

100\rho

; uncertainties are 95% confidence-interval half-widths across 30 calibration-pool resamples.

Panel of Table 5 — Table 5: Results of model merging methods at the 1.5B scale.

Panel of Table 6 — Table 6: Results of model merging methods at the 7B scale. Grid and random search use the same evaluation budget as Evo-L2S. For these static-search baselines, we apply the same non-dominated sorting procedure as Evo-L2S to select a Pareto-optimal checkpoint from the evaluated candidates.

Panel of Table 7 — Table 7: Results of all model merging methods at the 14B scale.

Problem	DeepSeek-R1-Distill-Qwen-7B	Evo-L2S-02
MATH500 IDX=16	Okay, so I have this math problem here: Compute $1-2+3-4+5-\dots+99-100$ . Hmm, let me try to figure this out step by step. First, I notice that the series alternates between adding and subtracting consecutive numbers. It starts with 1, then subtracts 2, adds 3, subtracts 4, and so on, all the way up to subtracting 100. So, the pattern is $+1,-2,+3,-4,\dots,+99,-100$ . I wonder if there’s a way to group these terms to make it easier to compute. Maybe if I pair them up? Let me try that. If I group the first two terms: $1-2$ , that equals $-1$ . Then the next two terms: $3-4$ , which is also $-1$ . Hmm, interesting. Let me check the next pair: $5-6=-1$ . It seems like every pair of consecutive numbers adds up to $-1$ . So, if each pair is $-1$ , how many such pairs are there in the series? Well, the series goes from 1 to 100, so that’s 100 terms in total. If I pair them two at a time, that would be $100/2=50$ pairs. Therefore, if each pair is $-1$ , the total sum should be $50\times(-1)=-50$ . Wait, let me make sure I didn’t make a mistake here. So, starting from 1, subtracting 2, adding 3, subtracting 4, etc., up to 100. So, each pair is indeed two numbers, one positive and one negative, with the positive being odd and the negative being even. Let me write out the first few pairs to confirm:1 - 2 = -13 - 4 = -15 - 6 = -17 - 8 = -1…99 - 100 = -1. Yes, each pair is -1, and there are 50 such pairs because 100 divided by 2 is 50. So, 50 times -1 is -50. Is there another way to think about this? […] the final answer is: − 50	To solve the problem $1-2+3-4+5-\dots+99-100$ , we can break it down into manageable steps. 1. Understand the Pattern: The sequence alternates between positive and negative terms. Specifically, the terms are $1,-2,3,-4,5,-6,\dots,99,-100$ . 2. Group the Terms: We can group the terms in pairs: $(1-2),(3-4),(5-6),\dots,(99-100)$ . Each pair sums to $-1$ . 3. Count the Pairs: Since we are pairing every two terms, and there are 100 terms in total, there will be $100/2=50$ pairs. 4. Sum the Pairs: Since each pair sums to $-1$ , the total sum will be $50\times(-1)=-50$ . Let’s implement this in Python to verify our reasoning.“‘python# Calculate the sum of the series 1 - 2 + 3 - 4 + 5 - … + 99 - 100 total_sum = 0# Iterate through the series for i in range(1, 101): if i % 2 == 0: # if i is even, subtract it total_sum -= i else: # if i is odd, add it total_sum += i print(total_sum)“““output-50“‘ The sum of the series $1-2+3-4+5-\dots+99-100$ is $\boxed{-50}$ .

Appendix

Table 8: Illustrative reasoning-trace comparison. Both models produce the correct answer. DeepSeek-R1-Distill-Qwen-7B ’s output is shortened by 3317 characters for readability. For both generations we remove special characters (e.g., newline).

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 356 papers

ExpThink: Experience-Guided Reinforcement Learning for Adaptive Chain-of-Thought Compression
May 8, 2026Tingcheng Bian, Yuzhe Zhang, Jing Jin +5Reward Shaping RL for Language Model Reasoning
1Baidu Inc. · 2Shenzhen University · 3Peking University +2
Shorthand for Thought: Compressing LLM Reasoning via Entropy-Guided Supertokens
Apr 29, 2026Zhenyu Zhao, Sander Land, Daniel M. Bikel +1LLM Compression LLM Interpretability
Writer, Inc.
Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning
Apr 10, 2026Yi Sui, Chaozhuo Li, Dawei SongCoT Compression Efficient Language Model Reasoning
Beijing Institute of Technology, China · Beijing University of Posts and Telecommunications, China · The Open University, UK

May 8, 2026J/K move · Enter open · S save

ExpThink: Experience-Guided Reinforcement Learning for Adaptive Chain-of-Thought Compression

Tingcheng Bian, Yuzhe Zhang, Jing Jin, Jinchang Luo+4

1Baidu Inc. · 2Shenzhen University · 3Peking University+2

Reward Shaping RL for Language Model Reasoning Efficient Language Model Reasoning Reward Design for RL CoT Compression Large Reasoning Models

Open paper AskPDF arXiv

Large reasoning models (LRMs) achieve strong performance via extended chain-of-thought (CoT) reasoning, yet suffer from excessive token consumption and high inference latency. Existing reinforcement learning (RL) approaches for CoT compression rely on uniform, static length penalties that neglect model capability dynamics and problem-level difficulty variation. We propose ExpThink\xspace, an RL framework that addresses both dimensions through two complementary mechanisms. First, experience-guided reward shaping tracks the shortest correct solution found so far for each problem and applies a three-tier reward: full credit for concise correct responses, discounted credit for verbose correct ones, and zero for incorrect ones. The threshold tightens automatically with model improvement, forming a self-evolving curriculum that requires no manual scheduling. Second, difficulty-adaptive advantage replaces standard deviation normalization with correct-count normalization, yielding monotonically difficulty-scaled gradients that amplify learning on hard problems to preserve accuracy while suppressing gradients on easy ones to encourage brevity. Together, these mechanisms enforce an accuracy-first, compression-second training objective. Experiments on multiple mathematical reasoning benchmarks demonstrate that ExpThink\xspace reduces average response length by up to 77% while simultaneously improving accuracy, achieving up to

3\times

higher accuracy-efficiency ratio (accuracy divided by average token count) than the vanilla baseline and outperforming existing RL-based compression methods on both metrics.

Abstract

Efficient Language Model Reasoning

Explore similar work

ExpThink: Experience-Guided Reinforcement Learning for Adaptive Chain-of-Thought Compression

Shorthand for Thought: Compressing LLM Reasoning via Entropy-Guided Supertokens

Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning