Organizations: School of Computer Science, Peking University · Qwen Team, Alibaba Group · EPIC Lab, SJTU · College of Computer Science and Technology, Zhejiang University.
Large Language Models (LLMs) excel at programming tasks but frequently fail at deterministic, fine-grained reasoning in natural language, relying heavily on semantic approximations rather than robust symbolic execution. To bridge this gap, we propose MIMIC, a framework that leverages executable code as a rigorous medium for reasoning data synthesis. MIMIC fundamentally transforms algorithms into verifiable reasoning trajectories through narrative fusion, code-guided test synthesis, and dynamic code instrumentation. Crucially, these explicit intermediate execution states naturally form a Code-Instrumented Reward (CIR), providing dense, high-fidelity process supervision for reinforcement learning without external reward models. Extensive evaluations reveal that models trained via SFT and GRPO on our synthesized dataset achieve substantial, consistent gains. Our method significantly elevates accuracy across general reasoning, complex mathematical benchmarks, and fine-grained deterministic tasks, demonstrating that the procedural rigor of executable code can effectively unlock and enhance the generalized reasoning capabilities of LLMs. Our code and data are available at https://github.com/zjy1298/MIMIC.
Figures & tables
Figure 1: The paradox of LLMs: while they excel at complex programming logic, they often fail at basic deterministic natural language reasoning. MIMIC bridges this gap by leveraging executable code to generate hallucination-mitigated reasoning trajectories.
Figure 2: Overview of the MIMIC framework. It transforms raw algorithmic problems into verifiable reasoning data through a four-stage pipeline: (1) Problem Classification and Mathematical Kernel Extraction, (2) Problem Synthesis via Narrative Fusion, (3) Code-Guided Test Synthesis and Narrative Input Fusion, and (4) Trace-Grounded Chain-of-Thought (CoT) Synthesis.
Model
Setting
General Reasoning
Math
Code
ARC
BBH
MIMIC
GPQA
MMLU
DROP
Avg
GSM8K
GSM+
MATH
AIME
Avg
HEval
CruxE
MBPP
Avg
(A) Main Results: Baseline → Ours (SFT) / Ours (GRPO)
Qwen3-4B
Base
79.6
76.0
40.6
40.1
66.3
50.5
58.9
89.8
73.2
78.3
15.6
64.2
31.7
69.3
39.7
46.9
Ours (SFT)
85.9
77.0
43.5
40.4
67.3
55.5
61.6 ↑ 2.7
91.0
74.9
77.6
16.7
65.0 ↑ 0.8
32.1
70.8
38.8
47.2 ↑ 0.3
Ours (GRPO)
80.6
77.9
51.3
40.8
67.5
58.3
62.7 ↑ 3.8
91.0
76.4
80.3
28.3
69.0 ↑ 4.8
31.7
72.7
43.6
49.3 ↑ 2.4
Qwen3-8B
Base
79.8
78.8
41.2
44.4
70.5
46.6
60.2
90.6
73.7
79.2
23.3
66.7
22.6
70.6
40.5
44.6
Table 1: Main Results and Ablation Studies. (A) Main results comparing base models with our MIMIC data via SFT and GRPO. (B) Data source ablation: our contextualized synthesis vs. raw Codeforces problems. (C) Cross-domain mixing with mathematical/benchmark data. (D) RL algorithm robustness. Avg = category mean; subscripts show absolute change vs. the reference row. Best per group in bold .
Model
Setting
General Reasoning
Math
Code
ARC
BBH
MIMIC
GPQA
MMLU
DROP
Avg
GSM8K
GSM+
MATH
AIME
Avg
HEval
CruxE
MBPP
Avg
Comparison with Closely Related Baselines
(A) SFT
Qwen3-4B
Ours (SFT)
85.9
77.0
43.5
40.4
67.3
55.5
61.6
91.0
74.9
77.6
16.7
65.0
32.1
70.8
35.8
46.2
CoE (SFT)
85.2
43.9
28.9
25.4
44.8
42.6
45.1 ↓ 16.5
71.5
54.9
40.4
6.7
43.4 ↓ 21.6
34.2
65.6
34.6
44.8 ↓ 1.4
ExecGrounded (SFT)
75.4
66.9
39.2
35.0
40.8
26.1
47.2 ↓ 14.4
77.6
63.4
74.4
16.7
58.0 ↓ 7.0
23.8
63.0
45.5
44.1 ↓ 2.1
Table 2: Comparison with closely related baselines. We compare MIMIC against representative recent methods under both SFT and GRPO. Avg = category mean; subscripts show absolute change vs. the reference row. Best per group in bold .
Model
Method
Maze
String
Char
Char+CoT
Qwen3-4B
Base
16.0
13.6
37.9
97.1
+SFT
18.0
22.8
40.4
97.8
+GRPO
22.0
26.8
39.1
98.2
Qwen3-8B
Base
22.0
26.2
38.6
98.0
+SFT
22.0
37.5
39.3
98.6
+GRPO
40.0
39.5
41.3
99.2
Table 3: Character-level and spatial reasoning benchmarks. We evaluate on benchmarks targeting fine-grained perception and rules (e.g., comparing string patterns like 9.11 and 9.9): CharBench, StringBench, and MazeBench. Best per model family in bold .
Format
Acc. (%)
Code Rate (%)
Acc. (%) w/ Code
Acc. (%) w/o Code
Original
23.8
38.7
18.4
27.2
Refined
34.4
29.9
30.6
36.0
Fused (Ours)
42.2
0.7
15.0
42.4
Synthesized
61.8
0.8
41.3
62.0
Table 4: Preliminary comparison of different problem presentation formats on Qwen3-8B . “Code Rate” denotes the proportion of cases where the model attempted to write code during reasoning.
Model
Reward
General Reasoning
Math
Code
ARC
BBH
MIMIC
GPQA
MMLU
DROP
Avg
GSM8K
GSM+
MATH
AIME
Avg
HEval
CruxE
MBPP
Avg
Qwen3-8B
Base
79.8
78.8
41.2
44.4
70.5
46.6
60.2
90.6
73.7
79.2
23.3
66.7
22.6
70.6
40.5
44.6
Binary (GRPO)
81.2
81.7
52.2
42.6
72.2
53.1
63.8 ↑ 3.6
91.7
78.1
81.0
26.7
69.4 ↑ 2.7
23.8
76.3
44.0
48.0 ↑ 3.4
CIR (Ours)
82.7
80.6
54.5
48.5
71.6
53.5
65.2 ↑ 5.0
92.6
78.0
81.4
30.0
70.5 ↑ 3.8
25.3
76.2
40.9
47.5 ↑ 2.9
PRM800K
79.8
79.3
43.8
42.7
71.6
50.9
61.4 ↑ 1.2
91.1
74.7
80.5
23.3
67.4 ↑ 0.7
23.2
71.8
40.5
45.2 ↑ 0.6
Math-Shepherd
81.4
79.8
43.9
43.1
71.4
52.7
62.1 ↑ 1.9
90.9
74.3
75.9
20.0
65.3 ↓ 1.4
21.3
72.3
44.4
46.0 ↑ 1.4
Table 5: Reward Signal Ablation. Comparing reward strategies under GRPO training: (1) Binary outcome reward; (2) CIR (Code-Instrumented Reward), our approach leveraging execution checkpoints; (3) PRM800K ; (4) Math-Shepherd . Subscripts show absolute change vs. Base.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Type
Category
Subtype
Description
Type 1
Text-Encoded Visual / Spatial Reasoning
1.1 Grid / board / pixel encoding
Layouts represented as character matrices or ASCII art, where the solver must reconstruct a 2D structure from text and reason about adjacency, connectivity, or shape.
1.2 Geometry / coordinates / directions
Spatial reasoning grounded in language, such as left/right/up/down relations, segment intersection, coordinate movement, or point-in-polygon logic.
1.3 Glyph or compositional character structure
Reasoning based on visual properties of symbols or characters, including radicals, stroke counts, mirrored forms, rotated shapes, or internal composition.
1.4 Textual charts / tabular layout understanding
Information encoded through alignment, indentation, spacing, or fixed-width formatting, such as tree structures or visually organized tables.
Fine-grained operations on strings, including counting symbol frequency, extracting characters at specified positions, or performing local edits and substitutions.
2.2 Character-wise mapping and ciphers
Per-character transformation rules such as Caesar shifts, ASCII-based conversion, or Base64-style encoding and decoding.
Appendix
Table 6: Reasoning taxonomy used in Stage 1. Each problem is assigned to the category that best reflects its dominant reasoning bottleneck.
Benchmark
HF Dataset
Dataset Summary
Primary Abilities Tested
Mathematical Reasoning
GSM8K
gsm8k
Grade-school math word problems (1,319 test) requiring 2–8 steps of elementary arithmetic Cobbe et al. (2021) .
Adversarially augmented GSM8K Li et al. (2024) with numerical perturbation, structural rephrasing, and distractors to reduce shortcut biases.
Robust numerical reasoning under perturbation.
MATH
competition_math
5,000 competition-level problems across 7 subjects (algebra, geometry, number theory, etc.) drawn from AMC/AIME contests Hendrycks et al. (2021) .
Formal math reasoning; symbolic manipulation; multi-step proofs.
AIME
AIME_2024 AIME_2025
60 problems from AIME 2024 and AIME2025 (I+II) Zhang et al. (2023) ; integer answers in 0–999 requiring creative multi-step reasoning in algebra, combinatorics, and geometry.
Competition-level mathematical problem solving.
General Reasoning & Knowledge
Appendix
Table 7: Benchmarks used in our evaluation, grouped into four categories.
Hyperparameter
Value
Fine-tuning type
Full parameter
Parallelism
DeepSpeed ZeRO-3
Number of GPUs
8
Per-device batch size
2
Gradient accumulation steps
8
Effective batch size
128
Appendix
Table 8: SFT training hyperparameters.
Hyperparameter
Value
Algorithm
GRPO
Parallelism
FSDP
Number of GPUs
8
Rollout engine
vLLM
Rollout samples per prompt ( n )
8
Train batch size
128
Appendix
Table 9: GRPO training hyperparameters.
Figure 3: Hierarchical distribution of Codeforces problems in the Stage-I taxonomy. The inner ring denotes the five top-level categories and the outer ring denotes the corresponding fine-grained subtypes. Colors indicate top-level categories, while lighter shades represent their subtypes. Sector sizes are normalized within each level for visual clarity, and the actual numbers of instances are shown inside the segments.
Top-level Category
Subtype (abbrev.)
Count
Top-level Share (%)
Subtype Share (Parent, %)
Type 1: Text-Encoded Visual/Spatial
1.1 Grid/ASCII Parsing
135
2.07
65.2
1.2 Geom./Rel. Grounding
69
33.3
1.4 Layout-aware Tables/Diag.
2
1.0
1.3 Glyph Decomp.
1
0.5
Type 2: Sub-Token/ Character
2.1 Char Count/Extract/Replace
527
6.47
81.3
2.2 Char Mapping/Crypto.
61
9.4
Appendix
Table 10: Hierarchical distribution of the cleaned Stage-I dataset. “Top-level Share” is computed over all 10,008 instances, and “Subtype Share (Parent)” denotes the proportion within each top-level category. Abbreviated subtype names are used for space efficiency.
Rating
Original
Refined
Fused
Synthesized
<1200
27.7
20.2
0.4
0.6
1200–1599
46.3
34.9
0.6
0.8
1600–1999
49.2
38.1
0.8
0.9
≥2000
44.9
35.5
0.8
0.7
Appendix
Table 11: Code-writing rate in model reasoning across different Codeforces rating bands under different problem presentation formats. Lower is better.
Rating
Original
Refined
Fused
Synthesized
Δ
<1200
46.6
55.1
48.6
72.3
−6.6
1200–1599
32.2
44.4
45.8
68.6
+1.4
1600–1999
19.3
30.6
41.4
65.1
+10.8
≥2000
13.0
24.4
40.6
56.9
+16.2
Appendix
Table 12: Accuracy (%) across different Codeforces rating bands under different problem presentation formats. The last column Δ reports the accuracy gain of Fused over Refined .
Type
Accuracy (%)
Total
Code Rate (%)
Visual/Spatial
37.2
2328
0.5
Char-Level
47.2
8045
1.2
Math
44.7
54121
0.9
Data Process
53.5
6011
0.5
Rule/State
37.2
57090
0.6
Appendix
Table 13: Model performance across top-level taxonomy types under the fused setting.
Rating
Official
Level A
Level B
Level C
<1200
52.4
51.8
48.3
42.8
1200–1599
44.8
51.9
48.0
38.4
1600–1999
36.1
50.7
45.1
33.1
2000–2399
31.5
52.0
46.0
32.2
≥2400
24.1
48.4
42.2
28.0
Appendix
Table 14: Accuracy (%) across different test sources and Codeforces rating bands.
Method
Unshifted
Marginal
Shifted
SFT vs Base
78.9%
15.6%
5.5%
GRPO vs Base
95.1%
4.8%
0.2%
Appendix
Table 15: Shifted token distribution under different alignment methods, measured against the base model. We report the proportion of unshifted, marginal, and shifted positions.
Figure 4: Word cloud of shifted tokens under SFT, measured relative to the base model. The shifted vocabulary is dominated by instruction- and description-level terms such as requested , allowed , sequence , algorithm , and grid , suggesting that SFT learns substantial surface-level regularities of the training data in addition to task behavior.
Figure 5: Word cloud of shifted tokens under GRPO, measured relative to the base model. Although the number of shifted positions is much smaller, the shifted vocabulary is more concentrated on reasoning-relevant terms such as count , sum , configuration , operation , and determined , indicating more targeted changes to computation and decision-making behavior.
Figure 6: GRPO training dynamics for Qwen3-8B on Dmixed . (a) Mean reward increases steadily from 0.61 to 0.81, indicating consistent learning. (b) KL divergence remains bounded below 0.004 (smoothed), confirming stable policy updates without reward hacking. (c) Mean response length grows from ∼ 850 to ∼ 1,330 tokens, reflecting the emergence of more thorough reasoning chains.
Figure 7: Benchmark scaling during GRPO training (Qwen3-8B, Dmixed ). All four benchmarks show consistent improvement over 190 training steps. Dashed lines indicate base model performance. MIMIC improves by +9.8%, demonstrating effective transfer from code-mediated reasoning to general problem-solving. Notably, MBPP (+8.9%) also improves substantially, suggesting that reasoning-focused training can reciprocally enhance code generation capabilities.
Figure 8: SFT training and validation loss for Qwen3-8B on Dmixed . The training loss drops sharply from 0.92 to ∼ 0.45 within the first 400 steps, then gradually converges to 0.39. The validation loss closely follows, plateauing at 0.39 with minimal train-validation gap.
Figure 9: Benchmark evolution during SFT (Qwen3-8B, Dmixed ). Downstream benchmarks show quick initial gains that plateau or slightly decline after step 800, contrasting sharply with the continuous improvement observed under GRPO (Figure 7 ).
Model
Setting
General Reasoning
Math
Code
ARC
BBH
MIMIC
GPQA
MMLU
DROP
Avg
GSM8K
GSM+
MATH
AIME
Avg
HEval
CruxE
MBPP
Avg
Qwen3-8B
Base
79.8
78.8
41.2
44.4
70.5
46.6
60.2
90.6
73.7
79.2
23.3
66.7
22.6
70.6
40.5
44.6
Qwen3-8B
Ours (All)
81.2
81.7
52.2
42.6
72.2
53.1
63.8 ↑ 3.6
91.7
78.1
81.0
26.7
69.4 ↑ 2.7
23.8
76.3
44.0
48.0 ↑ 3.4
(A) Single-Type Training: GRPO with only one task category
Qwen3-8B
Type 1 (Visual)
79.3
78.8
43.0
45.2
71.1
50.8
61.4 ↑ 1.2
90.3
74.6
79.9
26.7
67.9 ↑ 1.2
22.9
71.4
41.6
45.3 ↑ 0.7
Type 2 (Char)
80.0
79.7
42.9
46.1
71.4
51.5
61.9 ↑ 1.7
92.0
75.4
81.1
28.3
69.2 ↑ 2.5
22.9
70.4
43.6
45.6 ↑ 1.0
Appendix
Table 16: Task Type Ablation (Qwen3-8B). (A) Single-type training : GRPO trained with only one task category. (B) Leave-one-out : GRPO trained with all categories except one. Subscripts on Avg show absolute change vs. Base. Bold = best, underline = second best within each section.
Figure 10: LOO Impact on category averages (Reasoning, Math, Code). Positive values (green) indicate beneficial contribution; negative values (red) indicate interference when included.
Figure 11: Per-benchmark LOO Impact. Each cell shows the accuracy drop (in percentage points) when the corresponding task type is removed from training. Horizontal lines separate the three evaluation categories: Reasoning, Math, and Code.
Code reasoning refers to the task of predicting the output of a program given its source code and specific inputs. It can measure the reasoning capability of large language models (LLMs) and also benefit downstream tasks such as code generation and mathematical reasoning. Existing work has verified the effectiveness of reinforcement learning on the task. However, these methods design rewards solely based on final outputs or coarse-grained signals, and neglect the inherent consistency of the stepwise reasoning process in the task. Therefore, these methods often result in sparse reward or reward hacking, which limits the full play of enhanced learning capabilities. To alleviate these issues, we propose CodeThinker, a consistency-driven reinforcement learning framework for code reasoning. Specifically, CodeThinker has three key components: (1) a stepwise reasoning-aware model training module, which utilizes a consistency tracing paradigm as a template to synthesize training data that captures the stepwise reasoning process; (2) a dynamic beam sampling strategy, which aims to improve the quality of sampled outputs under a fixed sampling budget; and (3) a consistency reward mechanism that can effectively alleviate reward hacking. Experiments on three popular benchmarks show that CodeThinker achieves state-of-the-art performance across multiple LLMs. For instance, it outperforms the strongest baseline by 4.3% in accuracy when deployed on Qwen2.5-Coder-7B-Instruct. We also validate the effectiveness of CodeThinker on downstream tasks. Results show that, without additional training, CodeThinker obtains average accuracy gains of 5.33 and 3.11 percentage points on mathematical reasoning and code reasoning tasks covering 17 programming languages, respectively.
Zhanyue Qin, Jia Feng, Yibo Lyu +4
Harbin Institute of Technology · The Chinese University of Hong Kong
Reinforcement Learning (RL) has been shown to significantly boost reasoning capabilities of large language models (LLMs) in math, coding, and multi-hop reasoning tasks. However, RL fine-tuning requires abundant high-quality verifiable data, often sourced from human annotations, generated from frontier LLMs, or scored by LLM-based verifiers. All three have considerable limitations: human-annotated datasets are small and expensive to curate, LLM-generated data is hallucination-prone and costly, and LLM-based verifiers are inaccurate and slow. In this work, we investigate a cheaper alternative: RL fine-tuning on rule-generated synthetic data for multi-hop reasoning tasks. We discover that LLMs fine-tuned on synthetic data perform significantly better on popular real-world question-answering benchmarks, despite the synthetic data containing only fictional knowledge. On stratifying performance by question difficulty, we find that synthetic data teaches LLMs to compose knowledge -- a fundamental and generalizable reasoning skill. Our work highlights rule-generated synthetic reasoning data as a free and scalable resource to improve LLM reasoning capabilities.
Anmol Kabra, Yilun Yin, Albert Gong +6
Cornell University · University of Cambridge · Stanford University
Recent advancements in large language models (LLMs) have catalyzed the rise of reasoning-intensive inference paradigms, where models perform explicit step-by-step reasoning before generating final answers. While such approaches improve answer quality and interpretability, they incur substantial computational overhead due to the prolonged generation sequences. In this paper, we propose Tandem, a novel collaborative framework that synergizes large and small language models (LLMs and SLMs) to achieve high-quality reasoning with significantly reduced computational cost. Specifically, the LLM serves as a strategic coordinator, efficiently generating a compact set of critical reasoning insights. These insights are then used to guide a smaller, more efficient SLM in executing the full reasoning process and delivering the final response. To balance efficiency and reliability, Tandem introduces a cost-aware termination mechanism that adaptively determines when sufficient reasoning guidance has been accumulated, enabling early stopping of the LLM's generation. Experiments on mathematical reasoning and code generation benchmarks demonstrate that Tandem reduces computational costs by approximately 40% compared to standalone LLM reasoning, while achieving superior or competitive performance. Furthermore, the sufficiency classifier trained on one domain transfers effectively to others without retraining. The code is available at: https://github.com/Applied-Machine-Learning-Lab/ACL2026_Tandem.
Zichuan Fu, Xian Wu, Guojing Li +7
City University of Hong Kong · Tencent Jarvis Lab · Renmin University of China +1