Planning has been identified as an important factor for improved code generation with Large Language Models (LLMs). However, the interaction between planning capability and coding capability remains largely underexplored, especially when the two capabilities are from different LLMs. In this paper, we conduct the first comprehensive empirical study of planning-coding interaction in LLM-based code generation, where planning and coding are assigned to different LLMs (i.e., planning-coding separation). Specifically, we explore different planner-coder combinations to investigate the effectiveness for coders to utilize planning information from various planners and the impact of further training of coders on planner-coder combinations. Our results demonstrate that almost all planner-coder combinations are as competitive as using the same LLM for planning and coding, and the combination of a stronger planner and a stronger coder typically achieves improved effectiveness. Furthermore, using a large model as the planner paired with a much smaller model as the coder can sometimes be as competitive as using the large model as both the planner and the coder, and further training of the coder can typically boost various planner-coder combinations. These findings provide empirical evidence that planning and coding can be separated to a considerable extent in LLM-based code generation, which may offer a more flexible way to construct code-generation systems.
Figures & tables
Figure 1. Planning prompt. Planning prompt.
Coder
Direct
Planner
3B–4B
Qwen3-4B Gemma-3-4B-it Ministral-3-3B-Instruct
Qwen3-4B
24.57
24.95 23.43 (-6.1%) 21.52 (-13.7%)
Gemma-3-4B-it
18.86
19.62 (+14.4%) 17.14 17.71 (+3.3%)
Ministral-3-3B-Instruct
23.43
26.48 (+36.3%) 24.19 (+24.5%) 19.43
8B
Qwen3-8B Ministral-3-8B-Instruct
Qwen3-8B
24.57
26.29 23.62 (-10.1%)
Table 1. Planner–coder pairing results on LiveCodeBench (Pass@1). Parentheses show the relative change from the same-model pair for each coder. Bold marks the best plan-assisted result for each coder; gray marks the highest-scoring pair in each model group.
Coder
Direct
Planner
3B–4B
Qwen3-4B Gemma-3-4B-it Ministral-3-3B-Instruct
Qwen3-4B
24.88
22.58 18.59 (-17.7%) 17.82 (-21.1%)
Gemma-3-4B-it
21.20
22.43 (+23.7%) 18.13 16.59 (-8.5%)
Ministral-3-3B-Instruct
24.88
23.96 (+30.0%) 19.97 (+8.3%) 18.43
8B
Qwen3-8B Ministral-3-8B-Instruct
Qwen3-8B
24.88
27.04 23.20 (-14.2%)
Table 2. Planner–coder pairing results on Code2Bench (Pass@1). Parentheses show the relative change from the same-model pair for each coder. Bold marks the best plan-assisted result for each coder; gray marks the highest-scoring pair in each model group.
Figure 2. Planner performance across model scales with three small coders. Q3, Gem, and Min abbreviate Qwen3, Gemma-3-it, and Ministral-3-Instruct-2512; DS Flash/Pro abbreviate DeepSeek-V4.1-Flash/V4-Pro, and GPT abbreviates GPT-5.6-Sol. Six panels compare planner models with fixed Qwen3-4B, Gemma-3-4B-it, and Ministral-3-3B-Instruct-2512 coders, arranged in three rows. The left column shows LiveCodeBench and the right column shows Code2Bench. The horizontal axis lists planners and the vertical axis shows Pass@1 percentages. Lines connect Qwen3, Gemma-3, and Ministral-3 planners of different sizes. Separate points show Qwen3.5-27B, DeepSeek-V4.1-Flash, DeepSeek-V4-Pro, and GPT-5.6-Sol. Dashed horizontal lines mark direct generation by each coder. Increasing planner size generally improves small-coder performance, with exceptions among Qwen3 pairings.
Figure 3. Planner performance across model scales with three larger coders. Q3, Gem, and Min abbreviate Qwen3, Gemma-3-it, and Ministral-3-Instruct-2512; DS Flash/Pro abbreviate DeepSeek-V4.1-Flash/V4-Pro, and GPT abbreviates GPT-5.6-Sol. Six panels compare planner models with fixed Qwen3-32B, Gemma-3-27B-it, and Ministral-3-14B-Instruct-2512 coders, arranged in three rows. The left column shows LiveCodeBench and the right column shows Code2Bench. The horizontal axis lists planners and the vertical axis shows Pass@1 percentages. Lines connect Qwen3, Gemma-3, and Ministral-3 planners of different sizes. Separate points show Qwen3.5-27B, DeepSeek-V4.1-Flash, DeepSeek-V4-Pro, and GPT-5.6-Sol. Dashed horizontal lines mark direct generation by each coder. Within-family scaling trends vary by coder; Qwen3.5 and the advanced planners exceed the direct baselines in all six panels.
Coder
Direct
Planner
Qwen3.5-27B
DeepSeek-V4.1-Flash
DeepSeek-V4-Pro
GPT-5.6-Sol
Qwen3-4B
24.57
37.71 (+53.5%)
40.95 (+66.7%)
42.67 (+73.6%)
50.10 (+103.9%)
Qwen3-8B
24.57
39.05 (+58.9%)
42.29 (+72.1%)
43.24 (+76.0%)
51.24 (+108.5%)
Qwen3-14B
28.00
40.57 (+44.9%)
44.38 (+58.5%)
46.86 (+67.3%)
55.24 (+97.3%)
Qwen3-32B
27.43
40.19 (+46.5%)
44.57 (+62.5%)
42.10 (+53.5%)
53.52 (+95.1%)
Gemma-3-4B-it
18.86
25.14 (+33.3%)
27.24 (+44.4%)
28.00 (+48.5%)
26.10 (+38.4%)
Table 3. Coder performance across model scales with four planners on LiveCodeBench (Pass@1). Parentheses show the relative change from Direct for each coder. Bold marks the best planner for each coder; gray marks the best coder for each planner within each group.
Coder
Direct
Planner
Qwen3.5-27B
DeepSeek-V4.1-Flash
DeepSeek-V4-Pro
GPT-5.6-Sol
Qwen3-4B
24.88
30.57 (+22.8%)
32.10 (+29.0%)
32.87 (+32.1%)
34.56 (+38.9%)
Qwen3-8B
24.88
31.80 (+27.8%)
32.41 (+30.2%)
34.25 (+37.7%)
34.87 (+40.1%)
Qwen3-14B
32.26
33.64 (+4.3%)
33.95 (+5.2%)
35.02 (+8.6%)
37.94 (+17.6%)
Qwen3-32B
29.49
33.18 (+12.5%)
34.10 (+15.6%)
35.48 (+20.3%)
37.94 (+28.6%)
Gemma-3-4B-it
21.20
27.50 (+29.7%)
28.42 (+34.1%)
30.41 (+43.5%)
29.80 (+40.6%)
Table 4. Coder performance across model scales with four planners on Code2Bench (Pass@1). Parentheses show the relative change from Direct for each coder. Bold marks the best planner for each coder; gray marks the best coder for each planner within each group.
Coder
Setting
Direct
Planner
Qwen3.5-27B
DeepSeek-V4.1-Flash
DeepSeek-V4-Pro
GPT-5.6-Sol
Qwen2.5-Coder-1.5B
Base
10.86
23.05
24.95
25.90
28.00
SFT
13.71
24.19
24.95
27.43
28.95
Relative Change (%)
+26.3
+5.0
+0.0
+5.9
+3.4
Qwen2.5-Coder-3B
Base
14.29
26.67
29.14
32.95
34.48
SFT
20.00
29.14
32.19
34.48
37.33
Table 5. Base and SFT coder performance on LiveCodeBench (Pass@1). Relative Change is computed from Base to SFT. Bold marks the higher Base/SFT score for each condition.
Coder
Setting
Direct
Planner
Qwen3.5-27B
DeepSeek-V4.1-Flash
DeepSeek-V4-Pro
GPT-5.6-Sol
Qwen2.5-Coder-1.5B
Base
15.21
25.19
25.65
25.96
23.35
SFT
21.20
27.65
29.19
31.18
30.26
Relative Change (%)
+39.4
+9.8
+13.8
+20.1
+29.6
Qwen2.5-Coder-3B
Base
20.28
28.11
27.96
29.95
30.57
SFT
25.81
29.49
31.49
31.64
34.10
Table 6. Base and SFT coder performance on Code2Bench (Pass@1). Relative Change (%) is computed from Base to SFT. Bold marks the higher Base/SFT score for each condition.
Figure 4. Plans and generated code from Qwen3-8B and Ministral-3-8B-Instruct for LiveCodeBench problem 3714, comparing direct generation with four planner-coder pairings. The problem, input constraints, two generated plans, and six code-generation outcomes: both direct generations fail, and only Qwen3-8B planning paired with Ministral-3-8B-Instruct coding passes.