How code helps different tasks? A decompositional lens on LLM post-training

Zheng Yu; Yiwei Li; Yishen Chen; Xiang Li; Jiale Han; Benyou Wang; Jingbang Chen

cs.AISep 27, 2026

How code helps different tasks? A decompositional lens on LLM post-training

Authors: Zheng Yu, Yiwei Li, Yishen Chen, Xiang Li, Jiale Han, Benyou Wang, Jingbang Chen

Organizations: Sun Yat-sen University · Shenzhen Loop Area Institute · The Chinese University of Hong Kong, Shenzhen · Shenzhen Research Institute of Big Data

Abstract

Evaluating code data as a single corpus can obscure which types of code data benefit which models and downstream tasks. Effective data selection requires understanding both the benefits of individual categories and whether these benefits persist when categories are combined. We introduce a decompositional lens for studying these effects in LLM post-training. We first decompose an execution-verified code corpus into interpretable categories based on the computational patterns of its solutions. Through controlled fine-tuning experiments, we compare individual categories with a balanced mixture across instruction-tuned models on question answering, mathematics, and code generation. The resulting response maps reveal recurring gains in average question-answering performance, while the same category can improve one model or task and degrade another. The best-performing category also varies with the starting model and target task. We then compose compact mixtures guided by these results and examine whether benefits observed in individual categories persist under joint training. On selected model--task pairs, mixtures whose constituents each improve the target task outperform both their best constituent and full-corpus training while using roughly 10--15% of the full corpus. These exploratory findings illustrate a \emph{less is more} pattern and highlight how the value of code data in post training depends on which categories are combined for which model and task.

Figures & tables

Figure 1: Overview of the category-specific transfer study. Ten computational categories define single-category training configurations and an example-matched balanced mixture across six instruction-tuned backbones and eleven task configurations. The exploratory screen retains only mixtures whose constituents exceed Base and whose joint model exceeds both its best constituent and Complete on the same task. The heatmap is schematic.

Category	Representative patterns	$\boldsymbol{n}$
Math / NT	Modular arith. Counting Transforms	2,075
Dynamic prog.	Recurrence Memoization Tables	3,562
Graph / stateful	Traversal Connectivity Paths States	2,775
Greedy / search	Greedy Binary/local search Rules	1,706
Impl. / utilities	Direct impl. Bookkeeping I/O glue	5,621
Hashing / sets	Hash maps Counting Sets Dedup	5,077

Table 1: Ten computational categories and their source-pool distribution. Labels follow the dominant computational pattern (Section 3.1 );

n

counts examples in each category before sampling. NT denotes number theory; impl. denotes implementation. Icons are reused in Table 4 and the response maps.

Figure 2: QA gains are consistent across models but uneven across tasks. Bars compare Base and Balanced mix (three-seed means, %) for the five-task QA mean and each task. QA means use equal weights and released one-decimal task scores. Each row uses the same numerical span across models, with panel-specific truncated baselines and absolute-score ticks.

Figure 3: Code and mathematics gains depend on the backbone. Bars average HumanEval, MBPP+, and MBPP-Simple (Code), or GSM8K, MATH-M, and MATH-H (Math), with equal weights. Reporting conventions follow Figure 2 .

Figure 4: Backbone-level category–task responses. All $6\times 10\times 11=660$ cells show three-seed mean changes from the corresponding Base (pp). Rows are categories; columns group QA, math, and code tasks. Comparing rows within a task reveals category preferences; comparing panels reveals their dependence on the backbone. Blue/red denotes gain/decline on a shared scale. Outlined numbers mark the largest positive and negative response per task within each backbone, retaining ties; these are descriptive extrema. Appendix B gives every value and the balanced-mixture rows.

Figure 5: Two qualifying compact mixtures on Qwen1.5. Panel (a) shows Math + Hash on ARC-C; panel (b) shows Imp + Hash + Sort on MBPP-Simple. Selected denotes the jointly trained mixture, and Complete denotes training on all 35,974 examples. Bars show the means on truncated y axes; yellow crowns mark the highest mean. Counts and percentages give training-set sizes and fractions of Complete. See Appendix C.6 for all screened examples.

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Recorded value
Source and labels	35,974 cleaned KodCode training examples; selection by primary category only.
Single-category configurations	1,706 examples each; sampling without replacement.
Balanced mixture	1,706 examples drawn from the single-category subsets: six quotas of 171 and four of 170.
Sampling seed	20260730; no auxiliary-label reselection.
Input and loss	Native model chat template; assistant-only loss.
LoRA	Rank 16, scaling parameter 32, dropout 0.05; attention query, key, value, and output projections.

Appendix

Table 2: Documented sampling and training configuration. Every trained arm has the same example count and update count; token totals can differ. Base receives no additional training.

Configuration	Scored $n$	Metric	Scoring mode	Max. tokens
ARC-Challenge	1,172	Accuracy	Option log probability	—
FinQA	1,133	Accuracy	Generated answer	128
HumanEval	164	Pass@1	Generated code	2,048
LegalBench	1,689	Accuracy	Option log probability	—
MATH-M	105	Accuracy	Generated answer	512
MBPP+	378	Pass@1	Generated code	2,048

Appendix

Table 3: Documented evaluation configuration. Counts are the effective scored problems for each configuration. A dash indicates choice scoring, with no generation limit applicable.

(a) QA (5 configurations)
Training recipe	Qwen2.5-7B	Qwen3-4B	Llama3.1-8B	Gemma2-9B	Llama3.2-3B	Qwen1.5-7B
Base	43.3	43.1	44.8	44.5	36.2	36.2
Balanced mix	46.2	45.4	45.4	46.7	37.7	36.9
Math / NT	46.2	45.9	45.2	47.0	37.2	36.9
Dynamic programming	46.0	45.4	45.4	46.7	37.6	36.4
Graph / stateful	45.9	44.9	45.3	46.8	37.5	36.5

Appendix

Table 4: Code post-training transfers across tasks, with uneven gains across models. Entries are three-seed mean task-group scores (%), averaged from the reported one-decimal task scores. All 60 single-category training configurations improve the QA mean (a), whereas only 24 improve the combined code-and-mathematics mean (b). Blue/red shading indicates higher / lower performance relative to the same backbone’s Base, using averages of the reported base-relative deltas. Panel (a) saturates at

+3.5

pp and panel (b) at

\pm 3

pp. Base rows are unshaded; the balanced mix matches each single-category configuration’s example budget.

Figure 6: Overall responses depend on both the backbone and the training category. Cells show the reported three-seed mean Overall change from the corresponding base checkpoint, in percentage points, for ten single-category training configurations and the balanced ten-category mixture. Blue denotes improvement and red denotes decline, with one shared scale. Category icons follow Table 1 . Values are reported to one decimal place.

Figure 7: Qwen2.5-7B-Instruct: complete task responses.

Figure 8: Qwen3-4B-Instruct-2507: complete task responses.

Figure 9: Llama-3.1-8B-Instruct: complete task responses.

Figure 10: Gemma-2-9B-Instruct: complete task responses.

Figure 11: Llama-3.2-3B-Instruct: complete task responses.

Figure 12: Qwen1.5-7B-Chat: complete task responses.

Arm	Examples	Updates	Composition
Base	0	0	Original checkpoint
Personalized Top3	5,118	320	Three model-specific categories, 1,706 each
Shared Top3	5,118	320	Impl./util., Hash/sets, Sort/order; 1,706 each
Balanced	5,118	320	Math and Seq: 511 each; other eight: 512 each
Proportional	5,118	320	Stratified sample preserving pool proportions
Complete	35,974	2,249	Full source pool, one epoch

Appendix

Table 5: Mixture training arms. Compact arms match examples and updates, but not token exposure. Complete uses the complete 35,974-example source pool.

Model	Base	Personal.	Shared	Balanced	Proport.	Complete
Qwen2.5-7B	55.782	58.571	59.165	58.825	59.035	56.781
Qwen3-4B	55.242	56.872	56.361	57.361	56.651	57.975
Qwen1.5-7B	37.145	37.347	38.407	38.244	38.066	37.667
Llama-3.1-8B	51.496	52.077	51.726	52.363	51.589	52.900
Llama-3.2-3B	44.898	43.971	44.264	44.209	43.775	41.373
Gemma-2-9B	53.165	53.988	53.400	53.438	53.660	50.935

Appendix

Table 6: All mixture Overall means (%). Overall equally weights eleven task percentages, with GSM8K at 256 tokens. Bold marks the largest trained-arm mean in each row; it does not denote statistical significance or necessarily an improvement over Base.

Model	Personal. $-$ Bal.	Personal. $-$ Prop.	Shared $-$ Bal.	Shared $-$ Prop.
Qwen2.5-7B	$-0.255$	$-0.464$	$+0.340$	$+0.130$
Qwen3-4B	$-0.489$	$+0.221$	$-1.000$	$-0.290$
Qwen1.5-7B	$-0.897$	$-0.720$	$+0.164$	$+0.341$
Llama-3.1-8B	$-0.286$	$+0.488$	$-0.636$	$+0.137$
Llama-3.2-3B	$-0.238$	$+0.196$	$+0.055$	$+0.489$
Gemma-2-9B	$+0.550$	$+0.328$	$-0.038$	$-0.260$

Appendix

Table 7: Top3 recipes versus equally sized controls (Overall, pp). Each contrast is reported at the source precision. All arms here use 5,118 examples and 320 updates. Tiny differences are descriptive; no seed-stability claim is made.

Model	Base, 256	Base, 1,024	Change (pp)	Positive arms: 256 $\to$ 1,024
Qwen2.5-7B	76.346	91.054	$+14.708$	$5\to 0$
Qwen3-4B	76.649	93.025	$+16.376$	$5\to 1$
Qwen1.5-7B	59.591	63.154	$+3.563$	$1\to 0$
Llama-3.1-8B	72.176	78.317	$+6.141$	$5\to 5$
Llama-3.2-3B	72.328	76.801	$+4.473$	$0\to 0$
Gemma-2-9B	84.685	85.444	$+0.758$	$0\to 0$

Appendix

Table 8: GSM8K Base sensitivity to generation budget. Positive-arm counts cover the five trained recipes per backbone.

Model	Personal.	Shared	Balanced	Proport.	Complete
Qwen2.5	$7.51\to-1.36$	$9.93\to-1.97$	$7.88\to-1.21$	$8.57\to-2.58$	$7.51\to-2.12$
Qwen3	$10.46\to-0.45$	$10.01\to 0.30$	$9.33\to-0.08$	$9.33\to-0.38$	$13.12\to-2.35$
Qwen1.5	$-2.12\to-5.61$	$-2.81\to-6.52$	$-2.81\to-6.97$	$-2.43\to-5.53$	$0.23\to-2.50$
Llama-3.1	$3.49\to 3.18$	$2.73\to 0.99$	$3.34\to 2.43$	$3.26\to 2.27$	$5.99\to 3.64$
Llama-3.2	$-2.73\to-5.08$	$-5.31\to-8.72$	$-6.29\to-6.60$	$-3.64\to-6.44$	$-23.12\to-22.37$
Gemma-2	$-2.27\to-1.67$	$-2.20\to-1.90$	$-2.73\to-2.05$	$-1.59\to-1.29$	$-2.05\to-1.74$

Appendix

Table 9: GSM8K changes from same-budget Base (pp). Each cell gives the 256-token change followed by the 1,024-token change. These are separately generated evaluations, not forced continuations.

Figure 13: Complete mixture task scores: Qwen backbones. All six original arms and eleven task configurations are shown as reported three-seed mean percentages, at the source plots’ one-decimal precision. Base rows are unshaded; other cells are blue/red for gains/declines relative to the same model’s Base. The shared red–blue scale matches Figure 12 and saturates beyond $\pm 12$ pp, as indicated by the colorbar extensions. Shading uses differences between the displayed one-decimal scores; cell labels remain absolute percentages. Shared combines Impl./util., Hash/sets, and Sort/order. The primary GSM8K cap is 256 tokens; MATH M/H denote the Medium/High configurations. Overall is reported separately in Table 6 .

Figure 14: Complete mixture task scores: Llama and Gemma backbones. Presentation, precision, and the $\pm 12$ pp color scale follow Figure 13 . Base rows remain unshaded. Base and Complete are retained alongside every compact recipe so that reduced degradation can be distinguished from improvement over the starting checkpoint. No seed-level error bars or significance tests are inferred from these means.

Backbone	Task	Recipe	Best single	Mixture	Complete	$\Delta$ Base
Qwen2.5	ARC-C	$\text{{\color[rgb]{0.4844,0.5781,0.7539}\faIcon{cogs}}\penalty\hskip 2.20001ptImp}\,+\,\text{{\color[rgb]{0.5586,0.3555,0.6523}\faIcon{font}}\penalty\hskip 2.20001ptStr}\,+\,\text{{\color[rgb]{0.3672,0.6133,0.5508}\faIcon{hashtag}}\penalty\hskip 2.20001ptHash}$	59.64	61.52	59.13	$+10.07$
	ARC-C	$\text{{\color[rgb]{0.4844,0.5781,0.7539}\faIcon{cogs}}\penalty\hskip 2.20001ptImp}\,+\,\text{{\color[rgb]{0.3672,0.6133,0.5508}\faIcon{hashtag}}\penalty\hskip 2.20001ptHash}\,+\,\text{{\color[rgb]{0.082,0.457,0.4141}\faIcon{sort-amount-down}}\penalty\hskip 2.20001ptSort}$	59.64	61.18	59.13	$+9.73$
	ScienceQA	$\text{{\color[rgb]{0.4844,0.5781,0.7539}\faIcon{cogs}}\penalty\hskip 2.20001ptImp}\,+\,\text{{\color[rgb]{0.5586,0.3555,0.6523}\faIcon{font}}\penalty\hskip 2.20001ptStr}\,+\,\text{{\color[rgb]{0.3672,0.6133,0.5508}\faIcon{hashtag}}\penalty\hskip 2.20001ptHash}$	70.59	71.45	70.64	$+3.42$
	ScienceQA	$\text{{\color[rgb]{0.4844,0.5781,0.7539}\faIcon{cogs}}\penalty\hskip 2.20001ptImp}\,+\,\text{{\color[rgb]{0.3672,0.6133,0.5508}\faIcon{hashtag}}\penalty\hskip 2.20001ptHash}\,+\,\text{{\color[rgb]{0.082,0.457,0.4141}\faIcon{sort-amount-down}}\penalty\hskip 2.20001ptSort}$	70.59	71.18	70.64	$+3.15$
	GSM8K †	$\text{{\color[rgb]{0.4844,0.5781,0.7539}\faIcon{cogs}}\penalty\hskip 2.20001ptImp}\,+\,\text{{\color[rgb]{0.3672,0.6133,0.5508}\faIcon{hashtag}}\penalty\hskip 2.20001ptHash}\,+\,\text{{\color[rgb]{0.082,0.457,0.4141}\faIcon{sort-amount-down}}\penalty\hskip 2.20001ptSort}$	84.08	86.28	83.85	$+9.93$
	MATH-H	$\text{{\color[rgb]{0.4844,0.5781,0.7539}\faIcon{cogs}}\penalty\hskip 2.20001ptImp}\,+\,\text{{\color[rgb]{0.3672,0.6133,0.5508}\faIcon{hashtag}}\penalty\hskip 2.20001ptHash}\,+\,\text{{\color[rgb]{0.082,0.457,0.4141}\faIcon{sort-amount-down}}\penalty\hskip 2.20001ptSort}$	30.15	34.73	25.19	$+3.44$

Appendix

Table 10: All 21 broader three-category passes. Best single is the highest constituent score. Scores are percentages;

\Delta

Base is in pp. Bold mixture scores mark the 18 passes for which every constituent also exceeds Base.

Backbone	Task	Recipe	Best single	Mixture	Complete	$\Delta$ Base
Qwen2.5	ARC-C	$\text{{\color[rgb]{0.4844,0.5781,0.7539}\faIcon{cogs}}\penalty\hskip 2.20001ptImp}\,+\,\text{{\color[rgb]{0.5586,0.3555,0.6523}\faIcon{font}}\penalty\hskip 2.20001ptStr}$	59.64	60.75	59.13	$+9.30$
	ScienceQA	$\text{{\color[rgb]{0.4844,0.5781,0.7539}\faIcon{cogs}}\penalty\hskip 2.20001ptImp}\,+\,\text{{\color[rgb]{0.5586,0.3555,0.6523}\faIcon{font}}\penalty\hskip 2.20001ptStr}$	70.59	70.77	70.64	$+2.74$
	GSM8K †	$\text{{\color[rgb]{0.4844,0.5781,0.7539}\faIcon{cogs}}\penalty\hskip 2.20001ptImp}\,+\,\text{{\color[rgb]{0.5586,0.3555,0.6523}\faIcon{font}}\penalty\hskip 2.20001ptStr}$	84.00	84.84	83.85	$+8.49$
Qwen3	LegalBench	$\text{{\color[rgb]{0.7891,0.4336,0.1211}\faIcon{exchange-alt}}\penalty\hskip 2.20001ptSeq}\,+\,\text{{\color[rgb]{0.3672,0.6133,0.5508}\faIcon{hashtag}}\penalty\hskip 2.20001ptHash}$	84.67	85.08	84.25	$+1.78$
	MBPP-Simple	$\text{{\color[rgb]{0.7891,0.4336,0.1211}\faIcon{exchange-alt}}\penalty\hskip 2.20001ptSeq}\,+\,\text{{\color[rgb]{0.3672,0.6133,0.5508}\faIcon{hashtag}}\penalty\hskip 2.20001ptHash}$	78.60	79.38	77.82	$-3.89$
Qwen1.5	ARC-C	$\text{{\color[rgb]{0.7891,0.4336,0.1211}\faIcon{calculator}}\penalty\hskip 2.20001ptMath}\,+\,\text{{\color[rgb]{0.3672,0.6133,0.5508}\faIcon{hashtag}}\penalty\hskip 2.20001ptHash}$	57.17	57.76	54.69	$+2.90$

Appendix

Table 11: All 11 broader two-category passes. Top2 retains the two leading categories in each backbone’s earlier Overall ranking. Bold mixture scores mark the nine passes for which both constituents exceed Base. Other columns follow Table 10 .

	ARC-Challenge		ScienceQA
Backbone	Imp $-$ Base	Imp $-$ Other nine	Imp $-$ Base	Imp $-$ Other nine
Qwen2.5-7B	$+8.19$	$+2.28$	$+2.56$	$+0.33$
Qwen3-4B	$+6.23$	$+0.84$	$+5.53$	$+1.51$
Llama-3.1-8B	$+1.79$	$+0.36$	$+1.21$	$-0.04$
Gemma-2-9B	$+3.92$	$+0.52$	$+2.07$	$+0.11$
Llama-3.2-3B	$+3.41$	$+0.08$	$+1.21$	$-0.08$

Appendix

Table 12: Single-seed Imp contrasts. Accuracy differences in pp for seed 20260730. The final row averages the six backbones; it is not a training-replication estimate.

Backbone / item	Question and Base $\rightarrow$ Imp answer	Interpretation and comparison
Llama-3.1 / ARC arc_00372	One hour at 80 km/h, then one hour at 100 km/h: $80\rightarrow 90$ km/h.	Corrected equal-duration average speed; all other nine arms and the balanced mixture are wrong.
Qwen2.5 / ARC arc_00564	A substance expands and then melts when heated: initial state liquid $\rightarrow$ solid.	Corrected initial-state inference; all other nine arms and the balanced mixture are wrong.
Llama-3.2 / ScienceQA scienceqa_02187	Given dominant $E$ , recessive $e$ , and genotype $ee$ : brown $\rightarrow$ red eyes.	Corrected rule application; all other nine arms and the balanced mixture are wrong.
Qwen2.5 / ARC arc_00821	Imp correctly selects that plants existed before coal and oil formed; Base is wrong.	Factual improvement; all other nine single-category training configurations are wrong.
Llama-3.1 / ScienceQA scienceqa_02144	Both journeys take five hours: 75 miles $\rightarrow$ 60 miles as the faster journey.	Relational regression: Base is correct and Imp is wrong.
Qwen2.5 / ScienceQA scienceqa_01144	Identify an object that is not a mineral: skull $\rightarrow$ quartz.	Classification regression: Base and all other nine arms are correct.

Appendix

Table 13: Selected improvements and counterexamples. Answers are paraphrased from saved predictions. R/F descriptions indicate the analyst’s interpretation. These illustrative cases do not establish a mechanism or a prevalence estimate.

Benchmark	R/F $n$	R gain	F gain	R $-$ F	95% interval
ARC-Challenge	26/41	$+0.28$	$+2.89$	$-2.61$	$[-4.76,-0.54]$
ScienceQA	20/15	$+0.65$	$+1.48$	$-0.83$	$[-3.95,+2.04]$

Appendix

Table 14: Exploratory question-type comparison. R/F gains are Imp minus the other-nine mean, in pp. The intervals apply to R

-

F, not to training-seed variability.

Skill group	$n$	Imp $-$ Base	Imp $-$ Other nine
Physical relations (e.g., speed, force, energy)	141	$-0.35$	$+0.64$
Experimental constraints	61	$-1.37$	$+0.82$
Genotype, phenotype, and dominance rules	105	$+2.38$	$+0.67$
Inherited/acquired trait evidence	113	$+1.33$	$+0.02$
Remaining natural-science questions	623	$+4.07$	$+0.52$

Appendix

Table 15: ScienceQA natural-science skill groups. Descriptive six-backbone mean changes in pp, using all questions in each indicated metadata group. “Remaining” questions are not a pure factual control.

Backbone	QA5	QA3	QA5	QA3
	Positive arms		Mean change (pp)
Qwen2.5-7B	10/10	10/10	$+2.79$	$+1.85$
Qwen3-4B	10/10	10/10	$+2.39$	$+0.77$
Llama-3.1-8B	10/10	5/10	$+0.55$	$+0.004$
Gemma-2-9B	10/10	10/10	$+2.26$	$+1.95$
Llama-3.2-3B	10/10	10/10	$+1.45$	$+0.86$

Appendix

Table 16: QA composition sensitivity in seed 20260730. Positive-arm counts and descriptive changes in pp. QA5 weights ARC-Challenge, FinQA, LegalBench, MedCalc, and ScienceQA equally; QA3 retains FinQA, LegalBench, and MedCalc. Changes average ten single-category training configurations per backbone.

Backbone	Mean $\Delta$ (pp)	Positive/negative	Base tokens	Trained tokens
Qwen2.5-7B	$+7.09$	10/0	193.3	148.7
Qwen3-4B	$+3.71$	10/0	181.5	168.6
Llama-3.1-8B	$+2.84$	10/0	177.1	170.6
Gemma-2-9B	$-2.24$	0/10	100.0	89.0
Llama-3.2-3B	$-2.27$	0/10	175.6	160.9
Qwen1.5-7B	$-5.91$	0/10	169.4	113.7

Appendix

Table 17: GSM8K score and length changes. Seed 20260730;

n=1{,}319

questions per arm. Trained scores and lengths average ten category arms, not repeated training runs. Visible tokens are obtained by retokenizing saved text.

Backbone	CW	WC	Assertion	Helper	Indentation	Other
Qwen3-4B	154	36	131	10	0	13
Gemma-2-9B	85	129	53	5	27	0

Appendix

Table 18: HumanEval transitions and recorded failure types. Seed 20260730; sums over ten categories, each evaluated on 164 questions. The last four columns partition CW comparisons. Prompt-helper and indentation entries are compatibility diagnoses, not verified recoverable scores.

Backbone / arm / item	Observed change and interpretation
Qwen1.5 / String gsm8k_00972	Base confuses 40 oranges kept with the number sold; the trained response correctly gives 120 sold. Both answers are complete, and the correct response is shorter (117 $\rightarrow$ 105 visible tokens).
Gemma-2 / Sorting gsm8k_01070	The trained response halves the quantity of discounted butter twice, changing the correct answer 18 to 15. The response becomes longer (87 $\rightarrow$ 112 tokens).
Qwen3 / Imp HumanEval/26	Base removes every value occurring more than once; the trained code instead retains one copy of each value. This changes required semantics. The same regression occurs in all ten categories and the mixture on this item.
Gemma-2 / Sequence HumanEval/159	A shorter program correctly updates remaining inventory by subtracting the available amount sold. This is a successful relation update rather than a length-induced failure.
Gemma-2 / Sorting HumanEval/50	The decoding formula is retained, but the prompt-provided helper is absent from the assembled test program. The recorded regression is compatible with a wrapper failure.

Appendix

Table 19: Examples of solution and interface changes. Seed 20260730; analyst interpretations of saved outputs. Category names identify the inspected arm and do not imply that the change is unique to that category.

Explore similar work

May 27, 2026cs.AI

Efficient Post-training of LLMs for Code Generation With Offline Reinforcement Learning

Post-training using online reinforcement learning (RL) is an important training step for LLMs, including code-generating models. However, online RL for code generation involves LLM inference and verification of the generated output, which can take considerable time and resources. In this paper, we explore the application of offline RL to code-generating models by leveraging existing code datasets. Our experiments demonstrate that offline RL is an effective training strategy for improving LLM performance. We show that offline RL can be especially beneficial for small LLMs and challenging coding problems.

Mingze Wu, Abhinav Anand, Shweta Verma +1

Technische Universitat, Darmstadt, Germany · Hessian Center for Artificial Intelligence, Darmstadt, Germany · National Research Center for Applied Cybersecurity ATHENE

Code Generation Large Language Model Training Post-Training LLM Inference Optimization Offline Reinforcement Learning

Ask PDF

Sep 14, 2026cs.LG

Performance, Efficiency and Collapse -- Advantages and Challenges in Offline Post-training of Code LLMs

Post-training with reinforcement learning (RL) is a critical phase in the development of code-generating large language models (LLMs), as it ensures adherence to instructions and the production of functionally correct code. This process typically requires computationally intensive code sample generation from Transformer-based LLMs and substantial GPU-CPU communication for sequence verification. To address these computational challenges, this work examines whether RL-based post-training can be performed entirely offline by leveraging existing datasets rather than generating new samples. The findings indicate that, with only a few hours of training, zero-shot code generation performance of LLMs can be substantially improved without online sampling. Additionally, offline RL produces performance gains across models ranging from 0.5B to 7B parameters, although the extent of improvement varies among model families.

Abhinav Anand, Sanjana Reddy Pachika, Shweta Verma +1

TU Darmstadt · Hessian Center for Artificial Intelligence, Darmstadt, Germany · National Research Center for Applied Cybersecurity ATHENE

Reinforcement Learning Post-Training Large Language Model Training Post-Training Collapse Challenges Self-Verification

Ask PDF

Jun 16, 2026cs.AI

From Brewing to Resolution: Tracing the Internal Lifecycle of Code Reasoning in LLMs

Standard accuracy metrics cannot explain why LLMs handle variable tracking but fail on semantically equivalent loops. We study an internal lifecycle of code reasoning in which models first brew the answer, making it linearly recoverable many layers before it becomes self-decodable, and then diverge into one of four resolution outcomes: Resolved, Overprocessed, Misresolved, or Unresolved. Understanding this lifecycle matters because similar task accuracies can mask fundamentally different failure modes that surface-level evaluation cannot detect. We introduce a dual diagnostic framework pairing layer-wise linear probing with Context-Stripped Decoding (CSD) and apply it to six code-reasoning task families across 16 models spanning Qwen, Llama, and DeepSeek architectures. All four outcomes carry substantial mass in every task family: overall Resolved is only 41.5%, with multiple tasks below 30%. Controlled sweeps over structure, depth, and operators expose task-specific failure bottlenecks: Function Call Resolved plunges from 61.1% to 2.5% as call depth increases from one to three. Across architectures and scales, the brewing scaffold remains stable, with normalized brewing duration 24-42% across all 16 models, while resolution success varies with capability. This indicates that the scaffold is a stable empirical regularity across the tested decoder-only Transformer families, whereas resolution success covaries with capability, scale, and training. Code: https://github.com/euyis1019/llm-brewing

Siyue Chen, Yifu Guo, Yuquan Lu +9

1South China University of Technology · 2Sun Yat-sen University · 4Shanghai Jiao Tong University +5

LLM Reasoning Strategies Reasoning Skills Bottlenecks Lifecycle

Ask PDF